- Conduct appropriate planning.
- Identify the business problem to drive the exploration and analysis of it and determine potential business
value before staring your project.
“There should be a compelling use case, a competitive driver, cost driver, or some other issue that has been identified where the application of big data technologies is in the critical path to solving the problem (Selecting Your First Big Data Project).”
If you do not have a clear and concise definition of your expectations before you start, you should not be doing a big data project.
- Ensure engagement of business.
- Ensure commitment of sponsors.
- Define appropriate scope and objectives and develop an appropriate use case that will impact the business.
- Have a clear idea of who your user is.
“It is imperative to sit down with key people for each project to understand what their line of work does, how it interacts with the rest of the company and what its challenges are, says Ron Kasabian, general manager of Intel’s big data solutions, data center and connected systems group. By asking these questions, organization can identify actionable areas for improvement via big data, rather than spinning its wheels on interesting, but ultimately low-impact projects (Building Off Best Practices).”
- Make value to the customer a priority. Identify and prioritize data sources. Then connect the data to the needs and desires of your customers (How to Successfully Implement a Big Data Project in 8 Steps)."
- Adopt culture of data-driven decision making. Link customer data to company process. Each new data set provides an opportunity to change the way you deliver to your customers (How to Successfully Implement a Big Data Project in 8 Steps)."
- Provide access to the data.
“Companies with a culture of evidence-based decision making ensure that all decision makers have performance data at their fingertips every day (You May Not Need Big Data After All).”
- Focus on the effective use of data. The solution of a big data project will only be as
good as the data. Not all data is good data. And some data will be more useful than other data.
- Ensure veracity of the data.
Start by defining "one version of the truth" in the data. Establish one undisputed source of data.
- Focus on finding the useful data.
And some data will be more useful than other data.
Tamara Dull of the best practices team at SAS advises “Keep in mind that you don’t always need big data; you just need the right data. Having the “right data” which is relevant and accurate is critical. You must know how to separate the vital data from the rest of the data (Building Off Best Practices).”
- Effectively use the data.
Generate insights and make strategic decisions from these observations that will meaningfully effect the businesses (Do You Really Need Big Data).”
- Ensure veracity of the data.
- Technology leaders must agree to the overall feasibility of the project.
- Select the people before the technology.
“This may seem a little counter intuitive, but we've learned that selecting the people who are sponsoring and staffing the project is actually a more important predictor of success than the technology.
While we spend time up front making sure that we get the technology right (and this is where having a broad portfolio of technology options is a great help), I’ll know going in what team members from our side I am going to assign to the project. You should as well, since in my experience personnel selection is the biggest variable in the project’s success – even, or maybe especially, when dealing in emerging technologies.
Simply stated, it is not a best practice to put people in the critical path of your first project if they are overly vested in existing approaches and technology, or downright hostile toward new technologies.
Make sure you staff with resources who expect the project will be doing things differently than the way you currently do them, especially in comparison to how relational database and warehouse projects work.
Make sure the people involved understand that there will be different outcomes as a result of using this new technology. Your people need to understand that both the outcomes and how the data are exercised to get to those outcomes will differ from what they have done before (Selecting Your First Big Data Project).”
- Ensure resources and vendors have the adequate skill sets required to manage the complex technology.
- Identify success criteria and have a clear measurement of success.
- Choose the right methodology.
An iterative approach such is Agile is best suited for this type of project, since you will need to break down the analytics into smaller components. Smaller and more focused implementations offer many benefits. They are easier to manage, allow clients to see results more quickly, make it easier to handle issues, thus reducing risk, and offer flexibility to deal with changing needs, requirements, and data. Remember, big data is changing rapidly.
Management Best Practices for Big Data
The Kimball Group published a well-tested set of best practices for relationally-based enterprise data warehouses (EDWs) that big data efforts should leverage:
- Drive the choice of data sources feeding the EDW from business needs
- Focus incessantly on user interface simplicity and performance
The following are EDW best practices relevant to big data:
- Think dimensionally: divide the world into dimensions and facts
- Integrate separate data sources with conformed dimensions
- Track time variance with slowly changing dimensions (SCDs)
- Anchor all dimensions with durable surrogate keys
Architecture Best Practices for Big Data
Data Modeling Best Practices for Big Data
Data Governance Best Practices for Big Data
About Patti Gilchrist
Patti Gilchrist is a Sr. Technical Manager with 25 years experience implementing strategic enterprise initiatives.
Patti has a reputation for effectively translating business problems into innovative solutions and creating strategic roadmaps to achieve business goals.