What is Big Data?
Big data has become the latest buzzword within the IT community, revolutionizing business intelligence capabilities. But what is it really and how will it impact the project management community? With all the current hype around big data, many are eager to jump onto the bandwagon and treat all projects as big data projects. But remember, having large amounts of data does not necessarily require big data technologies.
Not all projects are big data projects.
What is the Project Manager's Role in Understanding Big Data?
Thus, Project managers must be capable of advising management not only about how, but also when the business can and should take advantage of big data technologies.
Big Data: the 3 V's
Big data in simplistic terms is often thought of as large volumes of data. But organizations have been managing large-scale data for years, long before these new technologies emerged on the scene. Excessively large amounts of data do not necessarily mean big data.
When thinking about big data, it’s not just about the large volume. Big data can be thought of in terms of 3 Vs: volume, velocity and variety. And some expand the definition further to include veracity and value as well.
The rate of change of the data is just as important as the large volume and must be taken into account. For large but static data sets that don’t change very often, analysis that takes days to complete may be acceptable. However, if the data is changing at a high rate of speed, faster processing time becomes much more critical.
It’s also important to understand the variety of data, which consists of unstructured and semi-structured data (such as posts to social media sites, digital pictures and videos, purchase transaction records, health records, emails, and cell phone GPS signals, XML, etc.). A typical RDBMS is capable of storing structured data. However, unstructured and semi-structured data needs a different storage platform. Enterprises must be able to handle the multi-structured data comprised of various data formats and types derived from web applications and social networks. The meaning and real value of big data extends beyond the ability to handle just large volumes of unstructured data and web log files.
Big Data - It's All of Your Data
In reality, big data is ALL of your data and the ability to sufficiently use it to discover patterns and “insights” to drive innovation.
Dealing with a variety and volumes of rapidly changing structured and unstructured data increases the complexity of storing, processing and analyzing the data. The challenge with big data lies in integrating and managing the diverse data sources. How do you find the valuable information buried in all that data and then make sense of all of it? Thus, the other challenge is filtering out pieces of useful information to derive knowledge and gain a competitive advantage for your business.
About Patti Gilchrist
Patti Gilchrist is a Sr. Technical Manager with 25 years experience implementing strategic enterprise initiatives.
Patti has a reputation for effectively translating business problems into innovative solutions and creating strategic roadmaps to achieve business goals.
Visit Patti Gilchrist's Other Sites
Uncovering Big Data, ProjectManagement.com
What is Big Data?, Forbes
Big Data, Webopedia
What is Big Data?, Strata O'Reilly
Big Data, Intel
Why Big Data is a Big Deal, Harvard Managazine
Big Data, PC Mag