Originally published in KDnuggets, July, 2019
The Era of Big Data is coming to an end as the focus shifts from how we collect data to processing that data in real-time. Big Data is now a business asset supporting the next eras of multi-cloud support, machine learning, and real-time analytics.
The Era of Big Data passed away on June 5, 2019, with the announcement of Tom Reilly’s upcoming resignation from Cloudera and subsequent market capitalization drop. Coupled with MapR’s recent announcement intending to shut down in late June, which will be dependent on whether MapR can find a buyer to continue operations, June of 2019 accentuated that the initial Era of Hadoop-driven Big Data has come to an end. Big Data will be remembered for its role in enabling the beginning of social media dominance, its role in fundamentally changing the mindset of enterprises in working with multiple orders of magnitude increases in data volume, and in clarifying the value of analytic data, data quality, and data governance for the ongoing valuation of data as an enterprise asset.
As I give a eulogy of sorts to the Era of Big Data, I do want to emphasize that Big Data technologies are not actually “dead,” but that the initial generation of Hadoop-based Big Data has reached a point of maturity where its role in enterprise data is established. Big Data is no longer part of the breathless hype cycle of infinite growth but is now an established technology.
Editor’s note: See also Google Trends for Big Data and Hadoop
When the Era of Big Data started with the launch of Apache Hadoop in 2006, developers and architects saw this tool as an enabler to process and store multi-structured and semi-structured data. The fundamental shift in thinking of enterprise data beyond traditional enterprise database assumptions of ACID (atomicity, consistency, isolation, and durability), led to a transformation of data use cases as companies realized that data previously thrown away or kept in static archives could actually provide value to understanding customer behavior, propensity to take action, risk factors, and complex organizational, environmental, and business behaviors. The commercial value of Hadoop started to be established in 2009 with the launch of Cloudera as a commercial distribution, which was quickly followed by MapR, Hortonworks, and EMC Greenplum (now Pivotal HD). Although analysts provided heady projections of Big Data as a potential market of $50 billion or more, Hadoop ended up being challenged through the 2010s as an analytic tool.
To continue reading this article, click here.
About the Author
Hyoun Park is the CEO and Founder of Amalgam Insights, a firm focused on the technology, analytics, and financial tools needed to support emerging business models. Over the past 20+ years, Park has been at the forefront of trends such as Moneyball, social networking, Bring Your Own Device, the Subscription Economy, and video as the dominant use of Internet bandwidth. Park has been quoted in USA Today, the Los Angeles Times, and a wide variety of mainstream and technology press sources.
Prior to becoming a technology observer and consultant, Park worked at Bose in managing both the IT project portfolio and the multi-million dollar network, telecom, and mobility spend. Prior to Bose, Park worked at Teradyne managing telecom, mobility, and conferencing accounts. Park has also worked at multiple venture-capital backed telecom startups, served as the Treasurer for a Boston city councilor, and worked on the first fantasy baseball website ever. Although he works with enterprise technologies, Park has a soft spot in his heart for new and innovative technology startups and is dedicated to supporting greater gender and cultural diversity in science, technology, engineering, and mathematics.
Park has an MBA from Boston University and a bachelor’s degree in Women’s and Gender Studies from Amherst College.
 |
 |
 |
