What's the Big Deal About Big Data?

My career has allowed me to learn quite a few things.  I have worked with software, testing it to ensure it met the requirement needs of clients. I have worked at a small cell phone company, trying to create a small and primitive data warehouse.  Now I work assisting in gaining adherence to ITIL methods for the entire government of a country.  These have all been great positions, but I have to say my first job out of college was by far my favourite.

At AIG, I was able to work with some very brilliant minds on the executive team, engage on some fraud reporting and saw my reports used throughout the United States.  I had a great manager, and awesome co-workers   However, by far the best part was working with data, reporting and writing SQL.  I loved it.  If you go back into my past a little further, you would see me asking my professor, after my class was finished for the semester, could I retain access to the database so that I could practice SQL more (can you say NERD!).

Nowadays my use of SQL is very limited and I miss it.  I miss reporting, seeing trends in data, and creating tools that guide decision making that can change the outcomes of the business and how everything interacts to create success.  So you should not be surprised that this term I have been hearing brings about a level of excitement for me.  It is Big Data.

OK, so I am sure you have heard of big data.  Maybe you know exactly what it is.  I am hoping to shed some light on this for people that have heard what it is, but have not had the time to look into it further.  Also I would like to convey why it will be important to healthcare.

What is Big Data?

When I was at AIG we had a data warehouse that had, most likely, millions and millions of records.  It was massive.  Big data is even bigger than that.  According to Wikipedia, big data is data sets so large that using standard database tools or data processing becomes very difficult.  The reason data is growing so much is because it is “increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing), software logs, cameras, microphones, radio-frequency identification readers, and wireless sensor networks."  Wikipedia goes on to say that, "The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; as of 2012, every day 2.5 quintillion (2.5×1018) bytes of data is created."

Wow, that is big!

This data is being used for data analytics, business intelligence and effective decision making.  Cliff Saran of ComputerWeekly.com alludes to a fact.  The bigger your data, the larger the gap grows in competitiveness that lies between you and your competitors.

With these advantages, come some challenges.  The biggest, ironically, is the size of the data.  Saran says that "big data will cause traditional practices to fail, no matter how aggressively information managers address dimensions beyond volume."  Another challenge is trying to understand how to use unstructured formats, such as text and video (McDonnell, 2011).  According to McDonnell, some other challenges include storage of this data and getting the most important data to the right people at the right time.  And of course there are going to be immense challenges when it comes to security and privacy.

Big Data in Healthcare

Is big data important to healthcare?  Irfan Khan (2012) says that if we estimate that each of our cells contains 1.5 GB of data, each one of us is walking around with approximately 150 zettabytes of data.  Thanks 150 billion terabytes.  So, it seems, big data is already in health care.  We just need to capture this data. By capturing this data there is a great potential for improved healthcare "including personalization of care, defining patient populations with a greater level of granularity, analysing unstructured data, mining claims data for insights that can improve wellness and patient compliance, advancing medical research, and helping governmental agencies detect fraud, identify best care delivery practices, and improve bio-surveillance (Terry, 2012)."

Big data has big possibilities, including better healthcare outcomes.  It also has big challenges, such as storage, privacy and how to work with new and unstructured data, to name a few.  The benefits outweigh the challenges, however.  mHealth, EHR and all of the technologies related to health informatics will continue to grow big data.  The big question is, are you ready?

----

Khan, I.  (2012 November 15).  Where's the big data in healthcare IT? Look in the mirror.  Retrieved from http://www.itworld.com/big-data/315298/where-s-big-data-healthcare-it-look-mirror

McDonnell, S.  (2011 June 21).  Big Data Challenges and Opportunities.  Retrieved from http://spotfire.tibco.com/blog/?p=6793

Saran, C. (n.d.).  What is big data and how can it be used to gain competitive advantage?  Retrieved from http://www.computerweekly.com/feature/What-is-big-data-and-how-can-it-be-used-to-gain-competitive-advantage

Terry, K.  (2012 October 15). Health IT Execs Urged To Promote Big Data.  Retrieved from  http://www.informationweek.com/healthcare/clinical-systems/health-it-execs-urged-to-promote-big-dat/240009034

Wikipedia.  (2012 December 23).  Big Data.  Retrieved from http://en.wikipedia.org/wiki/Big_data