Challenges in Big Data Analytics

Organizations from pharmaceutical manufacturers to health insurance companies to hospitals are looking to share, aggregate and connect disparate sources of healthcare data. In doing so, they hope to gain valuable insights and uncover opportunities to treat everything from obesity to dengue fever to finding a cure for cancer. Leveraging healthcare data for secondary uses is transforming the industry but there are numerous challenges to implementing Big Data Analytics (BDA).

While there are some mature players in the industry, many small organizations are unable to absorb the cost of advanced analytics solutions. Even large organizations with the funds to implement the technology may face cultural challenges. Too many do not encourage information sharing, lack executive sponsorship for analytics programs, or lack skilled employees for this work. While these challenges pose real barriers to the wider use of BDA, this section will focus on privacy and security challenges with respect to big data.

Privacy and Security in Healthcare

There has been a steady increase in the number of data breaches affecting healthcare organizations. Of significant concern are negligent and careless employees. There have been many stories of individuals that have lost devices containing unsecured PHI or who have stolen medical records themselves. Malicious insiders are just one reason that criminal attacks on healthcare organizations are up 125% over the past five years.

The digitization of healthcare has undoubtedly made this data a more attractive target for external hackers and criminals as well. The Affordable Care Act, as an example, has increased enrollments in private health insurance and Medicaid by more than 17 million individuals, enlarging the pool of records and providing individuals with nefarious intentions a bigger ‘prize’. Unlike credit cards which are easily cancelled and replaced, medical records include information that is not easily destroyed. Consequently, medical information is worth 10 times more than a credit card number on the black market.

IT departments in healthcare organizations are also facing challenges in protecting patients’ personal information. Hospitals and insurers often rely on legacy systems, another attractive target for hackers. These systems can contain old security vulnerabilities that, left unpatched, make the network easy to hack. Hospitals are also facing a compliance challenge from the volume of systems and data increasingly outside of IT’s control. This includes applications and devices that have been purchased without IT’s involvement, community physicians with their own smartphones, and patients’ mobile devices being brought into the facility. EMC2 estimates that 93% of the digital healthcare universe needs protection and that more than 40% of this data is not being adequately protected.

Security breaches not only present a risk to patient privacy, they also put a company at risk of running afoul of state and federal privacy legislation. In the event of a privacy complaint or breach, failure to demonstrate compliance with HIPAA and HITECH can result in severe penalties. The largest reported fine issued by the Health and Human Services Office of Civil Rights for a HIPAA violation was $4.8 million in May 2014. Since then, we have seen the largest health data breach yet when hackers compromised almost 80 million records at health insurer Anthem Inc. The result of this investigation and any subsequent penalties are still to be determined.

The ability to integrate unstructured data with sources of structured data allows a more detailed picture to be created of the patient journey — the progress of disease, the adherence and response to treatments and subsequent patient outcomes. As more data sources are integrated, however, those who are responsible for PHI need to continually re-evaluate patient identifiability. It is imperative that the risk of re-identification remains low to protect patients and to ensure regulatory compliance. Data anonymization prior to analytics is a valuable tool to achieve both.


Challenges in Big Data Analytics is the third in the Big Data Analytics Series by Privacy Analytics. Next: Dealing with Data Variety in Healthcare.



Free Webinar: De-Identification 101

Join Privacy Analytics for a high level introduction of de-identification and data masking.
Watch now

Free Download: De-Id 101

You have Successfully Subscribed!