Authored by: Colin Brookes
With legislative and business pressures for interoperability heating up this year, payers are searching for the right data integration solution. Though Apache NiFi has been available for about five years, it’s still a virtually unknown data integration platform in the healthcare industry. HealthLX uses the NiFi open-source tool as the foundation for our core platform, and we believe it’s worth a closer look for solving modern large-volume and large message-size healthcare data flow challenges.
NIFI’S BIG DATA BENEFITS
NiFi grew out of the NSA’s NiagraFalls data flow platform and was released as an open-source system in 2014. NiFi takes a blackbox approach similar to other integration platforms, but here’s where it’s different: Boxes in NiFi are loosely coupled, allowing users to separate their data flow and put each step on a dedicated server. By optimizing NiFi in this manner, organizations can effectively manage their high load requirements, a feature that seems tailor-made for handling high volumes of healthcare data at scale.
What else do we like about NiFi for healthcare data integration?
10X faster integration – NiFi’s building blocks are process-oriented rather than detail-oriented, enabling a higher-level, process-based integration that is remarkably faster than other platforms without any loss of functionality.
Data provenance – NiFi delivers an unprecedented amount of data lineage, which is becoming ever more essential in healthcare. NiFi gives up to 100X more provenance data than the original data.
HL7 processes – quite a few standard HL7 processes are available right out of the box with NiFi, which users can build into their own system in a matter of minutes.
Extensibility – users can rapidly develop and test extended code functionality that supports their organization’s specific business processes and goals. This same extensibility can be built into product suites like HealthLX that rely on NiFi for core workflows.
Reliability – NiFi persists everything to disk, so there’s no risk of losing data or transactions, which is an essential capability for the healthcare industry.
Security – NiFi has a mature security model, and gives organizations fine-grained control over viewing and modification access.
It’s free – NiFi is open-source technology that’s actively developed and well-supported by a large open community. It is a key component of Cloudera’s (Hortonworks) Hadoop large-scale data movement platform.
NiFi may be a well-kept secret for now, but it’s likely to see rapid, widespread adoption as more health organizations come to understand its distinct value in big data processing and distribution. If you’d like to learn more about moving interoperability forward in your organization, please reach out. We’d love to start the conversation with you.