Octopai's Gal Ziton
By 2025 it is forecasted that the world will generate 463 exabytes of data each day, much coming from the estimated 75 billion IoT devices expected on the market. With data increasingly the primary driver for business growth and decision-making, it is essential that industrial enterprises know how to leverage the data that they have.
Given the complex and intricate digital state in which we operate, leveraging data correctly can power any company to make good decisions, drive revenue, and influence success. The industrial-manufacturing space, in particular, is dependent on data for many day-to day tasks—onboarding new employees, operating and locating heavy machinery, reporting and tracking information to keep operations smooth. Accessing data assets within moments can be a game-changer for expanding company operations and creating better working procedures and internal processes.
An example of this can be seen in a recent study that demonstrated that manufacturing productivity in Indiana is falling behind largely because the industry is too slow to adopt new technologies, which is causing a decrease in productivity. Companies that adopt innovation and understand the importance of doing such will experience exponential growth and are charting a path of efficiency and productivity. We all know this, but it can be tricky to actualize.
One issue that commonly frustrates enterprises is achieving a full view of their data assets in order to maximize the powerful business value that data provides to an organization. In order to do so, business intelligence and analytics teams are using data lineage as a means of tracing the origins of data within an enterprise. While, traditionally, data lineage has taken the form of a one-dimensional view of column-to-column information, it is recommended to use a multi-dimensional approach to data lineage since data today is dynamic and no longer one dimensional.
A multidimensional approach to data lineage demonstrates both the horizontal and vertical movement of data while enabling a complete map of the data lifecycle—showing where it originated and how it flows within the ever-changing data landscape.
Automated data lineage with a multi-layered, multi-dimensional approach to the data journey makes it is easier for data analysts to trace the movement of data, enabling them to track errors, implement system migrations, create BI processes and procedures, and improve overall BI efficiency to generate the maximum value for the company.
There are three layers to data lineage that make it possible to properly analyze and understand the data:
-
Cross-system lineage which provides end-to-end lineage at the system level from the entry point into the BI landscape, all the way to reporting and analytics
-
End-to-end column lineage, which provides column-to-column-level lineage between systems from the entry point into the BI landscape, all the way through to reporting and analytics
-
Inner-system lineage, which provides micro level insight into column lineage within an ETL, report or database object
All three of these layers are crucial to providing robust insight into the data flow of an organization so that data can be leveraged to its fullest extent, fueling the energy lines of the company. The multidimensional approach provides data users with access to relevant data, when they need it most.
Taking a multi-dimensional approach to metadata management guarantees a full and clear map of all data moving through the company to make smarter and faster decisions that are foundational to company progress.
Gal Ziton is vice president and co-founder of Octopai
