Decentralizing data while making it “less dumb”

June 1, 2022
A centralized, top-down approach to data management and storage is a choke point.

Data, the theory goes, leads to better intelligence which, in turn, leads to better business decisions.

What to sell and to whom? How to build product most efficiently? When and in what quantities?

Each of these tasks is difficult; reaching the optimal outcome across all of them is almost impossible without relevant, timely information. 

The path to such information, however, is complex, time-consuming, expensive, and not always successful. Before the organization can even start analyzing its data, it generally must be assembled by importing, scrubbing and normalizing from multiple disparate sources, in different formats, using different data models, etc.

Current data-management systems and approaches assume that data is “dumb” and that data must be managed using sophisticated and complicated infrastructures. Organizations spend millions of dollars building and implementing fragile data hierarchies with multiple point-to-point connections. Data is stored in centralized data warehouses and lakes spanning large server farms.

This centralized, top-down approach to data management and storage is a choke point. Data is often stored far from where it’s needed. Analysis and decisions initiated far from the point of action do not guarantee the best outcomes. The centralized, top-down approach to data management does not scale, at least not economically, and the largest companies are finding it ultimately breaks down under the increasing flood of data

Web 3, a new iteration of the internet based on blockchain technology incorporating concepts such as decentralization and token-based economics, has underlying technology principles that are relevant to solving this problem, regardless of its more ambitious social aspects. Web 3’s core principles, from a technological standpoint, can be defined as: 

  • Process locally; there’s more than enough processing power
  • Store locally; there’s more than enough resources
  • Use fast local connections and networks
  • Operate peer-to-peer

Self-regulating operating and autonomous facilities and machinery should operate on application processing, data storage and analysis that takes place locally. Edge computing is a great example of distributed / decentralized “Web 3” computing.

Let’s look at cybersecurity (specifically data security) as an example of centralized vs. decentralized thinking.  

If data is the lifeblood of organizations, then it stands to reason that data is the asset most often targeted by cyber-criminals. Data-security challenges are magnified for manufacturing organizations—in fact any organizations with dispersed facilities. Edge analytics and operations management, for example, undoubtedly offer operational, safety and efficiency benefits, but also mean many more potential points of cybersecurity failure.

Cybersecurity defenses today are focused on keeping the bad actors out. Organizations deploy sophisticated defenses protecting infrastructure, hardware, databases, applications, etc. Cybersecurity defenses rely on people, and people make mistakes. No matter how strong your defenses may be, you must assume you’re going to get hacked; when you do, centrally stored data is a liability.

As demonstrated time and again, a successful ransomware attack can lead to catastrophic data losses and associated operational, reputational and financial damages (e.g., the Colonial Pipeline attack). 

Centralized data storage and security is like a large castle, with an impregnable wall surrounded by deep moats filled with tripwires and alligators. But people need to get in and out, so the wall has a thousand doors. It’s relatively easy for an attacker to get through a door that someone forgot to lock. Once inside the castle, the attacker finds a pile of gold coins unprotected behind the wall, easy to sweep up and steal. 

In the decentralized world, data is dispersed and there is no unprotected pile of gold coins. Instead, the attacker finds a series of locked rooms, each containing a single gold coin. Losses can be minimized.

The “holy grail” for organizations is the concept of “intelligent” data, where each data record interacts dynamically with a set of protections and rules that provide security, access and usage controls at the most granular level possible. Intelligent data can be stored where needed, accessed when needed and presented in the necessary format.

A fantasy? Hardly. The technology exists today, but organizations must counter years of traditional “centralized” thinking!

By Andrew Hopkins, president of PrivacyChain