by Rick Skriletz –
Businesses are inundated with data but have less success turning it into information quickly. To become data-driven requires managing enterprise data as an asset and reducing the time needed to collect and analyze it so it approaches zero.
You Must Deal with the Six Degrees of Separation that Keep You from Data
“In this world of so-called digital transformation and cloud computing that drives our always-on über-connected lifestyles, surely it would be useful to understand the what, when, where and why of data on our journey to then starting to appreciate the how factor.” (From Forbes’ The 13 Types of Data1)
Companies are working to become data driven. Being data-driven means2 operational actions are “determined by or dependent on the collection or analysis of data.” The challenge is the dependency on “collection or analysis of data” – it becomes a time-consuming effort to get data needed and analyze it. As the business world rapidly becomes real time, the time available for data collection and analysis continues to shrink.
In a digital business, the value of data correlates with the speed with which it is put to use. Making data into a business asset requires rethinking enterprise data and working to get value from it. Being data-driven means data needs must be identified and data collected, analyzed, utilized and, to meet the demands of digital business, processed in real time.
What Is Meant by ‘Enterprise’ Data?
At a simplistic level, enterprise data is all the data in an enterprise. After all, data is used in applications to keep a business operating. While this is true, enterprise data is3 often thought of as “data that is shared by the users of an organization, generally across departments and/or geographic regions.” Shared data makes enterprise data different than application and user data but doesn’t address the need for enterprise data to be correct and consistent everywhere it is used in a digital, real-time business.
The path to becoming data-driven requires working through the accumulation of data practices, including data marts, enterprise data warehouses, data virtualization, Big Data environments, analytic platforms, data fabrics, data meshes, and more. These trace the evolution of data in business and, while each approach has a kernel of validity, that evolution has created a hodge-podge of legacy data “solutions” that continue to coexist in companies today.
It is necessary to consider these technical approaches as we look at the evolution of data and its use, and what is needed to overcome factors that separate data from its use.
7 Steps to Data-Driven Success
Transforming an organization to be data-driven requires changes to the way people use data in their work, and operational processes, by introducing digital processes and practices. Nowhere is this more important than how a company handles its data, including in its applications. At its core, it requires changing application-centric business operations to data-driven ones.
Six Degrees of (Data) Separation
The six degrees of separation here covers the distance between you and data you need, much like the idea that everyone is six or fewer social connections away from each other. In this case, a degree of separation represents a use of data that has a magnitude of effort to collect and analyze it, with Degree 1 requiring the least effort and Degree 6 the most. The six degrees of separation cover the distance between you and your data.
The purpose of reviewing these six degrees is to understand them and understand what it will take to reach Degree 0 where there is no separation between you and your data which will allow you to be data driven. The issue that is the thread between each degree of separation is the need to eliminate latency when gathering, cleansing, standardizing, integrating, correcting, and synchronizing application and user data to make it usable for enterprise and digital needs.
Degree 6: Data in Application Silos
The beginning of IT was all about applications. Getting operational benefits from the use of an application was the objective and using data outside of an application was an afterthought.
Unfortunately, applications operate as independent silos with their own data definitions, structures, and rules. Data in application silos requires duplication of some data, like customer information, rules that operate the way the application works, separation of users from data.
This degree of separation is important to understand because it continues today, whether buying an application or acquiring one as Software-as-a-Service (SaaS) subscription. This pattern of application acquisition adds to the challenge of becoming data driven.
Degree 5: Data in Departmental Reporting Data Marts
Reporting databases, data marts, and data extracts for analytics were created to provide data and reports for operational managers and business analysts. Work to collect, prepare, and analyze data is applied to applications used by a department or business unit so its data and analytic needs can be satisfied.
But data in reporting databases, data marts, and data extracts also require duplication of some data, like customer information, and rules that operate the way a department or business unit works, separating non-departmental users from data:
This degree of separation is important to understand because departmental data is not always correct and consistent everywhere it is used, which is a requirement for data-driven, real-time, digital business.
Degree 4: Data in an Enterprise Data Warehouse or Analytic Platform
The need for complete, accurate, and consistent enterprise-wide reporting drove the practice of developing an enterprise data warehouse (EDW) to contain and organize enterprise data over the past 25 tears or so. Built using enterprise data models and implemented on relational databases, the process of designing and implementing an EDW took considerable time and effort.
EDWs could not quickly adapt to include data from a new application or to address a new need. This prevented EDWs from providing correct and consistent data everywhere it is used in an enterprise. Over the past several years, Big Data analytic platforms became popular because they provided more flexibility in their data architecture and lower costs using commodity platforms and open-source software. While providing useful capabilities, these platforms have not become substitutes for EDWs and have added another piece pf complexity to the data layer:
The data layer in every business has become a complex operation, requiring tools, technologies, and skilled personnel to get data to data users. This wasn’t a problem when data users accessed data from their desks using an office computer, but the move to mobile devised changed that.
Degree 3: Data Used by Mobile and Web Applications
Customers and younger professionals expect data to be delivered to their mobile devices. The speed with which mobile banking apps replaced banking website applications exemplifies this.
This represents a fundamental change to the data delivery mechanisms in stages 4 – 6: mobile users don’t care what happens behind the glass of their mobile device. They expect complete and accurate data to be delivered to their device instantly.
More than that, their experience with Amazon and other customer-centered businesses have trained them to expect to drop from one device to another as they interact with a company digitally. This requires a store of event-responsive data delivering that provides the continuity, data, and application capabilities that enhance their experience.
Digital engagement is preferred to face-to-face engagement by many customers today. It happens at their convenience, if a company’s mobile app and website provide them services and capabilities they want to use.
Degree 2: Data Used by Automated Analytics and AI
As businesses become digital, the need to understand customers, interpret digital interactions to understand their interests and behaviors, and entice them with information and offers in real time demand increasing automation. Automated analytics, machine learning (ML), and AI add complexity to becoming data driven.
The initial focus is typically on customers and personalizing interactions with them. The purpose is to increase sales, improve service, and enhance the customer experience. The fact that these are often new automated capabilities to support mobile app and website digital interactions means they can be implemented while minimally impacting existing applications and enterprise reporting in place also makes them easier to implement:
Digital engagement is increasingly sophisticated as analytic and computing capabilities become more capable. Inevitably, this means that
Degree 1: Data Delivery Acceleration by Using Microservices
Automation, ML, and AI can also be applied to the business itself. However, business processes use Degree 6 applications that are integral to how the business operates, presenting challenges to automating elements in them.
Microservices have been promoted as the technical practice to solve this problem. However, it is a fundamental fact that Degree 6 application silos will always require data to be collected, analyzed, utilized and processed in real time:
This is the central issue: it is very difficult to become data-centric if a business insists on being application-centric.
Degree 0: Enterprise Data in a Unified Datastore for All Enterprise Uses with No Separation of Data from Users
As digital transforms business operations with mobile applications, internet of things data, and automation, it is important to think of data as enterprise data rather than the current definition as ‘data in the enterprise’ that includes application data, data marts and EDWs, and user data. Digital operations need enterprise data to be correct and consistent everywhere, not data that must be gathered, cleansed, standardized, integrated, corrected, and synchronized to become usable for enterprise needs.
I believe we want enterprise data to have these characteristics:
- It would exist in one place, so it is always correct, consistent, and ready to be shared, eliminating work for gathering, cleansing, standardizing, integrating, correcting, and synchronizing application and user data shown here:
- To ensure data is always correct, only one set of rules is applied when data values are entered rather than needing to reconcile differing rules applied in applications and user data.
- Analytics will be consolidated and standardized for enterprise use, operate in real time, and support a range of mobile platforms and web browsers.
It would exist in one place, so it is always correct, consistent, and ready to be shared, eliminating work for gathering, cleansing, standardizing, integrating, correcting, and synchronizing data:
It would also operate across multiple clouds and hybrid environments. The core capabilities are having unified data and rules available everywhere, supporting all aspects of a business, and operating in real time.
Works Cited
1. Forbes (2018, July 5) "The 13 Types Of Data" Retrieved from https://www.forbes.com/sites/adrianbridgwater/2018/07/05/the-13-types-of-data/?sh=1eeccd133624
2. Oxford Languages https://languages.oup.com/google-dictionary-en/
3. Techopedia (2017, Jun 15) "What Does Enterprise Data Mean?" Retrieved from https://www.techopedia.com/definition/28048/enterprise-data