Data & Analytics require technology to deliver business results. Too often companies begin with a technology selection rather than an Information Strategy that determines what the business needs from data, their data use scenarios, and their analytic areas of opportunity. There are a wide range of products available for storing physical data, performing descriptive analytics, developing advanced analytics & insight, and technical data management. The challenge is to select tools that fit well together with minimal overlap and compatible technical approaches.
Technically, there are several technology trends affecting data acquisition and management that need to be considered in a technical architecture:
- Massively Parallel Processing: Distributing data over multiple disk drives will accelerate data access and manipulation;
- In-memory Database Management: Moving data storage from disk eliminates the time required to mechanically get data from disk, enabling high levels of performance;
- Big Data: Processing very large volumes of non-uniform, diversely-formatted, and unstructured data makes data movement from one data store to another impractical, if not impossible;
- Data Stream Analytics: Very large volumes of data presented as continuous data streams makes analyses of data at rest difficult, requiring the ability to analyze data as it “goes by;”
- Machine Learning Analytics: The ability to automatically adjust statistical models based on data observations is required to adapt to changes, identified as they happen, in data streams;
- Data Virtualization: Decouples analytic objects used by users from physical data structures, allowing physical data structures to change without impacting existing reports, dashboards, queries, and analytics.