PERSPECTIVES

For most of data warehousing’s history, ETL (extract, transform and load) has been the primary means of moving data between source systems and target data stores. Its dominance has coincided with the growth and maturity of on-premise physical data warehouses and the need to physically move and transform data in batch cycles to populate target tables efficiently and with minimal resource consumption. The “heavy lifting” of data transformation has been left to ETL tools that use caching and DDL processing to manage target loads.

However, the data warehouse landscape is changing, and it may be time to reconsider the ETL approach in the era of MPP appliances and cloud-hosted DW’s. These architectures are characteriz...

This blog series has examined the hybrid data portfolio as a mix of technologies and approaches to a data foundation for the modern enterprise. We’ve examined a variety of strategies and technologies in data integration, including virtualization, replication and streaming data. We’ve shown that there is no “one size fits all” approach to an integrated data foundation, but instead have seen how a variety of disciplines that suit specific business and technical challenges can make up a cohesive data policy.

This final chapter puts it all together under the umbrella of “time-to-value" and its importance to the agile enterprise data platform. No matter what the technology, data strategies invariably involve movi...

In our previous installment on the hybrid data integration portfolio, we looked at the role of data virtualization in a unified, multi-platform approach to creating a managed enterprise data foundation. In this chapter, we examine data replication and synchronization, i.e. the ongoing copying of data (without massaging or transformation) from one physical location to another, usually in conjunction with change data capture (CDC).

Data replication is often considered ETL without the "T", though where ETL is usually a batch-based delivery process, replication is often driven by "update-upon-change". Through this process, the target database only updates when changes occur to the source. Often referred to as "just-in-time" data, this repres...

What can EDA do?

EDA is the governed integration of disparate data collection applications, into a centralized “source of truth”, for enhancing business acumen to improve and optimize decisions and performance. EDA allows health plans to easily:

• Analyze provider performance in order to optimize pay-for-performance

• Evaluate treatment patterns across providers and better tailor care management program

• Identify underlying opportunities for health education and intervention with high-risk members

Sustaining fixed EDA support and overhead costs can allow an organization to continue growth and improvement, thus providing an infinite return on investment (ROI).

Why do you need an EDA?

• How are your patient outcomes improving?

• Are you doing the same things but expecting different results?

• Have your past investments become your “legacy” systems?

• Are you experiencing rising costs, long revenue cycles, and bad debt?

• Who are the health care provider/vendors that are performing well?

• What lost economic opportunity (LEO) have you missed?

• Are you maximizing your ROI?

If you answered yes to any of the above questions, then you need EDA to give you give the best assessment of you organization performance, so that you can make the best decision for your organization. Even the smallest improvement in everyday decision-making can improve corporate perfo...

The challenges facing organizations to integrate and make sense of the plethora of internal and external data continue to grow. Not only is there diversity in data sources, but the requirements from business units involve a mix of batch, real-time, on-demand and virtualized capabilities, often from the same source for different use cases. It is rare to find a "one-stop-shop" solution to these varied data integration needs, so organizations end up with a hodgepodge of redundant, overlapping products, or worse, rely on custom internal coding with no traceability or modularity.

In this series of blog posts, we will examine the "hybrid data integration portfolio" as a planned approach to handling multiple data integration requirements with a...

Two DV's in Tandem: Data Vaults and Data Virtualization

There has been growing interest in data vaults as an architecture suited for the archiving and preservation of all enterprise-wide data. The normalized, "hub-and-link" structure is suited for parallel loading from multiple operational systems, and the philosophy of "all the data, all the time", regardless of quality, lends itself to the rapid population of the vault without cumbersome governance and oversight. However, the data vault is ill-suited as a source for reporting and analytic querying, since it separates keys from descriptive metadata, propagates many-to-many joins, and makes no distinction between high-quality and low-quality (or relevant and irrelevant) data. Companies...

Pandata Group is happy to announce that Josh Erhardt has joined the company as a Senior Consultant. His expertise is in Information Architecture and he is influential in guiding our clients to adopt a successful foundation for data integration and data warehousing.

Josh will be leading Pandata Group’s practice in Enterprise Information Architecture. Our audience is hit with waves of market analysts’ perspectives and technology vendors’ promises. The Enterprise Information Architecture practice helps cut through the noise to design successful platform for your business that is based on best practice and, most important, experience.

We think you should get to know Josh and his role at Pandata Group. We’ve inclu...

The first half of 2016 has quickly passed us and we’re heading into the final half of the year at a staggering pace! We’ve enjoyed much success in the first part of 2016 with the addition of new clients, new team members, and even our first office in Madison, WI. It’s no small feat, and we thank all those who trust us to help in their data and analytics initiatives.

I joined the company in January of this year to help with crafting our business strategy and also executing our market development. Our initial existence centered on our expertise with SAP EIM and SAP BusinessObjects software. Since the two founders of the company each possess nearly 20 years of experience with SAP BusinessObjects, it made a lot of sense to ...

In the first three parts of our series on creating an enterprise-class business intelligence platform, we outlined the methodology for gathering and modeling the business requirements, identifying the current and future data flows, and deciding how to best serve the varying analytical and reporting needs of the users. In this final installment, we layout the scope and level of effort that goes into the BI development lifecycle. It will examine the people and processes needed to ensure a high level of performance and maximize the time-to-value of any enterprise BI project.

BI teams (and associated data warehouse teams) can follow a wide variety of structures, often depending on the corporate culture. But at minimum, any successful BI team...

In Parts I and II of our series on creating an enterprise-class business intelligence framework, we examined how to logically model business processes for analytic consumption, as well as optimizing data flows within an organization to best serve that logical model. In this installment, we lay out the business intelligence platform itself and show how data governance plays a vital role in insuring the value of BI delivery.

When designing the enterprise business intelligence platform, the primary objective is to enable the delivery of the right information to the right people and the right time. This requires a combination of:

• Identifying the enterprise's BI consumers and maturity level

• Prioritizing and purposing data sources

• Implementing a comprehensive data/information governance strategy.

An organization's BI consumers can be stratified into three general levels (in descending size of population): E...