PERSPECTIVES

BLOGS & NOTES

Aug
01
Pandata Group Partners with ThoughtSpot to Deliver Search and AI-driven Analytics
By Jorel

Pandata Group is pleased to announce a partnership with ThoughtSpot, to deliver search and AI-driven analytics to reshape the way mid-market and enterprise organizations are able to answer data questions, find insights and make decisions.

ThoughtSpot was recently positioned in the Leaders quadrant of the Gartner 2019 Magic Quadrant for Analytics and Business Intelligence Platforms. The software platform is a spark to what Gartner recognizes as the third wave of disruption to traditional BI in the form of augmented analytics. With ThoughtSpot, business people can type a simple Google-like search in natural language to instantly analyze billions of rows of data, and leverage artificial intelligence to get trusted, relevant insights pushe...


May
29
Talend v. Matillion for Cloud Migration
By Jorel

The two leading ETL/ELT tools for cloud data migration are Talend and Matillion, and both are well-positioned for moving and transforming data into the modern data warehouse. So if you’re moving to any type of cloud-hosted DW, whether it is a cloud-dedicated warehouse such as Snowflake, or part of a larger cloud platform such as AWS Redshift, Azure SQL Data Warehouse or Google BigQuery, which tool should you use to move your existing on-prem data?

Both Talend and Matillion can source any kind of on-prem data and land it in a cloud-hosted data environment. They can also move data to and from AWS’s cloud data-storage S3 as well as Azure’s Blob storage (which can be used to stage data for any cloud-based DW). From a dep...


May
28
Approaching an ERP Migration with Analytics in mind
By Jorel

Growth companies today rely on Enterprise Resource Planning (ERP) systems to manage their daily operations and collect and retain vital business data. The ERP space continues to evolve . . . cloud-hosted ERP, AI-based automation, digital transformation, etc.; eventually an organization will find itself upgrading or migrating to a more modern enterprise platform. This move will likely involve a Systems Integrator (SI) specialist to drive the effort, and an SI’s focus may need to incorporate more than just the operational systems at hand . . . the organization’s approach to data integration and analytics should be accounted for as well.

Your SI and internal migration team will initially be concentrating on the operations of t...


Mar
15
Data Integration Roadmap Series - Part Two: Master Data Management
By Jorel

When planning your integrated enterprise data environment, it is impossible to understate the importance of master data management. Much has been written about MDM, and it encompasses a broad range of (mostly non-technical) disciplines that are beyond the scope of a single blog entry. Here we will provide a broad overview of the four main areas of MDM to start your journey towards enterprise data governance. We will also examine the relationship between MDM and recent developments around other enterprise management programs such as Product Information Management.

What is master data management? Quite simply it is the administrative oversight of organizational “data as an asset” to maintain its consistency and credibility. It ...


Mar
08
Data Integration Roadmap - Part One: The Logical Data Model
By Jorel

Our recent blog series on the data integration portfolio introduced a variety of new architectures that help the enterprise manage their data resources, including replication, virtualization and cloud data warehousing. Organizations are now able to integrate multiple data management solutions to address a variety of business sources and requirements. But it is important to understand that the foundation of any enterprise data management portfolio remains the same . . . a roadmap to data management must be created that is independent of the underlying technology. This series of blogs will examine the three main elements of the data integration roadmap: the logical data model, master data management (including metadata management), and t...


Oct
18
ETL vs. ELT - What's The Difference and Does It Matter?
By Jorel

For most of data warehousing’s history, ETL (extract, transform and load) has been the primary means of moving data between source systems and target data stores. Its dominance has coincided with the growth and maturity of on-premise physical data warehouses and the need to physically move and transform data in batch cycles to populate target tables efficiently and with minimal resource consumption. The “heavy lifting” of data transformation has been left to ETL tools that use caching and DDL processing to manage target loads.

However, the data warehouse landscape is changing, and it may be time to reconsider the ETL approach in the era of MPP appliances and cloud-hosted DW’s. These architectures are characteriz...


Sep
06
The Data Integration Portfolio - Part Four: Putting It All Together (In The Cloud)
By Jorel

This blog series has examined the hybrid data portfolio as a mix of technologies and approaches to a data foundation for the modern enterprise. We’ve examined a variety of strategies and technologies in data integration, including virtualization, replication and streaming data. We’ve shown that there is no “one size fits all” approach to an integrated data foundation, but instead have seen how a variety of disciplines that suit specific business and technical challenges can make up a cohesive data policy.

This final chapter puts it all together under the umbrella of “time-to-value" and its importance to the agile enterprise data platform. No matter what the technology, data strategies invariably involve movi...


Aug
15
The Data Integration Portfolio - Part Three: Streaming Data
By Jorel

In previous installments of this series we examined recent trends in data integration, specifically data replication and synchronization, as well as data abstraction through virtualization. Taken individually, all of these approaches are suited for high data latency requirements around historical reporting and trending analysis. In this chapter, we look at real-time streaming data, and how it can complement high-latency data integration approaches to create a complete enterprise data foundation.

Streaming data delivery is often perceived to be the "holy grail" of data integration in that it provides users with immediate and actionable insight into current business operations. In reality, streaming has primarily been utilized in conjunc...


Jun
11
Feb
25
DATA INTEGRATION PORTFOLIO- PART TWO: REPLICATON
By Rafael Bacolod

In our previous installment on the hybrid data integration portfolio, we looked at the role of data virtualization in a unified, multi-platform approach to creating a managed enterprise data foundation. In this chapter, we examine data replication and synchronization, i.e. the ongoing copying of data (without massaging or transformation) from one physical location to another, usually in conjunction with change data capture (CDC).

Data replication is often considered ETL without the "T", though where ETL is usually a batch-based delivery process, replication is often driven by "update-upon-change". Through this process, the target database only updates when changes occur to the source. Often referred to as "just-in-time" data, this repres...