The Warehouse

BLOG & REsources

Jun
22
Matillion Data Loader: The Fast, Easy (and Free!) Way to Populate Your Cloud Data Warehouse

There is no question that companies are moving their on-premises data warehouses to the cloud at an increasing pace. The benefits of a cloud data warehouse (instant scalability, minimal up-front costs, rapid deployment, ubiquitous access, etc.) are being fully realized and appreciated by a growing number of enterprises both large and small. The major players in cloud DW (Snowflake, AWS Redshift, Azure Synapse, Google BigQuery) are all vying for market share, and the customers are seeing benefits from the varying and competitive costs and features of each platform.


But how do you get your data to the cloud? Isn’t the time and cost comparable to any data load project, whether it is on-pre...


Jun
22
Rethinking the Data Vault for Real-time Data

The data vault has long been viewed as a model best suited for historical and archival enterprise data. Its “insert only”, business-process approach to raw, unadulterated data is ideal for low-maintenance storage of all enterprise-generated information from all systems. Use cases for data vaults have traditionally revolved around historical tracking and auditing . . . however, the perception has largely been that it is ill-suited to analytics due to its many-to-many relationships and dispersed structure. In fact data vaults are often used as a “lightly modelled stage” for traditional star-schema data warehouses.


But the data vault may be best suited for a use case that...


Mar
30
Lakes, Swamps, and Puddles: The "Data Wetlands" Ecosystem

If you feel like you’re “drowning” in jargon and buzzwords surrounding the recent developments in data lakes and their ilk, you are not alone. A recent TDWI survey showed rapidly increasing adoption of data lakes as a source of big data analytics, though it also revealed barriers to success and confusion around implementation value. Much of this confusion stems from myths and misperceptions around the technical and business uses of a data lake. This article will examine the proper use of a data lake, and how proper governance can prevent it from becoming the dreaded data swamp.


To be clear, a data lake is not a data management platform, in that it is not an integrated, ce...


Mar
23
Where's the "T?" A look at ETL vs. ELT

In a previous blog post, we examined the differences between traditional ETL (extract, transform and load) and ELT, where the “heavy-lifting” of data transformation is handled by the robust and scalable (usually cloud-hosted) target platform. In today’s modern cloud data warehouse environment, ELT maximizes the speed at which data is staged and ingested, while leveraging massive computing power in the cloud to cleanse, aggregate and otherwise prepare that data for general consumption. But where is the best place to manage that transformation piece? Is it using cloud-friendly ETL tools, or is it within the management consoles of the cloud DWs themselves?


A common perception a...


Aug
01
Pandata Group Partners with ThoughtSpot to Deliver Search and AI-driven Analytics

Pandata Group is pleased to announce a partnership with ThoughtSpot, to deliver search and AI-driven analytics to reshape the way mid-market and enterprise organizations are able to answer data questions, find insights and make decisions.



ThoughtSpot was recently positioned in the Leaders quadrant of the Gartner 2019 Magic Quadrant for Analytics and Business Intelligence Platforms. The software platform is a spark to what Gartner recognizes as the third wave of disruption to traditional BI in the form of augmented analytics. With ThoughtSpot, business people can type a simple Google-like search in natural language to instantly analyze billions of rows of data, and leverage artificial intelli...


May
29
Talend v. Matillion for Cloud Migration

The two leading ETL/ELT tools for cloud data migration are Talend and Matillion, and both are well-positioned for moving and transforming data into the modern data warehouse. So if you’re moving to any type of cloud-hosted DW, whether it is a cloud-dedicated warehouse such as Snowflake, or part of a larger cloud platform such as AWS Redshift, Azure SQL Data Warehouse or Google BigQuery, which tool should you use to move your existing on-prem data?


Both Talend and Matillion can source any kind of on-prem data and land it in a cloud-hosted data environment. They can also move data to and from AWS’s cloud data-storage S3 as well as Azure’s Blob storage (which can be used to s...


May
28
Approaching an ERP Migration with Analytics in mind

Growth companies today rely on Enterprise Resource Planning (ERP) systems to manage their daily operations and collect and retain vital business data. The ERP space continues to evolve . . . cloud-hosted ERP, AI-based automation, digital transformation, etc.; eventually an organization will find itself upgrading or migrating to a more modern enterprise platform. This move will likely involve a Systems Integrator (SI) specialist to drive the effort, and an SI’s focus may need to incorporate more than just the operational systems at hand . . . the organization’s approach to data integration and analytics should be accounted for as well.



Your SI and internal migration team will init...


Mar
15
Data Integration Roadmap Series - Part Two: Master Data Management

When planning your integrated enterprise data environment, it is impossible to understate the importance of master data management. Much has been written about MDM, and it encompasses a broad range of (mostly non-technical) disciplines that are beyond the scope of a single blog entry. Here we will provide a broad overview of the four main areas of MDM to start your journey towards enterprise data governance. We will also examine the relationship between MDM and recent developments around other enterprise management programs such as Product Information Management.


What is master data management? Quite simply it is the administrative oversight of organizational “data as an asset” to...


Mar
08
Data Integration Roadmap - Part One: The Logical Data Model

Our recent blog series on the data integration portfolio introduced a variety of new architectures that help the enterprise manage their data resources, including replication, virtualization and cloud data warehousing. Organizations are now able to integrate multiple data management solutions to address a variety of business sources and requirements. But it is important to understand that the foundation of any enterprise data management portfolio remains the same . . . a roadmap to data management must be created that is independent of the underlying technology. This series of blogs will examine the three main elements of the data integration roadmap: the logical data model, master data ma...


Oct
18
ETL vs. ELT - What's The Difference and Does It Matter?

For most of data warehousing’s history, ETL (extract, transform and load) has been the primary means of moving data between source systems and target data stores. Its dominance has coincided with the growth and maturity of on-premise physical data warehouses and the need to physically move and transform data in batch cycles to populate target tables efficiently and with minimal resource consumption. The “heavy lifting” of data transformation has been left to ETL tools that use caching and DDL processing to manage target loads.


However, the data warehouse landscape is changing, and it may be time to reconsider the ETL approach in the era of MPP appliances and cloud-hosted D...


Pandata GroupLessLess
Pandata GroupLess

Madison

Pandata Group, LLC

316 W. Washington Avenue

Suite 525

Madison, WI 53704

(877) 350-5192

Madison

Pandata Group, LLC

316 W. Washington Avenue

Suite 525

Madison, WI 53704

(877) 350-5192

Less

Pandata Group © 2020

Pandata Group © 2020

Less


Chicago

WeWork/ Fulton Market

220 N. Green Street

Second Floor

Chicago, IL 60607


Chicago

WeWork/ Fulton Market

220 N. Green Street

Second Floor

Chicago, IL 60607

Less