Please insert this Google Tag:

The Warehouse

Short reads. Big insights.
Industry trends. Thought Leadership. Opinions. Hot Tips. And so much more.
 

Jun
20
Change Data Capture and Matillion Data Loader

As enterprise data volumes continue to grow and the velocity of data change increases, companies are more challenged than ever to provide timely and complete data sets for business analysis. Change Data Capture is a methodology that has been employed for some time to effectively extract changes into target data environments in near-real time by avoiding regularly scheduled batch cycles and loading source data as it updates. The objective is to have a provisioned data store be as reflective of the current source system data as possible. There are several approaches to CDC employed by data engineers, but in the end, which is the most effective and least costly strategy that insures complete ...


Oct
12
Data Modeling in the Cloud Era

We have all seen how more and more companies are moving to the cloud for their data management platforms. Snowflake, Azure Synapse, AWS Redshift, and Google Big Query are leading this charge towards low-admin, instantly scalable cloud database solutions. Accompanying this is a migration to cloud-hosted data integration and low-code ETL solutions like Matillion and Fivetran. It is tempting to assume that with all these low-overhead data management platforms the concept of data modeling may be a thing of the past, relegated to the pile of on-premise databases that this brave new world is supplanting.


In reality, data modeling is more important than ever. A key to understanding this importance i...


Pandata Group

Chicago

WeWork/ Fulton Market

220 N. Green Street

Second Floor

Chicago, IL 60607

Madison

701 E. Washington Ave

Suite 202

Madison, Wisconsin 53703

Send Message