The Warehouse

Pandata Group has always embraced and promoted the effectiveness of a robust data model as a means of “mapping” the enterprise data landscape (see our blog post here). Today this is more important than ever in the increasingly expanding cloud-native data analytics universe. A good data model provides not only a guideline for data engineers, but also a means of codifying and standardizing data usage from a governance perspective. Seamless integration of a data modeling tool into a cloud-native data management platform increases efficiency, collaboration and transparency. This is why we are so excited to be named a Gold Partner with SqlDBM, the leading cloud-based modeling solution...

We have all seen how more and more companies are moving to the cloud for their data management platforms. Snowflake, Azure Synapse, AWS Redshift, and Google Big Query are leading this charge towards low-admin, instantly scalable cloud database solutions. Accompanying this is a migration to cloud-hosted data integration and low-code ETL solutions like Matillion and Fivetran. It is tempting to assume that with all these low-overhead data management platforms the concept of data modeling may be a thing of the past, relegated to the pile of on-premise databases that this brave new world is supplanting.

In reality, data modeling is more important than ever. A key to understanding this importance i...

Enterprises running SAP Business Warehouse (SAP BW, BW/4HANA and BW on HANA) are keeping a close eye on challengers like Snowflake. Is cloud data warehousing the answer to all the challenges for organisations with a large SAP footprint? And how does a cloud data warehouse fit into the data architecture? Should it replace SAP BW or is there still value in SAP BW?

Why do most enterprises with a large SAP footprint run SAP BW?

The dominant position of SAP BW is easy to explain from a historic perspective, but it would not do justice to SAP BW to ignore its current strengths as well. Let us start with the latter. SAP BW is still the only data warehouse platform which delivers all data warehouse f...

In 2015, I was fortunate enough to lead the sales development effort of a cloud-based supply chain visibility product. The product was geared toward the manufacturing sector to allow a more transparent and collaborative platform to enable information sharing. We leveraged the Salesforce cloud to build the application, and it was intended to support digital transformation efforts within the supply chain to optimize the flow of information, provide real-time data to the supplier, empower collaboration, and scale for adoption. Unfortunately, the ability to execute on the vision failed and it was tied to one key component – the part on delivering real-time data.

Let’s fast forward to...

The data vault has long been viewed as a model best suited for historical and archival enterprise data. Its “insert only”, business-process approach to raw, unadulterated data is ideal for low-maintenance storage of all enterprise-generated information from all systems. Use cases for data vaults have traditionally revolved around historical tracking and auditing . . . however, the perception has largely been that it is ill-suited to analytics due to its many-to-many relationships and dispersed structure. In fact data vaults are often used as a “lightly modelled stage” for traditional star-schema data warehouses.

But the data vault may be best suited for a use case that...

If you feel like you’re “drowning” in jargon and buzzwords surrounding the recent developments in data lakes and their ilk, you are not alone. A recent TDWI survey showed rapidly increasing adoption of data lakes as a source of big data analytics, though it also revealed barriers to success and confusion around implementation value. Much of this confusion stems from myths and misperceptions around the technical and business uses of a data lake. This article will examine the proper use of a data lake, and how proper governance can prevent it from becoming the dreaded data swamp.

To be clear, a data lake is not a data management platform, in that it is not an integrated, ce...