Browsing through the articles and documentation on the web about Data Warehouses leaves you in a state of mind that this is for the big end of town. They talk about Terabytes of data, sometimes Petabytes, streaming from so many sources, and combining with a handful of systems that must cost a ton to run - and therefore provide double that ton of benefits. Awesome solutions. But what about the rest of us? What about those that just want better insights into our ERP Systems, or our CRM or we have a custom Line of Business application that we would want to merge with ERP or CRM - or we have an obsolete system sitting there whose data could not be migrated to your new shiny system? Where is the data warehouse for the rest of us?
We at Pebble IT are simple folk. We start with the simplest solution to a problem and let requirements drive the complexity rather than starting with complexity. Less complexity means less cost. It does not mean less security, and it does not mean less capability. We are a big fan of starting as simple as possible, getting runs on the board, and growing from there.
We can help you realise your strategy for having flexible reporting and data insights regardless of the source of your data. We don't start with the most complex solution and work our way down, instead we start with the most effective solution and where your requirements dictate, add more to the mix until we get to the right solution mix for you.
We use the term "Reporting Warehouse" to represent a simple data warehouse where the data is mainly sourced from a single system, typically an ERP that optionally may have some historic data loads added in as well. This data is not subject to much cleansing, typically there will be a rolling 3-5 year data window - data older than this just hasn't got relevance.
The reporting from this is also somewhat simpler, whether it be extracting into Microsoft Excel for formulas and the typical slice and dice that you do in spreadsheets - or reporting that previously would have been done by the likes of Crystal Reports, but now is the domain of Tableau or our preference, Power BI. By utlising Power BI, you get a lot of power at your fingertips using reports that we can prepare for you or if you have the skills, go down the self service path and discover the wealth in your data.
When you start adding more requirements for cleansing data, filtering data, merging it with other sources and aggregating across multiple dimensions, you are getting into data warehouse territory. The reporting warehouse can easily grow into that, or you can start with that. The key is to understand what bells and whistles you want rather than buying all the bells and whistles upfront.
So what does a reporting or data warehouse look like from Pebble IT? The diagram below shows the major components that we can assemble together -
We work through a series of questions with you to be answered that will drive the key elements of design required. Integration (the extraction of data into your data warehouse) is becoming a lot easier to perform, and the performance of manipulating large data sets is becoming more accessible to more of the business community. We believe that there is no company too small to be able to afford the data warehouse solutions we can assemble for you.
As required, we can integrate extra services including Analysis Services for powerful aggregation and KPI measurements performed on the Cloud giving you access to valuable insights in seconds, not hours; unlimited processing power that can scale from small workloads through to the largest of requirements, integration of your cloud services whether it be relational data or non-relational streams, and the ability to make your data available to data automation solutions (Microsoft Flow, Azure Logic Apps) and web apps in general. This is all driven by what challenges you wish to overcome in your business and we can help you decide on how to phase the delivery of solutions so benefits are delivered in manageable chunks that lowers the risks to your business.
The reporting warehouse can morph into a data warehouse and this can grow with you as sophisticated as required to deliver the strategic benefits that your business strives for. The below diagram gives you an idea of the possibilities of making your data in and outside of your business work for you -
We are database experts and we tune, migrate, re-platform, upgrade and manage databases for our clients. We have for years and that is our bread and butter. It makes complete sense that we extend this expertise into data warehouses, we have helped companies migrate data into on-premise and cloud data warehouses and transform the data to enable meaningful insights.
When you wish to discuss what challenges you wish to overcome, you won't be presented with an army of consultants looking to add complexity to drive up their bills, instead you will be presented with a transparent and honest discussion on what is best for your business. If that sounds like something of interest to you, please reach out via our contact page.