KeyLogic Blog: BI/Information Visualization

Learn more about the advances and expertise our KeyLogic team brings to their respective fields.

Why or When Do You Need a Data Warehouse?

Oct 29, 2013

Whether you are the CEO of a fortune-500 level company or a director working on a mid-scale project for the government, chances are that you need to deal with lot of business data. If you want to analyze all that data in a way that can enhance the decision-making and customer-service capabilities of your business, you might want to consider the data warehouse route.

The key features of a data warehouse or DW are: integrated, subject-oriented, time variant, non-volatile data; data structured mainly for query, analysis and reporting; usage of special tools and techniques. To make the decision on whether your business will benefit from a DW, these are some points to consider:

1) Disparate sources of data – Do you have data coming in from a variety of systems? Some legacy systems, some spreadsheets, flat files, some transactional systems or different databases? You might want to build a central repository, namely a data warehouse, that draws in only relevant data from all these scattered sources.

2) Cleansing and Transformation needed – Is the data in the different sources in 1 above not consistent? Does it need some kind of massaging to make sense for reporting later? Does the same figure mean different things in different source-systems? Or do the various systems represent the same thing in different ways? Are there incorrect records that have crept into the system? If so, then ETL i.e. Extraction, Transformation, Loading process will help you in maintaining a system containing consistent, correct and relevant data.

3) Custom reports – If you need reports that the present schema of data does not support or that will need a considerable data-remodeling, a well thought out and modeled data warehouse will be useful to generate those custom and specialized reports. Some applications have a requirement that users be able to generate their own custom reports. As long as the database supports the underlying schema of the reports, it is easier and faster to achieve this using a DW.

4) Multiple views – Slice and dice, Drilldown, Rollup, Pivot are different techniques for viewing and understanding the data. While it is possible to incorporate some of these using standard database tools, it becomes easier to do this after modeling for DW. 

5) Performance – If a very fast performance on a huge amount of data is an absolute necessity for your reports, grids or charts then having denormalized data design as in a DW helps. A lot of features of DW such as transformed and cleansed data, denormalized database schema, Materialized Views, Aggregations etc. are catered towards fulfilling this need for faster performance of queries.

6) Data mining – All the above relate to viewing the data in various forms. But if you want to discover ‘hidden patterns’ of data-flows and groupings, building a DW could be the first step to such Data mining. For example, determining who the best customer for your business is and for which products, comes under data warehousing. Finding which two products when grouped together will maximize your profits will probably fall under Data Mining.

If major needs of the enterprise are being fulfilled by existing systems and processes, then it may not be worthwhile in terms of time and money to spend building a DW. But if your business needs any of the above capabilities in the way it takes in, processes, stores and interprets its data, then data warehousing is the way to go.

Written by: Sanjivita Misra

You are not allowed to post comments.