As this is my last blog post for the year, I thought it would be fitting to explore how the Business Intelligence (BI) landscape has changed over the years, especially regarding toolsets. Having worked in the BI sector for numerous years – in the last five years the BI toolset landscape has not only expanded, but actually exploded. Of course, the hype around big data, the availability of more processing power and especially memory, as well as the ever ubiquitous mobile platform has fueled this explosion.
When BI first made an impact on the industry, organisations merely had to make a simple choice between a few mainstream BI tools – a data warehouse database management system (DBMS), an extract transform and load (ETL) tool, a reporting package, and if they were really progressive, maybe a data dictionary or a data quality or profiling tool. Simple, you could do the evaluation on a single spreadsheet. My goodness, the Inmon vs Kimball religious debates took up more time than the toolset evaluations. In those good old days you could actually source your entire BI technology stack quite easily from a single vendor.
Today, more often than not, most companies have one or more data warehouse DBMSs, depending whether you only have structured and dimensional data, or whether you have any of the unstructured types of big data as well. In which case you may have implemented Hadoop or in some cases a more specialised NoSQL database, which simply provides a method for storing and querying of data that is more applicable to those data types than conventional relational databases. In fact, you may even have moved the entire data warehouse, or only the time-critical parts of it, directly into an in-memory database like SAP HANA.
However, it is not just the data warehousing and storage models that have changed – in fact on the information exploitation front, things have really taken off. Take for example the fact that today, an organisation with mature information management and utilisation, may have one, of each, of the following tools in their organisation:
- Data exploration tools – also called visual data discovery tools, to investigate the data, detect patterns and analyse it interactively in further depth – and then develop insights from this. Examples of these toolsets are Tableau or Qlikview or SAP Lumira.
- Data reporting and dashboarding tools – where a business intelligence dashboard is a data visualization tool that displays the current status of metrics and key performance indicators (KPI’s) for an organisation. Dashboards consolidate and arrange numbers and metrics. Popular tools in this genre include SAP BusinessObjects, IBM Cognos, Microsoft SSRS, Microstrategy, Oracle OBIEE and many, many more – way too many to list. In fact, the Gartner report on BI toolsets enumerates 40 tools, of which the majority fall into this category, and that’s is not even a complete list yet.
- Advanced analytical modelling tools – which are used to create and run statistical, mathematical and other predictive models on existing data to generate insights previously unknown. It is very important that the right model needs to be implemented in order to get to the right insights from the data, but that’s a topic for another blog another time. For example, organisations can use SAS, SPSS, R or Python to generate such business insights, and this list is not exhaustive either.
In a very federated organisation, or even where there simply aren’t a lot of architectural governances, you may even see two or more of each type of toolset floating around, serving different groups of information consumers.
You can see, just from the above, the selection of BI toolsets is not merely a quick selection and decision-making process anymore, but needs to be a well thought through process that really requires insights into the informational needs and maturity of the organisation. Often internal staff are influenced by past successes or failures, and especially by what they have become accustomed to, that they may need external guidance and support to approach it objectively. More often than not they battle to understand their own requirements, and even more so, their target maturity level and what they need to get there. Today, BI specialists – are just this – specialists and experts in their field and are crucial assets in your organisation. Referring to the toolsets above, you may have data analysts; BI report and dashboard developers; and data scientists or analytical modelers where each group respectively uses one of the toolsets described above. And depending on the culture of the organisation and its relationship with the tool vendors, those toolsets may even be sourced from totally different vendors.
So as we go into 2014 – the information age is set to explode even further – given not only the amount of information that organisations are dealing with, but of course the advent of social platforms, which further drives information and sharing internally and externally – all of which is relevant to how organisations view and define their information exploitation strategies. So don’t merely try to stick to the old way of using a single BI tool, but rather take advantage of this expansion and use it to the benefit of your organisation.
Happy holidays!
