Agile Information Architecture

In this blog post I cover the concept of an Agile Information Architecture. The terms “agile” and “architecture” in one phrase almost seem like a contradiction. So, here I unpack this term to clarify its meaning, ascertain its relevance within the broader field of BI, and to discuss a useful tool that can be utilised to assist in its implementation.

 Information Architecture

Before we can discuss how it can become “agile”, let us first consider the definition of an Information Architecture.

In fact, there are many definitions of Information Architecture. In generic terms it is used to describe the structure of a system, i.e the way information is grouped, the navigation methods to get to it and the terminology used within the system. Information architecture is most commonly associated with websites and intranets, but it can be used in the context of any information structures or computer systems. Because of their high data and informational content, information architectures are often utilised to document BI systems.

In the BI world, the information architecture describes the data structures used to store data in the data warehouse, the content thereof, where it is sourced from, how it flows into the data warehouse, how it flows downstream from the data warehouse, and where and how it is used in reports, dashboards and other analytical applications. An effective information architecture enables people to logically work through a system to find, trace and enquire about the information they are interested in. In reality, the information architecture in a BI system encompasses all the metadata related to that system, details about its physical implementation, and more.

Technology evolution

Through the evolution of technology, with aspects like Big Data and the cloud emerging as strong business drivers over recent years, the information architecture within an enterprise needs to change and develop.  This change is not merely to allow a company to keep up with the times and be relevant, but more importantly, to ensure that it becomes agile in its approach, and to ensure it remains competitive.

The reality today is that with all these influencing factors, and the many different solutions and configurations in which they can be deployed, the possibilities for very different enterprise architectures are endless. The seemingly simple configuration of multiple source systems, a single data warehouse, some downstream data marts, and a bunch of reports and dashboards is fast becoming inadequate. Even throwing in an ODS and some superfluous layers in the data warehouse does not cut it. Nowadays we need to access and integrate external and unstructured data, have feedback loops with analytical and CRM applications, and so forth. In fact, there is no “one size fits all” design to information architecture anymore.

So given that there is not a one size fits all approach anymore – how does a company ensure its Information Architecture is developed and deployed correctly? Well, you have to build it from the ground up, and you have to keep updating it as the business requirements and implemented systems change. However, to do this effectively, the organisation must be cognisant of separating related workloads and host data on relevant and appropriate platforms, which are then tied together by certain elements, including:

• Data integration
• Data virtualisation
• Master data management
• Data and program governance

Agility

In order for the information architecture to keep up with the fast-paced changes in the business, and hence in the data and information flows throughout the organisation, it has to become more agile. In fact, for an information architecture to become agile, the crucial elements of data governance must already be in place, as without them, inefficiencies and suboptimal information delivery could be the result. Inefficient or incorrect information delivery to decision makers could be particularly harmful to an organisation as it could negatively impact the tracking or adherence to strategic goals.

And so, based on this, more and more organisations will be going the agile route and begin to embrace agile approaches for managing their information architecture. Yet, such an approach comes with its own set of challenges. For example, you don’t want such rigorous documentation approaches and standards that takes longer to update the documentation than it takes to change the underlying systems. But you cannot run wild with undocumented and ungoverned systems either.

Planning and control

A common criticism of the agile approach is a lack of planning and architecture, coupled with a lack of governance and control. But that is symptomatic of the agile approach being implemented in a cowboy style. Seriously, you can have those same complaints if the old traditional highly structured waterfall approaches are not properly implemented and governed too.

A very useful tool, which comes out of Ralph Kimball’s work on dimensional modeling, is the ‘enterprise data warehouse bus matrix’. The bus matrix is a high level view of the main business entities (indicated as dimensions), and what should be measured about each business process, per these listed entities (indicated as high level facts). It represents the information architecture of the data warehouse and the downstream BI components at a very high level, in a very concise, consistent but useable format. Business people can understand it at a glance, and it contains a lot of information for information managers. As such, it offers a ‘master plan’ for agile development and it identifies the reusable common descriptive dimensions that provide data consistency and reduced time to market delivery. It is also very useful to identify the information requirements of strategic or changing business processes.

Having been in the BI industry for a number of years now, I really like this extended utilisation of the bus matrix. Agile is definitely the way to go, and so using a tool like the bus matrix as described above is a strong starting point and a very useful on-going management tool to get the information architecture right and keep it on track, without falling in the trap of maintaining a hundred complex, too detailed documents through all the changes.