It is that time of the year for all and sundry to come up with‘Trends for 2015’, so here is my commentary on two of the technology trends that have received a lot of attention and have made a big impact in the previous year. I am sure you will agree that the “technology” which has stood out is “Big Data”. Likewise, the “role” that has received most attention over the previous year, and one that is likely to stay prominent in 2015, is that of a Data Scientist.
The two terms are somehow related. Data Science represents a cross-over from the business analyst role into the realms of data analysis, visualisation and the application of advanced analytics to get to valuable business outcomes, with a healthy dose of data management and data manipulation thrown in as well. The most published application of this role is on, but not necessarily limited to, Big Data initiatives. What should set a Data Scientist apart is his/her ability to address the right problems that will add the most value to the organisation, whether Big Data is required or not. Interesting enough, that particular aspect has not necessarily really come to the fore in many organisations’ big data initiatives.
Anyway, with that in mind, I came across a great article entitled ‘13 New Trends in Big Data and Data Science’ – and based on my interest in Big Data and the Data Scientist role, I want to comment on some of the proposed trends to watch. So this is not another set of trends to watch, but rather commentary on those that the authors of this paper highlighted.
The rise of data plumbing
At first this seemed like a silly concept, however, the more I read about it, the more I believe it has merit and will receive more traction as we go forward. It refers to a dedicated person who ensures that the big data technology runs smoothly and safely and is reliable for the business. Loosely using an analogy from the conventional BI world, it is a Big Data ETL Analyst. Like the ETL Analyst, the Big Data Plumber is a tech, with a aptitude and skills to manipulate big data sets, and make it available using Big Data technology. Like the ETL Analyst, the Big Data Plumber has little business insight or interest. But like in the BI world, it is a necessary function, and if done well will free the “real” Data Scientist to focus on the analytics and application thereof in the business, thereby getting to value part.
The use of Data Science in more fields
The article cited unusual fields such as astrophysics, which is valid of course because scientific data is one of the best examples of “real” Big Data. However, I personally think we will see more data scientists focusing on the conventional data, from which a lot value has not yet been unlocked in many organisations. So I foresee more Data Scientists applying the “data science” methods and approaches to conventional structured data, with or without scatterings of Big Data, to address business challenges and to drive more actionable business value from such data.
Putting more intelligence into big data applications
This will assist to detect and eliminate unhelpful or irrelevant data. As organisations start to understand their Big Data sets better, they will start filtering the data earlier on the process, only keeping the data that is relevant to the organisation, thereby eliminating the useless noise as soon as possible. I think more and more organisations will apply some logic to filter and only store the relevant data, instead of blindly dumpng everything and keeping it forever.
Increased awareness of data security
As broadband becomes more easily accessible, more and more security and privacy breaches are likely to occur. The expectation is that the roles of Data Science and Big Data will likely look to detect hackers through specific data and predictive analyses. While I believe that analytics can help detect security and privacy breaches, just as it can play a role in detecting the probability of fraudulent claim behaviour, I would much rather see organisations putting better security and privacy protection mechanisms in place than having to detect potential breaches and dealing with them accordingly.
The rise of mobile data exploitation
The article lists a vast array of unrelated example, but what stands out for me is marketing the right product in real-time at the right place (via opt-in, user-customized text messages), for example promoting a discounted item a customer walking close by in a shopping mall. The challenge is more about capturing and reacting to the right signal, rather than absorbing and digesting big volumes of data. Of course the resulting information must be presentable and consumable on a wide range of mobile devices, but organisations are getting quite good with that, if we look at the wide array of mobile apps already available.
Death of the fake Data Scientist
The article drops this one-liner as a trend to watch for in 2015. That is a very loaded sentence right there! But it is not a trend. Sure, there are many out there who are deliberately trying to ride the data science wave, even though they may not be suitable qualified, or have a serious shortage of skills and/or business experience. Sure, some of them are going to be found falling short. Some will obviously “BS” their way through it all, like you find in any job in any industry. However, part of the confusion is because of the unregulated and unstandardised job descriptions we find in the IT industry. For instance, what do you call the really bright advanced analytical modeller who has no business experience but who plays a crucial on the data science team? Is he a data scientist or not? A data scientist in the making? Until job functions and roles are as regulated as in the engineering, accounting and healthcare disciplines, we will always have those grey areas, and no matter how much rigour and formality you put around it, you will always have the chancers too…
Concluding remarks
Considering the above (and these are only some), we can see that the world of Business Intelligence (BI), Big Data and Data Science is set to be an exciting one in 2015. As the year evolves, BI, Big Data and Data Science will not only be considered an important part of IT, but will be mandated; given that the value it can offer an organisation is phenomenal. However, one needs to take care that some degree of sense and sensibility is maintained.
