Is the hottest trend in BI taking us all in the wrong direction?
To call the recent push towards self-service (SS) analytics a proliferation would be a major understatement. In fact, you would be hard-pressed to find a business intelligence (BI) vendor that hasn't effectively deprecated its full-service (FS) or guided BI offerings in favor of a model very heavily skewed towards SS—and certainly this is true of all three "Leaders" of Gartner's Magic Quadrant, who have been championing the movement. Not surprisingly, this is a self-perpetuating economy at the moment. Tableau does something, so Microsoft has to compete. Microsoft does something, so Qlik has to compete. Qlik does something, so Tableau has to compete. And around and around we go.
So What's the Problem?
There are several fundamental concerns with leveraging SS analytics for users other than analysts. Here are some of the biggest:
- Loss of Focus: Every minute that users are deep diving into data is a minute that they are not working on their primary jobs. I'm all in favor of professional development, but we all have a finite amount of time in each working day, and it is important to use it in a way that brings the most value to our organizations and teams. I remember that I once lost two full work days trying to learn d3, before I realized that my time is better invested focusing on my core strengths, and delegating the rest.
- Finding All the Goodies: Data is fluid and frequently updated, and insights are often very time-sensitive. Once an opportunity is gone, it may be gone for good. Entrusting everyone with their own data analyses and hoping they find all relevant insights in a timely manner is not a particularly smart strategy.
- Best Practices: Even within a partially-guided SS framework, you run the (near 100%) risk of users creating visualizations that are not valid, whether statistically or aesthetically. Picture a line chart creating a false trend between discrete data points rather than over time. Or a pie chart with 100 slices, that obscures an important insight which would have been readily apparent with a different visualization type. Or averaging percentages incorrectly. You get the idea. It seems that the major selling point of all the SS solutions is "look, all you have to do is drag, drop, and boom! you've got yourself a bar chart in 7 seconds flat."
Suffice it to say that creating good BI entails a bit more than that; to any BI professional reading this, I sincerely hope you do not think you are making the big bucks for creating bar charts? (tweet) Just about anyone can throw some data into a bar chart, but it's like the old story goes: that'll be $1 for the hammering, and $999 for knowing where to hammer. - Standardization: No matter how hard IT, Data Governance, and Branding tries, encouraging everyone to create their own analyses will mean that they will...create their own analyses. This will likely mean that things like color usage will not be consistent (a fairly minor quirk) but can also mean that customers get segmented in contradictory ways, business rules are applied inconsistently, etc. Subsequently realigning all the different analyses can be a huge drain on time and resources.
Data Literacy
The recent push by all the big players towards "data literacy" is, of course, meant to mitigate some of these problems (and help their bottom lines, coincidentally).
LinkedIn and Twitter feeds these days
For those fortunate souls who have been living under a social media rock the past year, the crux of the movement is to teach virtually all employees best practices in terms of not only data visualization, but even denser concepts such as data modeling and MDM. Not everything, of course, but enough to make them "literate." The premise here is that, given the torrential amount of data that suffuses us on a daily basis, data analytics is (and will continue to an even greater extent to be) the new "language" of successful organizations; wouldn't you want everyone to be able to at least speak the common tongue? And, of course, once everyone is literate, they can all start slicing and dicing their data in every way imaginable, unearthing a veritable treasure trove of insights; oh, happy day!
If you want to teach data literacy, great! Teach everyone how to read and interpret more advanced chart types. All too often, developers of FS solutions feel constrained in terms of what charts they can use, and have to settle for a sub-optimal choice because the general population doesn't understand any visualization types beyond pivot tables, bar charts, and line charts. So yes, if the goal is to introduce folks to scatter plots and normal distribution curves, then sign me up. But, for all the reasons above (and more), it is a fundamentally misguided effort to turn everyone into an analyst. (tweet) Let your sales reps sell; let your doctors see patients; and let your BI specialists craft your data analyses. There is nothing wrong with having core competencies; in fact, it is essential.
The Right Path Forward
So what do I view as the future of business intelligence? With the bulk of the world moving towards self-service, I am moving in the opposite direction: not only towards full-service (polished, well-thought-out dashboards), but towards actually spoon-feeding insights to users as much as possible. The big vendors are right about one thing: the amount of data available to organizations these days is simply staggering, and the growth of that data is exponential. But that is precisely what makes this full-service BI's time to shine! The job of truly excellent BI is to provide clarity to a world that sorely needs it. It is not to make users first wade, then swim, and finally drown in their own data, while desperately hunting for insights. (tweet)
I have personally seen the relief on business users' faces when we create solutions that simply give them the answers they are looking for, without making them work for it. (tweet) In the healthcare space, for instance, imagine outputting relevant insights directly into providers' EMR systems, and letting them bypass all BI dashboards altogether. Isn't the success of a BI solution measured by its adoption? Well, I personally guarantee that consolidating your systems, rather than providing yet another portal to your already-overwhelmed customers, will improve your adoption numbers. More importantly, increased use means more insights get seen by more eyes, and business improves dramatically as a result. Power your business with BI, but don't make it about BI. (tweet) Trust me, I have seen the promised land, and it is beautiful.
In the past few months, I have very slowly started to notice certain organizations catching on to the fact that what we need are answers, not more questions. I sincerely hope more and more businesses follow suit; that would be a trend I would enthusiastically get behind! But let's all hop off the self-service bandwagon, shall we? (tweet)
Quite interesting!
In fact, I have been thinking about something similar: Is the self-service killing the “Intelligence” component of the “Business Intelligence” concept?
Regards and congrats!