Practical Business Intelligence

The days of locking a bunch of smart people in a room until they understand and deliver on a static set of narrow requirements has been over for a while now. But if there was ever any doubt, the cross-functional, politically charged, somewhat esoteric requirements of business intelligence should put an end to the debate. And if that doesn’t do it for you, how about the insane number of BI projects that fail every year due to lack of communication and coordination between IT and the business.  (Need to read more?  Try here or here or timeless truths from a few years back, here.)

I would suggest that one reason why so many BI efforts fall short (or outright miserably fail) is that they are approached fundamentally wrongly. In my mind, all the articles cited are saying similar things: there was too much “lock smart guys in a room” and too little “collaboration between IT and the business.”

*pounds table*

I want process agility.

I want continual interaction with the end user community, business requirement drivers, and project stakeholders.

I want more communication, milestone sign-offs, RAD modeling, and the like.

Maybe you could get away with writing your Visual C++ app in a vacuum 12 or 13 years ago, but those days are long gone. If you don’t put the business guy on the development team (he’s called the Product Owner … feel the Agile software development love … see also here and here), make cross-functional business teams responsible for the requirements, the priorities and the data (read: governance), and develop software incrementally in short iterations, then I submit hitting the fast-moving target of the organization’s requirements for BI will be next to impossible.

“The world is changed. I feel it in the water. I feel it in the earth. I smell it in the air. Much that once was is lost, for none now live who remember it.”

This is the opening line of J.R.R. Tolkien’s The Fellowship of the Ring. It’s also a great way to describe the state of the IT universe. These changes are significant, and have not only served as a driver for the need for Business Intelligence as a science, but have greatly impacted the way BI must be implemented in an organization for it to reach its maximum potential.

Globalization, the maturity of the web, commoditization of ERP and CRM systems (among others), struggling economies, and the like have resulted in many different changes. Today I’m focused on the impact these and other forces have had on requirements management … particularly as related to your BI initiative. The truth is that gathering, understanding, managing, and implementing on business requirements these days just isn’t the same (read: isn’t as easy) as it used to be. And I thought it’d be valuable to throw out a few of the pitfalls out there, and throw out a suggestion or two on how to navigate them.

Requirements in the BI universe are:

… Hard to Nail Down

The truth is that business owners don’t really know what they want. I think this has always been true, but the complexity and abstract nature of much of BI has made an already difficult problem worse. With traditional IT projects, gathering requirements was about getting a business leader to tell you what a single piece of technology had to do for them in a very limited domain to make a very specific source of pain go away. Gathering requirements in the modern era for which BI is the solution is about asking that same manager (and his boss, and his boss, and managers / bosses in 5 other departments) to tell you what report or dashboard they want from data they didn’t even know the company collects to help them do something they already do better or mitigate some vague sense of pain that they can’t define. BI is still typically just too “out there” for many business leaders to give IT leaders succinct requirements. And let’s face it, BI’s still pretty “out there” for a lot of IT folks too, so that doesn’t help.

… Cross-functional

Business Intelligence, if on the path to meet its full potential, is not implementing one manager’s or department’s or group’s requirements; it’s attempting to implement the requirements of the entire business. So, now you have to get a bunch of business leaders–who may or may not talk, may or may not know what each other do, may or may not be willing to share IT bandwidth or priorities, may or may not have coinciding schedules to make a meeting, etc–into a room together, and get them to understand and agree on what a single system can do for them.  Ugh.

… Politically Charged

Enterprise-wide BI inherently means there are a lot of cooks in the kitchen, and it’s a given that each will have their own agenda. This is a natural outcome of the cross-functional reality of BI requirements. But it’s more than that. The business processes that are typically on deck for becoming a part of the BI initiative–for a widget company, the “sell widget” process, or for an association, the “new member joins” process, or for a group of investment bankers, the “risk assessed” process–are the most core processes in the business. They are the most important processes, have typically been around the longest, “enjoy” the most scrutiny, have the most people believing they know best how to do them, engender the most heated debate, and are surrounded by the most safeguards / people unwilling to change the way things are. This makes them pretty hard to deal with and makes gathering requirements about changing them a non-trivial activity. Who has the authority to order the change? Who really knows what’s best? How do you get all the cooks in your BI kitchen to agree on which spices to use (or sometimes even what to call the spices)? The fact is that they very things that make these processes so hard to work with are exactly the things that puts them in the in cross-hairs for your BI initiative in the first place. Thus, hard by definition.

… Constantly Changing

Of all the changes the reality of our times has wrought upon business, perhaps the pace of change is the most significant. It’s not just that things are different, it’s that they keep changing. And the incredible agility with which these changes must be addressed in order for BI (or IT in general for that matter) to demonstrate crucial value to the business is staggering.

… Typically Shrouded in Eroded Credibility

The fact is that too many attempts at BI (or other “killer app” concepts billed as “silver bullets”) have been made in too many organizations without understanding that there is a right and a wrong way to change. (I’ve got another post brewing on this topic.) First, stop billing BI (or the data warehouse or CRM or ERP or Web 2.0 or cloud computing or whatever else might get you excited this week) as the magic answer to anything. It’s not. These are tools in the toolbox of the business; nothing more. They will not turn the world rosy or solve world hunger. And they certainly won’t without incredible cost and intentionality and discipline. No wonder IT’s credibility has been tarnished in the eyes of the business. Information technology geniuses have spent far too much time selling the next cool thing, and far too little time listening to the needs of the business and investing in learning to think like their systems’ stakeholders. As a result, initiatives have cost too much for too long, delivered too little too late, and have generally lowered the trust the business has in IT to advance the mission of the business (which is what IT’s role is in the first place).

So, now what?

Here’s the bottom line … Talk less, even listen less, and observe more. And for Pete’s sake, embrace the reality that technology is not going to solve the problem. Good people following good processes and practices are.

If it’s not you, get someone in your shop with a trained eye, and spend time with the business users. Observe the way the organization works and doesn’t work. Really get to know the pain of the person / people who will be using that report or dashboard you think you should create. Understand the vision, mission, strategic goals, and key performance indicators of the organization. If you can’t articulate the following key elements for a BI software component, then you aren’t ready to build it:

• How this component will help achieve the goals of the organization?
• Who needs to understand the value of the component and the unit of measure in which they measure value?
• What is the pain that this component will alleviate and who is feeling it?
• Where does their pain fit in the overall priorities of the organization?
• What is it going to cost the organization to build and not to build this component?
• Who are the people who have the most to gain and who have the most to lose if you build this component?
• Who are the people who are the most skeptical that you can deliver value with this new IT wonder?

Also, get some processes and tools in place to help your cross-functional, politically-minded requirements drivers to collaborate effectively. On the tools side, companies like Balanced Insight and 37signals can help with that. On the services side (assessment, advisory services, etc), so can companies like Capstone (my company).

Last, there’s the myriad best practices that are beyond the scope of this post, but which I’m sure we’ll discuss here over the life of this blog. Some examples off the top of my head: start small, build iteratively and incrementally, interact frequently with the users / stakeholders, deliver value early and often, target detractors with quick wins, embrace the 80-20 rule, don’t wait for data to be perfect before you expose it, learn to love the phased rollout, start with bedrock data and radiate outward from there, avoid silos in your data warehouse, always get sign-off on project milestones, and the list goes on.  If there isn’t a blog entry out here on every one of these concepts in the next 6 months, then I’ll be a monkey’s uncle. So, stay tuned.

Yes, this is more than you needed to know to whip out a VB system or an ASP.NET app in days gone by. But that’s to be expected. If the complexity of the requirements has skyrocketed, why would the process to deliver on them not be more complex as well? Don’t long for the good old days (where they really?). The potential power and value offered by BI and other IT constructs in the modern era is just as significant (if not more) than those of 10-20 years ago. They just require a new way of thinking.