Kimball Dimensional Modeling Practices Waterfall Only?

Kimball Dimensional Modeling theory and practices are the most widely accepted processes for consolidating data from different sources into a central “Delivery Area” for consolidated cross functional reporting. Or simply a process for normalizing and standardizing data from several data marts into a data warehouse for Business Intelligence (BI) reporting.

From “The Data Warehouse Tool Kit”; Second Edition; Kimball, Ross; 2002; John Wiley and Sons, Inc.; Page 22:

Finally, dimensional models are gracefully extensible to accommodate change. The predictable framework of a dimensional model withstands unexpected changes in user behavior. Every dimension is equivalent; all dimensions are symmetrically equal entry points into the fact table. The logical model has no built-in bias regarding expected query patterns. There are no preferences for the business questions we’ll ask this month versus the questions we’ll ask next month. We certainly don’t want to adjust our schemas if business users come up with new ways to analyze the business.

The main tool used to discover the applicable Dimensions for a model is the Business Process Dimensions Matrix, see Figure 1. Each row represents a Business Process and Each Column a Dimension available within the Delivery Area that is populated manually or through the consolidation of the source data.

BPM

Figure 1. Kimball, Ross; et al.; Page 79.

On this surface this looks like a Waterfall approach… Identifying all the requirements upfront before development starts. I disagree. This document should be an organic repository that is constantly updated with changes like new Dimensions or Business process as additional data sources are added to the system. Further, I believe this matrix is the perfect primer for authoring User Stories. For Instance the first Business Process would translate to:

As an Inbound Contact Center Supervisor I want to see Voice, Chat, Email and Fax metrics summarized by Date, Time, Agents (Users), Goals and Locations So that I can…

The last section of the User Story where the reason or benefit is recorded also derive from a central tenant of BI practices. That central tenant is, “Every report must answer a question to aid in the conclusion of one or more business decisions.” The question we’re answering does not show up on the matrix, but should be part of the BI project management artifacts and the User story is the perfect place to record it.

If you’re reading this you may be a BI solution developer and suffered the frustrations of pointless and repetitive presentations (reports and dashboards) because your customers don’t know what they want. Someone on the project must take it upon themselves to get the stake holders to commit to the questions they want to answer. In Agile Scrum, it would make sense that the Product Owner maintains the matrix and the user stories and therefore should be responsible for those commitments. Here’s an Example of the resulting user story.

As an Inbound Contact Center Supervisor I want to see Voice, Chat, Email and Fax metrics summarized by Date, Time, Agents (Users), Goals and Locations So that I can more accurately forecast future staffing needs.

The wide acceptance of Kimball practices predates the wide acceptance of Agile Iterative Development practices. Therefore, several professionals in the space are unwilling to adapt their practice of Kimball methodologies. Hopefully this discussion will aid in efforts to convince these BI resources to modify their approach to conform to the Software Development Life Cycle (SDLC) methodology the rest of the development team uses.

 

Work Yourself Out of a Job

Has anyone in your management structure ever told you that if they could just clone you 4 or 5 times the company would be worry free? That is great feedback. You have earned yourself many years of job security. In fact you might say you’ve made yourself so indispensable at your current position, you’ll never be able to grow out of it.

 

For a lot of professionals in the world that’s exactly the position they want to be in. Joe is the best administrator BigCo Corp has ever had. No issues with any of the servers under his care. His management gets a little nervous when he takes a vacation because there is no one else who can do what he does the way he does it. Management doesn’t want to add any new variables to the mix and Joe will never be burdened with management training, team building, Agile methodology, mentoring, or other distractions. Likewise, Joe isn’t really all that motivated to share his expertise, both because he doesn’t like trying to explain what he does to others, and because he’s worried about losing some of his organizational expertise currency.

 

If you’re like Joe, read no further, because we’re going to talk about Sally. Sally, another administrator at BigCo Corp, likes having uninterrupted vacations. She has back filled her own position by training a few of her colleagues to do support her when she needs a break. Sally has taken the time to create process and systems documentation and publish it where others can reference it. Management notices that Sally can take care of her normal duties and build a team to add more redundancies to the processes. Management decides to send Sally to team building and management training. Sally becomes Joe’s new boss.

 

More than once in my career I worked really work and waited for my supervisors to recognize my talent and give me that promotion to management. I gotten the “Clone you” comment many times and sat on it. It took some advice for a mentor to open my eyes to the power of creating your own opportunities. You may not be able to personally improve the company’s bottom line and create more revenue that results in more growth and upward movement opportunities. If you do a great job, create a backfill for yourself and haven’t made yourself a pariah in your organizations politics, there’s a great chance you’ll get top consideration when an opportunity does show up.

Agile: The Consultant’s Savior

Thanks to Jeff Nall for contributing to this post.

How many times have you delivered to spec and hit your milestone only to get the “That’s not what I asked for.” feedback? Guess what, your customers don’t always know what they want much less what they really need. They might think they do, but if that were the case they would have been able to staff the project internally or with some new direct hires.

I once worked on a project where the company actually hired a consultant to translate corporate jargon into generic tech and software development terms. Apparently, they were struggling to find new hires or consultants with the skills they were looking for. Additionally, the resources they would gamble on were so lost trying to understand requirements, the development departments turned into revolving doors. The “Demystification Consultant” had a full time job translating specs and RFP’s so vendors could understand what to bid on. It might have been cheaper to adopt language learning methodologies and switch to a common communication device, illustrations.

In Agile that translates to frequent demonstrations of development progress. Rather than placing the entire project’s success on a nearly finalized demonstration of a product 2 weeks from delivery, Agile iterative development practices frequent illustrations of what the product will be so stake holders can approve or make changes with enough lead time to actually see the modifications implemented.

How does a consultant benefit from this process? Well like it or not, no matter what the contract says, an unhappy customer can remove a consultant and withhold payment if the customer believes they can argue in court that the contract was violated. Consultants are burdened with not just delivering what was promised on the agreed schedule, but also executing the contract in such a way that the customer wants to work with them again or act as a reference for other potential clients. Conducting frequent demonstrations: illustrates your responsiveness to your customers’ needs, reaffirms progress made on the deliverables  and keeps the lines of communication open for timely reactions to change. Agile is the best defense against “That’s not what I asked for” in the final days of your project.

The customer isn’t always right–they’re hiring you to answer the question for them. It’s your job to read between the lines of what the customer says they want and give them what they need. The point is the answers you get from someone who isn’t an expert in YOUR field can’t logically be the solution to the problem. You’re dealing with breadcrumbs, not road maps.