Data Disconnect, Data Overload

I don’t know when I first noticed that restaurants know how to talk to children better than to adults. I suspect the prerequisite for this knowledge was having children. Since then, the differences in communication have become obvious.

For example, I’ve noticed the difference between the regular menu and the children’s menu. The regular menu has many more options, sometimes so many that it’s difficult to make an easy decision. The children’s menu, however, usually has just five to seven items, making it easy for a child to select what he or she wants. The regular menus at some restaurants can be so difficult that experts are employed in order to help patrons. At a fancy enough place, there may be a sommelier to help people choose the right bottle of wine to go with their dinner.

A recent experience made me realize how similar the complex dinner menu is to what happens inside businesses of all sizes. The experience involved too much data not presented in a clear way, groups of people that could not communicate with each other, and experts being brought in to help everyone make sense of it all.

We’ve all experienced the silos that exist inside of corporate IT. Teams that need to be able to work together struggle to communicate with one another. Database and systems administrators are a prime example. The rise of DevOps is simply raising awareness to an issue that has existed for decades: We are drowning in data, much of it with little or no context. And making matters worse, each silo has its own data about what’s going on within the infrastructure --  its own version of the truth, if you will.

As data grows in importance and value for corporations and individuals alike, data is also becoming easier to obtain and distribute. So much data is flowing into, and out of, our hands each day. But without context and a way to ensure that everyone is aligned, like some kind of Rosetta Stone, once the data leaves the silo it originates in, it has largely lost its meaning.

Figure 1:

This is why the different groups that make up the traditional IT department struggle to communicate. In a way, they aren't speaking the same language. So it's no surprise that silos are formed over time. As data professionals, we can do better. We must do better.

Here's what I see as necessary in order to facilitate better communication across all IT teams in any organization.

As data professionals, we need to find ways to reduce the need for experts to help teams communicate. We need to have aligned data that can be shared in a meaningful way. And we need to help people understand what actions, if any, are necessary based on this data.

Otherwise, you end up with the equivalent of unhappy children being served food they don’t like. Trust me, nobody wants that.