How to Become a Data-Driven Company Without the Resources? DataOps + The Right Tooling

It's no surprise that COVID-19 is on everyone's mind these days. I was talking with a friend about the crisis and wondered how much more progress we could achieve with the right resources in place, at the right time. What if we could somehow empower people from all walks of life with the data and expertise they need to operate healthcare systems? 

Without the right strategy and tools, technology can only achieve so much—especially if we’re short on resources. In some ways, the healthcare crisis is a reminder of the challenges that enterprises are up against with data. In both cases, the key is getting the right data to the people who need it most and have the domain expertise to get the most value from it. And you’ve got to do it fast. Many of today’s latest-and-greatest modern data platforms in the enterprise are based on Open Source technologies. Open Source can give you the most powerful, leading-edge capabilities, so it’s widely popular. But it’s not a panacea. In fact, in some ways, it can even add pressure to your IT systems, and the people who need to run them. Open Source technologies are really cool, but they can also be complex and highly distributed. You won’t get the most out of them if you don’t have enough people with the advanced skill sets to apply them. 

When it comes to bringing people closer to data, DataOps practices can offer real advantages in data management. It's collaborative, highly automated, and puts data first while leveraging technology. To really tap the full value of your data, you need great technologies and an approach that provides context and understanding based on your business imperatives. With the right strategy and tooling, you can take a big step forward in being data-driven—even if your resources are limited.

Putting the right data tools in reach for more people

What tools will provide the strongest start for your data optimization strategy? You’ll want to start with tools that bring together best-of-breed technologies. Evaluate and choose the technology that’s best for your use case and your platform, not just because it’s trendy. Whether they are Open Source or commercially developed, the point is to pick the tool that can best deliver the outcomes you’re trying to achieve. 

Visibility is another must-have. If you want to get business users on board, you need to provide visibility into the system. What you’re looking to do is free up your users to concentrate on the business objective—not spend time wrestling with complex management systems and interfaces. 

When choosing your tools, be sure they build around a common skill level understood by everyone in the business domain, such as SQL. Make your data technologies accessible to everyone, not an elite, highly technical few.

You’ll want a good architecture to support your tools, and a data mesh provides a strong, nimble foundation. It lets you move away from the monolithic data lakes of yesteryear, to focus on discoverability and the correct use of data. A data mesh is all about picking the correct technology, giving individual domain teams access to it, and sharing insights faster.

Look at the business behind the data

In a data-driven company, technology is the enabler, while data is the protagonist. Data remains the constant. The technology is simply a way to visualize data and extract value from it.

We can see the data-driven approach with Vortexa, an energy analytics provider. Vortexa relies on an Apache Kafka streaming data platform to support its traders who are making decisions on the market. They take real-time time data feeds from oil freight, then process and resell analytics to organizations who trade based on the insights it delivers. With better visibility into its real-time data flows, Vortexa can give its analysts the business knowledge they need to understand data faster and respond to it. They can turn on a dime and respond to change right away, without having to circle back and engage with data infrastructure teams.

Just like any other initiative, building a more data-driven company can create risks.  Opening up more data to more users creates more potential for misuse. As you become more data-driven, it’s even more critical not to overlook compliance and governance—and data ethics. Be sure to stay aware of who is accessing data and why—and set up the right controls and transparency to ensure that data ethics are upheld.

Ultimately, becoming data-driven is all about bringing data closer to the business. The right tools, visibility, and strategy are fundamental to jump-starting your data optimization strategy. When technical expertise is backed up by business context and insights, you can move beyond “technology for technology’s sake”—and open the door to truly drive business outcomes.