Overcoming the Challenges to True End-to-End Automation

Digital transformation needs to be viewed as a strategic, holistic initiative to improve business operations. And to be truly considered holistic, it must include the network. To be highly effective, network automation strategies must be part of initial discussions. They must encompass the entire planning, execution, and analysis of digital transformation deployments instead of plans focusing solely on implementation and activation of use cases.

To focus on just a small aspect of network automation during digital transformation planning and execution is a disservice to the network, the IT teams expected to support it, the company running on the network, and the customers impacted by those decisions. In fact, most network automation implementations are limited by:

Diving deeper into these challenges, we can determine the strategies and initiatives needed to overcome them, as network automation becomes a priority for company growth.

Understanding the value of end-to-end automation

For the math-inclined, Metcalfe's Law posits that the value of a communication network increases proportionally as the square of the number of nodes in the network increases. For those of us who are mathematically challenged, what Metcalfe's Law was attempting to do was illustrate how the value of a network grows exponentially with each new connection. We see this not only in communication networks but in other types of networks. For example, the value of a platform like Facebook was limited when the number of users and data was small, but as the number of Facebook users increased, the value of the platform increased dramatically, both to users and to the stock market. While there have been debates on the precision of the math in Metcalfe's Law, the truth of the underlying idea is self-evident -- value increases more than linearly as the number of connections increases.

It’s now time for the network automation industry to recognize the relevance of Metcalfe’s Law to network automation and how it can help address the challenges to automation implementations.

Extending the reach of automation

Integration drives value – the more systems that an automation platform connects to, the more activities it can participate in. Each one of those connections replaces one or more manual steps that an engineer would normally need to perform on separate systems - including collecting appropriate data and then using it to perform a task.

The gating factors in most environments that prevent mass integration are cost and complexity. Integrations typically require developers to create and to maintain the integration code. Due to steep integration costs, organizations are forced to make difficult choices, which leads to prioritizing the systems they can afford to integrate immediately – and which fall victim to budget cuts.

But cost can no longer dictate priorities when it comes to network automation. Now, organizations must anticipate that the future of automation will be driven by integrations, which require a reprioritization of initiatives that promote flexibility. Only then can they properly extend the scope of their automation initiatives.

Managing data through federation

Data federation is the ability to provide aggregation and harmonization of data from multiple sources, such as network controllers, orchestrators, inventory management systems, and performance systems. Since automations are only as good as the data available, federation becomes a required capability to manage the volume and variety of data that comes with large scale integration.

The aggregation of data via a federated system differs from the way traditional management systems collect information. Traditional systems rely on copying massive amounts of data to create a master database. This leads to issues concerning the timeliness and quality of the data in addition to ongoing synchronization challenges.

Federation eliminates these concerns and accommodates the rapid changes to the network by creating a meta-map of the systems and then querying those systems in real-time during the execution of workflows or other business logic.

Providing data transformation

The last historical hurdle has been how to extract information from one source and to modify it so that it can be sent to a target such as a device. Even in relatively simple use cases, a change to a service might require information from multiple sources – such as an ordering system, an inventory system, and an IP address management system. The native structure and formats of this data will likely be very different, so an engineer would need to determine which fields would be relevant and which fields that data should map to – then create a script or some other code to parse the data, restructure it, and compose the payload. This is a time consuming and complicated process.

To address this challenge, complete data transformation is a necessity to manipulate and format data appropriately, without needing to develop extensive coding, ensuring that all integrated systems can speak the same language.

Most importantly, organizations must continuously dive into the challenges mentioned above to be constantly aware of how to effectively extend the full scope of their network automation across systems, devices, and teams. Only by becoming aware of these obstacles and by implementing effective mitigating strategies can teams begin to work towards true end-to-end automation.

Morgan Stern is a VP of Automation Strategy at Itential.