Machine Learning, Cloud, And Big Data: A Perfect Storm

I have spent 30 years of my life making a living in wireless. In the last 10 years, I have also been learning the art of wine making. Both passions have their good days and their challenging days. But that's not the only similarity between the two. It's interesting to see that recently they both started having something else in common – big data and machine learning. While these technologies have had an impact on several verticals like healthcare and retail, it is great to see that their applicability is now extending to even more areas.

How did we get to this scenario? 

Machine learning has been with us for some time. In the 1950s, A.M. Turing wrote a paper that defined machine learning as a machine that could learn and communicate with you without being distinguished from a human being. While we are still some way from the Turing definition, machines already famously exist that can beat humans at chess (IBM Deep Blue in 1996) and Jeopardy (IBM Watson in 2011). 

Recently, cloud and big data technologies also have emerged that complement machine learning in various ways, enabling us to take even more advantage of that technology. Amazon delivered the first public cloud service at scale with Amazon S3/AWS in 2006, Google supported Hadoop in 2003, and Spark was created from the Berkeley AMPLab in 2010.

From this perfect storm of technologies, we are seeing a proliferation of new machine learning applications. For example, Fruition Sciences is using sensors in vineyards to monitor the transpiration of vines to save water in drought stricken places like California, while at the same time improving the quality of the wine. Similarly, the inventors of Watson are using cognitive reasoning to guide oncologists on the most appropriate cancer treatment options based on a patient’s medical history. And biotech companies are using artificial intelligence and big data to find new targets for FDA approved drugs. 

It is interesting to note that there is also a trend whereby machine learning is being used by IT for IT. Splunk pioneered this space by using unstructured elastic searches to monitor, analyze and visualize server data for detailed IT metrics and fast problem remediation. Now Mist is using machine learning to automate wireless operations, Fireye uses it to detect security risks, Brocade is using it for automated network operations. Others use machine learning to optimize WAN paths, detect software bugs, and much more.

These applications are only possible today thanks to the perfect storm of machine learning, cloud and big data technologies. We can now collect and analyze data like never before, driving operational efficiency in business and delivering unprecedented user experiences.

With new applications popping up every day, we are at the forefront of a new era of machine/human interaction that will completely transform the way we live, work, and play – a change on par with the emergence of the internet itself.