Multi-cloud environments are challenging enough for IT enterprise teams to run. Throw in both traditional and cloud-native applications based on microservices, and you really put a strain on teams tasked with providing excellent quality of service in a multi-layered architecture. Yet this scenario is a reality for many enterprises and their IT teams today. According to a report published by IDC, the typical cloud-native application can have from five to 15 dependencies on other services, resources, or application programming interfaces (APIs) — each of which generates its own set of operational risks and associated metrics and data*.
This increase in the overall number of dependencies and therefore potential points of failure requires IT teams to rethink their capabilities. More specifically, it requires them to expand their journey into observability to better understand what is going on under the hood.
This trend has led to the explosion of the observability market — one of the hottest topics on our radar. It’s an incredibly dynamic space in which consolidated vendors and startups are battling to address the growing needs of customers.
As active investors and acquirers in this arena, our job is to constantly take the pulse of the market and align our strategy to what the market is telling us. At Cisco Investments we have the luxury of being able to do this by leveraging two of our most valuable resources: our customers and portfolio companies.
To this end, we recently hosted a roundtable discussion with four executives from Fortune-100 companies in the banking, pharmaceutical, and automotive sectors to get their insights into observability.
To gather additional insights from the companies responsible for growing the emerging ecosystem, we also spoke with Liran Haimovitch, CTO and co-founder of Rookout. The company is a four-year-old Israeli startup in Cisco Investments’ portfolio and was recently named a “Cool Vendor in Monitoring and Observability” by Gartner.
What challenges are companies facing on their journey to observability?
Three key points stood out to us during our customer roundtable discussion:
Different levels of cloud maturity = different observability approaches. Cloud adoption still varies greatly from business to business, even within the same industry. Some were all in on multi-cloud, and one had zero cloud adoption. Security and data protection were the main deterrents to cloud adoption. Accordingly, observability varies greatly from customer to customer, and different vendors are called upon to offer different capabilities to address the various facets of the market opportunity.
Tool sprawl and lack of cohesive strategy. There is massive fragmentation in the observability technology landscape with myriad solutions having sprung up in recent years to address specific use cases or provide visibility into certain layers of the stack. There’s a similar fragmentation of tooling adoption among our customers. This is often due to a lack of cohesive strategy around observability, and to different business units embracing a reactive approach.
Tool rich, data poor. All our discussion participants reported using several solutions, and even solutions from several vendors within the same family of products. Yet all seemed to struggle with gleaning insights from the data, and more importantly with the lack of a common data model to be able to contextualize and correlate different data sources. This lack of integration among tools and lack of a common data plane curtails visibility and understanding, especially for customers with a multi-cloud footprint.
How are enterprises responding to these challenges?
Here’s what our customers are saying:
Organizational change. In response to growing IT complexity, our customers are adopting new organizational models and developing cohesive roadmaps at a company level. Enterprises that are further ahead on the journey report the need to appoint a leader who is responsible for all observability initiatives in the company, as well as the need to rethink organizational models that emphasize product ownership over a platform approach.
Centralized strategy and federated execution. These new organizational models call for the observability strategy to be centralized and, by extension, for the selection of tools to be centralized too (which may be good news for vendors). At the same time, they strongly emphasize the decentralization of operations, which customers said should be in the hands of application teams versus the “old” approach of centralized logging and monitoring.
Investment in developer tools. With the business demanding ever-greater speed and agility, developers are under pressure to deploy code more frequently. According to discussion participants, this results in more bad code being sent to production — perhaps the greatest source of problems. To prevent this from happening, customers are investing more in tools that help developers automate the CI/CD (Continuous Integration/Continuous Delivery) pipeline and improve the quality of code shipped.
It is clear from the above that enterprises are gearing up for an increase in complexity and looking at observability to manage it. This will likely result in a need for more solutions and, to meet this need, more innovation generated in the startup ecosystem. To get a reading from that ecosystem itself, we turned to Rookout.
What we learned from Rookout
Rookout’s Liran Haimovitch echoed most of what we heard from our customers, in particular the growing demand to provide developers with tools that would allow them to increase the quality of code shipped. Rookout, which allows developers to do live debugging of applications, was intended to support application debugging in production. But Liran told us it’s now being used five times more in staging and pre-production environments — an insight that seems to underscore the need to improve observability in the code development and testing phases.
According to Liran, he foresees a future of observability in which the mainstream MELT approach (ingestion of four data types — Metrics, Events, Logs, and Traces) will be coupled with a more on-demand approach to observability. In this scenario, users will utilize tools (like Rookout) that allow them to investigate potential issues without necessarily having to rely on a predefined set of information. Furthermore, Liran believes we’re moving toward a democratized approach to observability where observability tools will be wielded by multiple teams with unique needs (from SREs and front-end engineers to back-end engineers, QA, support teams, data engineers, and more). And the ability — and agility — to access data and get on-demand insights from it, coupled with the MELT stack approach, will be key.
Observability is a must-have
Based on our conversations with customers and portfolio companies: with even the more conservative enterprises starting to adopt microservices and more and more workloads being moved to the cloud, observability is no longer merely a nice-to-have but is rapidly becoming a must-have capability to transition from just monitoring to a paradigm that delivers shared context across teams and enables companies to deliver exceptional digital customer experiences.
Want to know more about how we’re helping modern enterprises accelerate their observability journey? Contact us at work-with-cisco-investments@cisco.com.
* IDC - FULL-STACK OBSERVABILITY: Expanding the Digital Experience and Impact with Advanced Business Context