https://blog.catchpoint.com/2019/06/19/why-apm-synthetic-monitoring-isnt-enough-to-protect-your-revenue/?utm_campaign=Nurture_2020&utm_medium=email&_hsmi=88637303&_hsenc=p2ANqtz-_xjaHkZMYnlgqOAah55uNQpvYMJDiZu4KJykIVz-kHXvvhzxPzmiaZumFAMQOBVYIJ0jnrSk6KV30hAREkKLhFmjuTVg&utm_content=88637303&utm_source=hs_automation
When synthetic monitoring was first invented in the mid-1990s, it was a revolutionary way to ensure that your digital properties were performing as expected for your end users. Back then, synthetic testing was mostly used for availability testing of your external websites and services. The majority of the monitoring budget was spent on infrastructure, network, and applications. End-user monitoring was merely a luxury and an afterthought.
This worked well at the time because the majority of your digital architecture was managed in-house, and the customer-facing product was far simpler. Synthetic tests of first-party components were therefore able to uncover almost all issues that could be acted upon immediately to repair the problem.
That, however, is no longer the case.
Complexity within our digital architecture has exploded in recent years with the advent of cloud infrastructure, content delivery networks (CDN), DNS providers, third-party API services, traffic steering services, third-party tags and tag management systems, and a number of other component types that can represent performance bottlenecks and/or single points of failure.
As a result, a monitoring tool like an Application Performance Monitoring (APM) platform, which provides code-level monitoring and tracing, is insufficient for providing data from all the different layers of your digital delivery because there are so many other places where the problem may lie. While many of these providers may offer a “synthetic” solution, the actual telemetry that they provide only covers a small percentage of the potential root causes.
Additionally, these APM synthetic solutions are increasingly only hosting their agents on cloud providers like AWS, Azure, and Google Cloud, rather than on the Internet backbone infrastructure and consumer ISP networks that end users actually use to access your site, application, and services. As a result, these solutions leave you completely blind to issues that originate outside of the cloud providers, which not only hampers your ability to respond to and repair an issue in a timely manner, but also leaves you without any data to enforce your Service Level Agreements (SLA).
What’s needed is a unified platform that combines a TRUE synthetic monitoring tool (i.e. one that actually emulates the end-user experience) with real user monitoring (RUM), internet intelligence, and SaaS monitoring. Together, these tools create a comprehensive digital experience monitoring (DEM) platform that covers nearly every aspect of your digital footprint and can be deployed alongside your code-based APM solution.
While APM is a valuable tool for discovering issues within your own code, by not testing from the end user’s perspective – i.e. directly from backbone, broadband, last mile, and wireless networks – you are guaranteed to miss critical issues (false negatives) that are impacting your end users’ experiences, causing you to lose both revenue and brand credibility when those users inevitably grow frustrated with unacknowledged service disruptions.
Given that a true synthetic solution is intended to emulate the experience of actual end users, the inability of APM vendors like Dynatrace, NewRelic, and AppDynamics to test from the actual networks and geographies where they’re located, as well as the different devices and browsers that they use, means that they really don’t have a synthetic solution at all. Fantasy monitoring might be a more appropriate term.
The reason why these vendors have decided to take this approach is pretty simple – running a global infrastructure like Catchpoint’s that supports true synthetic monitoring from locations and networks all over the world is an expensive and challenging enterprise. Placing a few dozen nodes on cloud providers is far cheaper and easier.
To sell this solution, these vendors will tell you that the data collected from cloud nodes is “cleaner” or “more stable” – i.e. fewer performance spikes and alerts that you have to worry about – which is like saying that you never have to worry about buying an umbrella as long as you measure rainfall from inside your house. But in the meantime, you remain blind to all of the issues that lie outside of APM’s limited purview.
Here at Catchpoint, we’ve seen the problems that are caused by APM-based synthetics first-hand. For example, at one point global weather service AccuWeather switched away from Catchpoint in order to consolidate their synthetic monitoring under their APM vendor. However, they quickly realized that the limited geographic locations provided were woefully insufficient to service their global user base, necessitating a switch back to Catchpoint to provide a more complete monitoring strategy.
“Catchpoint has the infrastructure set up where we can protect our global brand with global monitoring,” says Stephen Savitski, Sr. Director of Enterprise Monitoring at AccuWeather . “The ability to get a test up and running in five seconds from the exact location where we’re seeing issues – as opposed to trying to configure a VM somewhere in the cloud – is huge for us.”
In addition to the geographic testing limitations, the APM-based synthetic tool that AccuWeather used before switching back to Catchpoint did not provide the analysis capabilities that are necessary for quick detection and repair times. “Most of our problems tend to be with client-side issues like third-party JavaScript, which could be video, which could be advertising, which could be content,” said Savitski. “Catchpoint’s synthetic monitoring helps us narrow our focus, analyze, and isolate specific waterfalls versus specific sessions and immediately see the problem. Then we can share that data with those partners, or wherever the problem may be, and rectify that quickly.”