Keysight uses a digital twin to enable businesses to test actual user experiences.
The societal impact of the pandemic has been well documented. Everything has changed as businesses have reinvented the way they operate. The biggest shift for in-office workers is the transition to home/hybrid work, causing an uptick in software-as-a-service (SaaS) applications. Cloud-based apps are ideally suited for remote/hybrid work because they let users operate from anywhere with an identical experience – regardless of location.
One of the underappreciated impacts of this shift is the change to application testing. App testing is a massive industry. Global Market Insights estimated the industry at over $40B in 2020 and expects it to grow at a 7% CAGR through 2027. This is why there are literally hundreds of point products meant to test specific apps or even parts of apps. The model of testing applications in isolation has historically worked because the apps were deployed in silos. For example, SAP was deployed on its own servers, storage and other infrastructure; Oracle was deployed similarly, and so on. Testing the application independent of the other infrastructure made sense in these scenarios.
This is no longer the case, however, because more and more applications have moved to the cloud. In this modern application environment, understanding user experience requires more than testing the application. Testing also needs to extend to all areas that could potentially impact the performance of the application, including the network, cloud, data center and interconnect systems. This is driving the concept of “full-stack testing,” where testing an app extends beyond the traditional silo approach of testing it in isolation.
Full stack is required for understanding actual user experience
This concept of “full stack” has been something the performance management vendors have embraced. For example, Cisco has spent billions in acquiring companies such as Thousand Eyes (internet performance) and AppDynamics (application performance). Riverbed has made Aternity (end-user management) the focal point of its digital experience management vision and combined it with its network information. This has become broadly known as full-stack observability, where IT pros have an end-to-end view of how the application performs.
This concept is just emerging in the world of testing. While there are some good application-testing products, such as Provar, Copado and Worksoft – and these are fine products – they lack the visibility outside of the application itself to find the root cause of some problems. Obviously, if the problem is isolated to the application, a siloed tool would be sufficient, but that’s rarely the case. Users often call the help desk complaining about Salesforce issues when in fact, the problem has nothing to do with the application itself. Instead, the problem is caused by a slow page loading, a poorly configured API or something else altogether.
This lack of end-to-end visibility is why IT is often working reactively, rather than proactively. In fact, my research shows that 75% of help-desk tickets are opened by the user, not the IT department, putting IT always in firefighting mode. To make matters worse, many vendors with products that work in a silo perpetuate the problem by using the phrase “mean time to innocence” as a value proposition. This is to indicate that the tool can help a specific group in IT claim that it’s not their problem and punt the issue to another group. I’ve always disliked this mentality because it does nothing to help the user.
Keysight enables full-stack testing
One vendor that’s taken a different approach to testing is Keysight, with its recently acquired Eggplant test & automation intelligence software. Eggplant was created in 2008 to enable software engineers to test end-to-end digital user experience versus only the individual application in isolation. The core of the Eggplant solution is the company’s Test Automation Intelligence platform, which uses artificial intelligence and machine learning (AI/ML) to test the actual customer experience. It creates a digital twin of an application, like Salesforce, enabling businesses to test real user journeys by continuously tracking actual user actions, paths and states through the application.
Salesforce is an excellent example of a digital experience that requires full-stack testing. The SaaS application is one of the most widely deployed globally and a Top 5 enterprise software company. Despite its massive size and reach, Salesforce can only control the experience within the application. The true Salesforce experience is dependent on the custom configurations on which the workflows run, the devices the worker is using and several other factors.
Testing Salesforce itself would be akin to testing a car’s engine to understand the driving experience. True experiences require putting the car in motion and running several scenarios, such as dirt roads, poor lighting, etc. This is the concept of full-stack testing for an application like Salesforce: rapidly expand test coverage beyond the happy path to ensure your custom configuration modals, action buttons, menu types, etc. were not altered.
What differentiates Eggplant is its ability to test at every layer – from application UI and digital through the application presentation and digital journey. Businesses can test any device, operating system, browser, or any other technology including user interfaces, APIs and even databases. It’s important to note that this includes automating all major aspects of testing within the CI/CD pipeline – from low-level test execution to high-level test design.
The platform uses AI/ML to automate bug hunting and the auto-generation of test scripts and snippets to ensure functional and usability test results are focused on the actual end-user experience. Without this capability, software testers would need to take the results and infer user experiences through manual correlation of the data, which is both time-consuming and error-prone.
Salesforce is one of the most critical enterprise applications, and full-stack testing ensures an optimal user experience. Software engineers should take a lesson from the Salesforce example and apply the principles of full-stack testing to all of their digital experiences.
