(This post was written by Danny Moore and Dror Halperin, NV R&D. Edited by Itzik Gan-Baruch)
It is well understood that real-world network connectivity is one of biggest factors impacting mobile application performance. It turns out that application design also has a large impact on performance. Unfortunately, these two key performance issues are not always well considered during the development lifecycle which results in the delivery of poorly performing mobile applications to productions.
Anyone who cares about releasing high-performing mobile applications to the market must test the application on real-world network conditions prior to deployment to identify performance problems and fix application design problems.
In this blog, we will discuss how you can use Hewlett Packard Enterprise Mobile Center 2.50 with HPE Network Virtualization (NV) to extend automated Appium testing capabilities. This is accomplished by applying a variety of emulated mobile network conditions to mobile applications under test. There is a two-fold benefit for this automated practice: the ability to identify real mobile end user experience—before deployment—in different locations with various mobile connections, and, when identifying a mobile application with poor performance, you can leverage the HPE NV Insights report to accelerate performance.
Adding Network Virtualization to Appium Tests
Let’s begin with a simple Appium script. The following sample script doesn’t do much, it just opens our testing application, navigates to the ‘laptop’ page, returns to the first page and closes the application. Using Wi-Fi, the script takes 25 seconds to complete.
Now let’s throw in some network conditions and see the impact.
A few prerequisites before we begin:
If you don’t have it already, install HPE Network Virtualization (NV) and configure HPE Mobile Center (MC) to integrate with NV. Check Mobile Center’s online help to see how to do this.
After NV is installed and configured, make sure that the network traffic exiting the mobile device is routed to the NV instance. You accomplish this by configuring the device proxy settings to use the NV proxy server and port. If enabled, NV proxy is part of the NV installation running on the same machine and listening on port 8888, by default. You can check out these demo videos to learn how to configure the proxy settings of the device.
If your test includes secure SSL communication, download and install the NV proxy certificate on your mobile device. Again, check out these movies to learn how to accomplish that.
Now that everything is set up correctly, all that is left to do is to set the NV profile to use as a capability (‘3G Poor’ in our example):
To download the HAR file, just replace the response object in the code above. From: Object response = driver.executeScript("mc:net:getReport"); To: Object response = driver.executeScript("mc:net: getHAR");
You can use the HAR file to easily create a realistic mobile application load script with tools like HPE StormRunner cloud load testing tool.
Analyzing Test Results
To show the impact of bad network connections on test execution, we enhanced the test a little. We ran the script twice using NV, once using a good Wi-Fi connection and once over a slow 3G network connection. You can see the comparison of the test execution in this video.
We downloaded the NV Insights reports for both tests and compared them. The results quickly show that the network plays a huge role in the user experience.
Total test time increased from 25 seconds using Wi-Fi, to over 35 seconds using 3G.
Many TCP errors occurred on the 3G connection (98 timeouts retransmissions) while none were observed over Wi-Fi.
Maxing out the network bandwidth caused multiple network errors and delays. Max throughput in the Wi-Fi case was around 4.96Mbps (62KB per sample rate of 0.1 second, as shown in the graph below), and it took just 2 seconds to download the bulk of the data while in the 3G case we saw max throughput around 2.16Mbps (27KB per sample rate), and it took more than 4.5 seconds to download the bulk of the data.
The largest resource more than doubled its download time, from 0.9 second to 2.5 seconds (and we never actually saw it on screen).
Latency doubled for all endpoints increasing download times.
What can you do about this?
At this point you are probably thinking, “Sure, that’s the result. What do you expect when you use a slow network?” We don’t disagree, but the real question is what you, as the developer, can do to make your application or site, run faster on slow connections to improve the user experience.
Luckily the NV Insights report gives you some pointers. For instance:
Many large images took three times longer to download over the 3G connection. NV Insights found that they can be compressed by 50 percent just by using a different compression algorithm. Add to that the fact that some images are too large to display on a mobile screen anyway, and there is a lot of performance you can gain when you take this into consideration.
Another interesting fact—NV Insights found images that are downloaded more than once. Clearly, that’s not the best idea.
Let’s discuss compression. NV Insights found a 77.5 percent size reduction if compression had been used on some resources. Most likely, all you need to do is modify the compression settings of your web server.
Use proxies better. Instead of using query string parameters, use a fully qualified resource name. This will allow proxies to cache the resources and increase performance.
As clearly shown in this blog, the network the application runs on impacts performance tremendously, especially for mobile and the cloud. Inconsistent bandwidth, high jitter, increased latency and packet loss all work to degrade application performance. While you might not be able to control mobile or most cloud networks, you can build and test apps with these network conditions in mind. This gives organizations the best chance to optimize app performance before the network impacts are felt by users. With Mobile Center 2.50 mobile testing software, adding network conditions to your Appium tests is a breeze. Just configure your Mobile Center and devices once to use Network Virtualization and the rest is easy. There is really no reason why you would not want to add NV to your tests and start improving your app performance.
WW Product and Technical Marketing Manager at HPE Software. Over 18 years of experience with enterprise software, expertise in complex Software demonstration, demo booth design, promotional videos production and customer facing collateral.