This post was written by Istvan Kadar from PerfCore RnD
Several statistics show that the number of IoT devices is significantly increasing. Predictions show that for example in 2025, the number of connected devices is likely to surpass 75 billion. This device explosion is why an industry-standard communication protocol is needed. It must be simple, extremely lightweight, reliable in a network with low bandwidth, and that supports different communication models (1:1, 1:N, M:N).
MQTT (MQ Telemetry Transport) is a publish/subscribe messaging protocol and is described on its official website, mqtt.org, as being ideal for “machine-to-machine” (M2M) or IoT type connected devices. MQTT is known not only for supporting the above requirements but also for minimizing bandwidth and device resource requirements.
The network of an IoT system can be quite complex and the variety of client devices is very large. The more complex an IoT system is, the more important it is to perform load tests. By performing these tests, we can make sure the system can handle all the messages—without losing any data during data processing and/or execution.
A system with a various collection of devices (clients) might be hard to load test manually. This is why Micro Focus decided to add support for load testing IoT devices in the recent Micro Focus LoadRunner 12.55 release, by including the MQTT protocol.
Is MQTT load testing difficult? Not with LoadRunner! The benefits of adding MQTT protocol:
Detect service-capability limits for particular configurations of subscribers/publishers
Check how the broker handles connectivity problems
Test different topics
Although the MQTT protocol is not secure on its own, it can be wrapped with SSL/TLS which is supported in LoadRunner.
The creation of a multiple-protocol script is also supported. MQTT can be integrated into a recorded Web – HTTP/HTML script. For details, see MQTT Overview.
Supported MQTT features in LoadRunner:
The MQTT protocol script supports both subscriber and publisher mode, allowing you to control the Quality of Service and configure the automated connection restoring and keep-alive features (if needed).
The protocol supports both Windows and Linux Load Generators.
How does it actually work?
By creating a load test with LoadRunner, you can monitor how your MQTT broker handles the situation of a heavy load.
First, you'll need to create a test script in VuGen (Virtual User Generator). This script implements clients which connect to the broker—either as a publisher or a subscriber.
Then, creating a test scenario in Controller simulates multiple users based on the test script, running either locally or on remote machines, for a pre-defined time period.
Creating and using virtual users to load-test a server
In one test script, clients can be both publishers and subscribers.
Create your test script in VuGen
Creating an MQTT test script in VuGen is quite straightforward. Select “New Script and Solution” > “Mobile and IoT” > "MQTT (Internet of Things)".
MQTT is a non-recording protocol for scripting only. Therefore, a new script will show a suggested template for all actions (including init and end). This MQTT Overview can be used as a starting point to writing your test script. A full list of the MQTT APIs can be found in the Function Reference. Just modify it in your script as needed.
/* Optional SSL/TLS settings, applicable for SSL/TLS connections only */ mqtt_set_tls_certificate(client, "<path to a certificate file, e.g. cert.pem>", "<path to a private key file, e.g. key.pem>", "<password for the private key>"); mqtt_set_tls_parameters(client, MQTT_TLS_DEFAULT, MQTT_DEFAULT_CIPHERS);
/* Connect to an MQTT broker */ mqtt_connect(client, "tcp://<enter an MQTT broker host name>");
return 0; }
/*Publish a “Hello world!” message to topic: “topicA”.
Execute your test script in a load test scenario using Controller
You can create your load test scenario using the MQTT script you created.
MQTT scripts will report the following measurements by default:
MQTT Message Rate
The e number of messages published and received per second at any given time
The amount of data (in bytes) that the Vusers received from or published to the MQTT broker at any given time
MQTT Active Connections
The number of active MQTT broker connections at any given time
MQTT In-flight messages
The number of in-flight messages on the MQTT broker at any given time during the load test scenario run. In-flight messages are QoS 1 or 2 messages that can be in the process of being transmitted, including messages currently going through handshakes and messages that are being retried
MQTT $SYS Topics
The MQTT broker machine’s system resource usage during the load test scenario run. This can help in isolation server performance bottlenecks
An example of a load test result in Controller:
Data point storing of MQTT messages: Statistic Graphs in Controller
From these results we can conclude, that the script and scenario we executed generated an average publish rate of 7.727 messages/sec, and an average receipt rate of 44.714 messages/sec, with a throughput of 493.332 bytes/sec (receipt) and 82.092 bytes/sec (publish).