HPE Software is now Micro Focus
HPE Software is now Micro Focus
LoadRunner and Performance Center
cancel

Test your MQTT protocol-based IoT system with LoadRunner

Test your MQTT protocol-based IoT system with LoadRunner

hilale

This post was written by Istvan Kadar from PerfCore RnDIstvan.png

 

 

Several statistics show that the number of IoT devices is significantly increasing. Predictions show that for example in 2025, the number of connected devices is likely to surpass 75 billion. This device explosion is why an industry-standard communication protocol is needed. It must be simple, extremely lightweight, reliable in a network with low bandwidth, and that supports different communication models (1:1, 1:N, M:N).prediction.png

                                                     Connected devices installed worldwide

MQTT (MQ Telemetry Transport) is a publish/subscribe messaging protocol and is described on its official website, mqtt.org, as being ideal for “machine-to-machine” (M2M) or IoT type connected devices.  MQTT is known not only for supporting the above requirements but also for minimizing bandwidth and device resource requirements.

In 2014, the MQTT protocol became an open standard maintained by the OASIS group. Nowadays it is considered to be one of the most commonly used protocols in the IoT industry.diagram.png

                                                     Example for an IoT system driven by MQTT protocol

The network of an IoT system can be quite complex and the variety of client devices is very large. The more complex an IoT system is, the more important it is to perform load tests. By performing these tests, we can make sure the system can handle all the messages—without losing any data during data processing and/or execution.

A system with a various collection of devices (clients) might be hard to load test manually. This is why Micro Focus decided to add support for load testing IoT devices in the recent Micro Focus LoadRunner 12.55 release, by including the MQTT protocol.

Is MQTT load testing difficult? Not with LoadRunner!   The benefits of adding MQTT protocol:

  • Detect service-capability limits for particular configurations of subscribers/publishers
  • Monitor internal broker statistics (through $SYS-type topic)
  • Check how the broker handles connectivity problems
  • Test different topics

Although the MQTT protocol is not secure on its own, it can be wrapped with SSL/TLS which is supported in LoadRunner.

The creation of a multiple-protocol script is also supported. MQTT can be integrated into a recorded Web – HTTP/HTML script. For details, see MQTT Overview.

Supported MQTT features in LoadRunner:

The MQTT protocol script supports both subscriber and publisher mode, allowing you to control the Quality of Service and configure the automated connection restoring and keep-alive features (if needed).

The protocol supports both Windows and Linux Load Generators.

How does it actually work?

By creating a load test with LoadRunner, you can monitor how your MQTT broker handles the situation of a heavy load.

  • First, you'll need to create a test script in VuGen (Virtual User Generator). This script implements clients which connect to the broker—either as a publisher or a subscriber.
  • Then, creating a test scenario in Controller simulates multiple users based on the test script, running either locally or on remote machines, for a pre-defined time period.howdoesitwork.png

                                                                Creating and using virtual users to load-test a server

 

In one test script, clients can be both publishers and subscribers.

Create your test script in VuGen

Creating an MQTT test script in VuGen is quite straightforward. Select “New Script and Solution” > “Mobile and IoT” > "MQTT (Internet of Things)".createvugen.png

 Creating a new MQTT script in VuGen

 

For general details on creating scripts, see MQTT Overview.

MQTT is a non-recording protocol for scripting only. Therefore, a new script will show a suggested template for all actions (including init and end). This MQTT Overview can be used as a starting point to writing your test script. A full list of the MQTT APIs can be found in the Function Reference. Just modify it in your script as needed.

Example of a test script:

vuser_init()
{

client = mqtt_create();

/* Optional connection settings */

mqtt_set_client_id(client, "<insert Client ID>");
mqtt_set_credentials(client, "<username placeholder>", "<password placeholder>");
mqtt_set_lwt(client, "<topic>", "<sample lwt payload>", MQTT_AUTOMQTT_DEFAULTMQTT_RETAIN);

 

/* Optional SSL/TLS settings, applicable for SSL/TLS connections only */
mqtt_set_tls_certificate(client, "<path to a certificate file, e.g. cert.pem>", "<path to a private key file, e.g. key.pem>", "<password for the private key>");
mqtt_set_tls_parameters(client, MQTT_TLS_DEFAULTMQTT_DEFAULT_CIPHERS);

/* Connect to an MQTT broker */
mqtt_connect(client, "tcp://<enter an MQTT broker host name>");

return 0;
}

Action()
{

/*Publish a “Hello world!” message to topic: “topicA”.

mqtt_publish(client, "topicA", "Hello world!", MQTT_AUTO, 2, MQTT_RETAIN);

 

return 0;
}

vuser_end()
{

mqtt_disconnect(client);

return 0;
}

Execute your test script in a load test scenario using Controller

You can create your load test scenario using the MQTT script you created.

MQTT scripts will report the following measurements by default:

MQTT Message Rate

The e number of messages published and received per second at any given time

 

MQTT Throughput

The amount of data (in bytes) that the Vusers received from or published to the MQTT broker at any given time

 

MQTT Active Connections

The number of active MQTT broker connections at any given time

 

MQTT In-flight messages

The number of in-flight messages on the MQTT broker at any given time during the load test scenario run. In-flight messages are QoS 1 or 2 messages that can be in the process of being transmitted, including messages currently going through handshakes and messages that are being retried

 

MQTT $SYS Topics

The MQTT broker machine’s system resource usage during the load test scenario run. This can help in isolation server performance bottlenecks

 

 

An example of a load test result in Controller: 

resexample.png

                                Data point storing of MQTT messages: Statistic Graphs in Controller

 

From these results we can conclude, that the script and scenario we executed generated an average publish rate of 7.727 messages/sec, and an average receipt rate of 44.714 messages/sec, with a throughput of 493.332 bytes/sec (receipt) and 82.092 bytes/sec (publish).

 

// Download LoadRunner 12.55 and start your test for MQTT deployments today! //

 

 

  • Performance Engineering
About the Author

hilale