As we know, the IoT is growing rapidly, with a proliferation of machines (or devices) communicating over the new network. Those objects will collect and communicate information mainly via sensors, meaning a set of rules that govern how they can publish and subscribe to data over the Internet is required. MQTT provides a standard protocol for data transfer – and also a useful way of extracting data from other sources, such as network XDRs.
MQ Telemetry Transport (MQTT) is a real-time publish / subscribe protocol that’s optimised for the efficient distribution of data. It’s an efficient way of managing any data, for instance, the filtering of XDRs from network data before adding them to a configuration file that’s exported via the protocol to subscribers that need this information.
MQTT has a long history – it’s widely deployed in machine-to-machine environments and it’s particularly suited to IoT use cases because it’s designed for connections with remote locations where devices with resource constraints or limited network bandwidth are often located These devices have to run over a transport protocol that provides ordered, lossless, bi-directional connections – typically TCP/IP.
Not surprisingly considering the above characteristics, MQTT is already the most used messaging protocol for the Internet of Things (IoT) – which means that it lends itself to other applications, such as XDR filtering and export. Let’s explore MQTT in more detail.
MQTT has long been deployed for both messaging and data exchange between IoT and industrial IoT (IIoT) elements such as embedded devices, sensors, industrial PLCs, etc.
The protocol is event driven. It connects devices in its network via a publish /subscribe (Pub/Sub) pattern. Two parties, the sender (Publisher) and the receiver (Subscriber) are decoupled from each other but communicate via Topics within the protocol. The connection between the two is handled by the MQTT broker which filters all incoming messages and distributes them correctly to the appropriate Subscribers.
This is an important function, particularly in the burgeoning IoT world where robotics and automation are important features of the landscape, and their related use cases present a continuous need to capture and visualise real-time data sourced from sensors and actuators. These components, and the data they generate, provide not only functional information but also a window into how a system is performing, helping to diagnose issues that might arise within it.
Collecting IoT data without MQTT would be a challenge, one previously tackled by custom development and complex tool kits, a sub-optimal approach for familiar reasons (cost, time). As a result, it comes as little surprise that use of the protocol is taking off.
So how does MQTT work? The protocol defines and contains two types of entity within the network. These are:
The protocol organises the information that needs to be communicated into a hierarchy of topics. A new item of data results in the generation of a control message sent to a connected message broker. The broker forwards that information to all clients that have subscribed to the topic in question. This means that the source of the information (the publisher) doesn’t require access to any data on the number or locations of subscribers, and subscribers and the data recipients do not have to be configured with any data about the publishers.
Anomalies are catered for within the protocol. For instance, where a broker receives a message on a topic that has no subscribers, it discards the data unless the publisher has flagged the message to be retained. Otherwise, every client subscribing to a specific topic pattern that matches the topic of the retained message receives a retained message immediately after they subscribe. Clients only interact with a broker, but a system may contain several broker servers that exchange data based on their current subscribers’ identified topics.
Despite the volume of IoT data in play, MQTT is a lightweight protocol with control messages often as little as two bytes of data in size (though control messages can carry as much as 256 megabytes of data if required to do so). There are fourteen defined message types that can be used to connect and disconnect a client from a broker, to publish data, to acknowledge receipt of data, and to supervise the connection between client and server.
Generally, MQTT uses TCP for data transmission though MQTT-SN provides an alternative where different transport mechanisms such as Bluetooth are in play. Connection credentials are submitted in plain text format and any required measures related to security or authentication are generally supplemented via the use of Transport Layer Security mechanisms for encryption.
The MQTT protocol demonstrates (once again) that capturing, monitoring, and utilising data, in this case data sourced from IoT connected devices, is a required ingredient to ensure optimal service delivery and, ultimately, to achieve commercial success. On an even more basic level than that, simply to ensure that everything works. Data sourced using the MQTT protocol communications is critical to IoT service assurance as well as for analysis, troubleshooting, and problem solving.
As the Internet of Things proliferates ever more rapidly, operators need to ensure that they can keep pace with the growing number of devices across their networks. This requires effective system performance and oversight. MQTT thus represents a string most will add quickly, if they haven’t done so already, to their bow.
We use MQTT for exporting data captured from monitoring probes. We can convert any such data to MQTT format – not just data that was originally sent over such interfaces. We enable users to select relevant information from XDRs, which we tag. Users of the monitoring solution then select the tags of interest, which we export in MQTT format to the MQTT broker, ready for further processing. This step requires converting the collected data to a text format such as JSON or CSV before it is published to an MQTT broker.
Making sure that monitoring and its related optimisation challenges are met is a foundation stone for IoT performance. At Utel, we’d welcome the opportunity to further discuss with you how we can enhance the insights you may already secure from your network, with filtered, mediated information, ready for processing.
So, if you’d like to learn more, please get in touch directly by clicking here.