Engineering in the Kitchen

Subsection 5.1.1 Introducing MQTT

MQTT was originally developed for small sensor nodes monitoring the length of an oil pipeline and occasionally reporting data back to a central server via satellite link. As a result, the protocol is based on certain assumptions:

• Keep it simple. The protocol defines a basic mechanism for computers to send data payloads to each other with some simple guarantees. It assumes that you're writing the code for both the sender and receiver, so the payload format and structure is entirely up to you.

• Communication is expensive. Sending and receiving data is costly in terms of power (battery) and possibly dollars (someone has to pay for the satellite!) Therefore, the protocol is optimized for short messages and does not add a lot of junk on top of the data that needs to be sent.

• Many talkers, many listeners. Many sensor nodes may be reporting data, and there may be multiple consumers listening. Reporting nodes should not have to know or care how many consumers are listening; messages should just get delivered.

• Listeners are often sleeping. Battery-powered sensor nodes spend nearly all their time in sleep mode in order to save power, and cannot continuously listen for incoming messages. Some messages may need to be held on the central server until a node wakes up and reconnects.

Subsection 5.1.2 MQTT structure

% Broker / client system % Anyone can "publish" % Anyone can "subscribe"

Subsection 5.1.3 Publishing MQTT messages with MicroPython

Publishing MQTT messages in MicroPython is straightforward using the umqttsimple.py library which you can download here¹ (Note that the download is named simple.py; you probably want to rename it to umqttsimple.py.)

There are three essential functions:

1. First, we create an instance of the client using MQTTClient(client_id, server). The arguments are a client identifier (this can be any identifying string) and the hostname or IP address of the MQTT broker.

2. Second, we initiate the connection to the server by calling connect(). This does not take any arguments; it uses the ID and hostname that the client object was created with.

3. Finally, we can begin publishing messages. The publish() function takes two arguments: the topic and the payload.

Below is a complete example which connects to WiFi and publishes a message. For the topic it uses the clientId string plus the subtopic ip. The payload is the device's current IP address.

import network
from umqttsimple import MQTTClient
from time import sleep

wlan = network.WLAN(network.STA_IF)

# If the network isn't already connected, try to connect
if not wlan.isconnected():
    wlan.active(True)

    # Try to connect to Tufts_Wireless:
    ssid = "Tufts_Wireless"
    print("Connecting to {}...".format(ssid))
    wlan.connect(ssid)
    while not wlan.isconnected():
        sleep(1)
        print('.')

print("Connected!")
print("IP address:", wlan.ifconfig()[0])

# Note that this server is only accessible from the Tufts network
mqtt_server = "en1-pi.eecs.tufts.edu"

# Some kind of unique identifier, esp32-[YOUR UTLN] is good
clientId = "esp32-sbell"

client = MQTTClient(clientId, mqtt_server)
client.connect()

# Identify ourselves
client.publish(clientId + '/ip', wlan.ifconfig()[0])

Subsection 5.1.4 Quality of Service (QoS)

MQTT defines three levels of service, and any client can decide which QoS level to use for its communication with the broker.

• QoS 0: Messages will be received at most once.

• QoS 1: Messages will be received at least once.

• QoS 2: Messages will be received exactly once.

Implementing QoS level 0 is simple: the device fires off the message to the broker (or vice-versa) and hopes for the best. Maybe the message arrives, or maybe it doesn't.

Implementing QoS level 1 requires one more step: The device sends the message to the broker, and the broker responds with an acknowledgment. If the acknowledgment is not received after some time, the device re-sends the message. The device will continue re-sending until it receives the acknowledgment, at which point it knows that the message has been successfully delivered.

QoS level 1 guarantees that the message will reach the recipient, but there is a catch. Suppose the message reaches the recipient, but the acknowledgment gets lost on the return trip. Since the sender did not receive the acknowledgment, it will send the message again — and the recipient will get two copies of the message.

If the message is an updated sensor value, a duplicate probably doesn't matter. But if the message triggers an action, then a duplicate might cause the action to be launched multiple times. Imagine a message that tells a robot to move forward 1 meter: duplicate messages could be catastrophic!

One solution would be to keep a list of messages which have been previously received and processed, and ignore repeat messages as they come in. However, for this to be completely foolproof, the recipient would have to keep a record of every message that it has received. This isn't practical, so MQTT uses another solution to ensure "exactly once" delivery.

With QoS level 2, the sender and receiver engage in a 4-part handshake for each message:

1. First, the device sends the message, just like in QoS level 0 and 1.

2. Next, the receiver replies with an acknowledgment, like QoS level 1. It keeps a record of the message so that it can detect duplicates.

3. When the device receives the acknowledgment, it replies with an acknowledgment of the acknowledgment (called a "release"). This tells the receiver that it can now discard its information about the message. Now that the sender knows its message has been received, it will not try to resend, and the receiver no longer has to worry about getting a duplicate.

4. Finally, the receiver responds to acknowledge the release. When this is received, the transation is complete.

If any of the handshake messages are not received after a specified timeout, they will be retried. However, because the receiver keeps a list of recently-received messages, it can ignore any duplicate messages. And because of the release/complete handshake, the receiver can keep the list of recently-received messages small.

To specify the QoS for a published message using umqttsimple.py, add the qos named argument to the publish() call:

# Publish an MQTT message with QoS level 2:
# Replace TOPIC and PAYLOAD with your topic and payload
client.publish(TOPIC, PAYLOAD, qos=2)

Subsection 5.1.5 Retaining messages

One final feature to note: When a message is published, the sender can mark it to be "retained" by the broker. When the broker receives a message with the "retain" flag set, it forwards it on to any subscribers as usual but then keeps a copy of the message. When a device subscribes, the broker sends copies of any previously retained messages with a matching topic.

If multiple messages are published to the same topic, only the most recent one is retained.

This means that if a device disconnects (perhaps to save power, or due to an unreliable network connection), it will receive the latest retained message(s) when it reconnects.

A message can be marked as retained by adding retain=1 as a named argument to the publish() method:

# Publish a message with the retain flag set
client.publish(TOPIC, PAYLOAD, retain=1)