Properties for IoT Messages in Azure Stream Analytics
In this post I want to show how to use properties that are added to messages that IoT devices are sending to Azure IoT Hub in Stream Analytics. And while talking about properties, let’s even use message enrichment 🙂
Sample Message
The green properties will be added by the Message enrichment feature of IoT Hub, as the data is not most likely not known on the IoT device or does not need to be transferred with each message.
1 2 3 4 5 6 7 8 9 10 11 12 13 | { "body": { "messageId": 2300, "temperature": 28, "humidity": 66 }, "enqueuedTime": "2020-05-08T09:55:24.886Z", "properties": { "temperatureAlert": "false", <span class="has-inline-color has-vivid-green-cyan-color">"CustomerName": "Microsoft Deutschland GmbH", "CustomerId": "4711"</span> } } |
Sample IoT Device
This message is sent by a sample C# client. I used this one: https://github.com/Azure-Samples/azure-iot-samples-csharp/tree/master/iot-hub/Samples/device/MessageSample
The code that sends the message with the alert property has been adjusted to this:
1 2 3 4 5 | string dataBuffer = $"{{\"messageId\":{count},\"temperature\":{_temperature},\"humidity\":{_humidity}}}"; using (var eventMessage = new Message(Encoding.UTF8.GetBytes(dataBuffer))) { eventMessage.Properties.Add("temperatureAlert", (_temperature > TemperatureThreshold) ? "true" : "false"); |
Configure IoT Hub
Device Twin
In most cases the IoT (Edge) device does not know which customer it is associated, as it does not need to know. For further processing of the data – or for device management – this information is relevant. Therefore we add this information to the device twin in Azure IoT Hub.
1 2 3 4 5 6 7 8 9 | "version": 3, "tags": { "customer": { "id": "4711", "name": "Microsoft Deutschland GmbH" } }, "properties": { |
The property names do not need to match the desired properties that will be added via message enrichment. You can choose a structure that fits best.
Message Enrichment
We want to add the customer number and id from the device twins to the message before it is being passed along to an endpoint.
As you can see the name of the property that is added does not need to match the name of the twin properties. Make sure you add the message enrichment to the right endpoint(s). You can decide to add different properties to messages that are routed to different endpoints.
Azure Stream Analytics
In the Stream Analytics job we use a SQL like query to filter the incoming message stream and route the messages to endpoints. The query will work fine as long as you use only the columns that are in the body of the messages (like “temperature” or “humidity” in this examle).
To be able to use the values in the properties, we need to use the GetMetadataPropertyValue function. Please take not of the sentence on the docs page: This function cannot be tested on the Azure portal using sample data
Query
1 2 3 4 5 | SELECT GetMetadataPropertyValue([IoTHub-Messaging], '[User].[temperatureAlert]') AS temperaturealert, GetMetadataPropertyValue([IoTHub-Messaging], '[User].[CustomerName]') AS customername, GetMetadataPropertyValue([IoTHub-Messaging], '[User].[CustomerId]') AS customerid, * |
The first three columns are our property and message enrichment columns while the other columns are all added as well.
Output
Let’s assume we want to add all message to a storage account where the customer id is part of the path.
This will work, as we added the customerid column in the query and it can be used for the path. Remember this is a demo and we only use the customerid as part of the path.
In the architecture diagram at the beginning of the post an Alert route is drawn. You can achieve this by adding a second query to the job which routes certain messages to that output.
