Defining the MQTT Web & Device transport mechanism

[NEAL SINGH]

Hi Koch,

Yes, definitely, we don’t want ascii – Juan also said the same last night.

 

The wildcard makes sense, so from my understanding, to enroll a new device, we will need to create the Server Subscribe & Publish topics for each new unique Device and have the Device publish to the same Server Topic that all the devices use  and Subscribe to its own unique Server Topic?

 

Can you elaborate more about how that is setup on the Server and the Device.

 

RE: MessageNumber:  Will that be a 1 Byte rollover counter ?

 

In regards the GPB, Juan says, it wont be  a problem, just define the spec and he will implement it.

 

The data coming from  the Devices must go into a MySQL table and to push data to the devices, there must be an exit Table – does this make sense and are you in agreement with this approach ?

 

Here how I picture an initialization of a new device

1. Each Device will have  a 16 Byte unique number.
2. The Web Administrator will create  a profile for that Unique number on the Web and assign a setup of the number of Pumps, Tags etc and the SiteID and other such details.
3. The uninitialized Device can be invoke to check-in from a keypad enter by the user. If there is a No Setup, it will receive such a message that the LCD will display.
4. Once it see that there is something for that Unique Number (16 byte), it will receive its first download, which will be the Initialization Table (Command 0x01) contained in that will be the SITEID#.
5. Once this is store on the Device, the Device will reset and operate in default mode of knowing its SITEID# and use the Default heartbeat period (set in firmware) to check into the Web.
6. In the Out-Table on the MySQL Db, there will be a series of downloads that will then commence which will become the process  to configure the Device. (The LCD will give  a report of the Progress of the downloads)

 

Now I suspect that there might be the need for separate Topic to achieve the  Initialization of  a New device as the SITEID# would not have been assigned by then.

 

Let know your thoughts on that.

 

Regards

Neal

 

 

 

 

From: Koch de Jager <[hidden email]>
Sent: Friday, June 7, 2019 6:39 AM
To: Neal Singh <[hidden email]>
Subject: Re: Master Control Phase1 Download Tables 06 June2019.xlsx

 

Hi Neal,

 

I suggest we use Google Protocol Buffers to transfer the data. Rather than sending ASCII we will then send binary data which will be smaller where numbers are concerned. Even if we do  not use Protocol buffers, I suggest using binary rather that ASCII as this is easier and faster for the uC to decode.

 

An MQTT message consists of a topic and a payload.

 

We use the SiteID in the MQTT topic. We can also put the command and a unique number in the topic.

 

So if we look at one device talking to the server, it will be set up as follows:

 

Server

- publishes to topic 'Server/SiteID/Command/MessageNumber'

- subscribes to topic 'Device/SiteID/#' (# is a wildcard and means that the server will receive everything starting with 'Device/SiteID/'

 

Device

- publishes to topic 'Device/SiteID/Command/MessageNumber'

- subscribes to topic 'Server/SiteID/#' (# is a wildcard and means that the server will receive everything starting with 'Device/SiteID/'

 

The Command identifies the structure of the payload of the MQTT message. So there is no need for a header, as the recipient derives the structure from the Command in the topic.

 

The MessageNumber is used to identify the specific command and is incremented for each new command. It will then be in the response/acknowledgement of the command, and can be used to detect duplicate messages (if for some reason the message was resent but it was already received by the intended recipient)

 

From what I read above the communication is always initiated by the device.

 

case 1

Device 123

publish 'Device/123/Hearbeat/1'

 

Server has no data

publish 'Server/123/Ack/1'

 

case 2

Device 123

publish 'Device/123/Hearbeat/2'

 

Server has data

publish 'Server/123/Upload/2'

 

Device 123

publish 'Device/123/Ack/2'

 

case 3

Device 123 has data

publish 'Device/123/Upload/3'

 

Server has no data

publish 'Server/123/Ack/3'

 

We need to also look at security. I will do some research on that.

 

This is off the top of my head. Let me know what you think.

 

Regards

Koch

 

 

On Thu, Jun 6, 2019 at 5:07 PM Neal Singh <[hidden email]> wrote:

Hi Koch,

 

If you look at the Tab on this Spreadsheet of Driver_Addr you will see this picture which represence how the data is moved to the device. Data to the Web  was to have a similar structure.

 

Now it was just my idea that we need to have the SiteID etc but all that is really needed is the Packet Type which will be unique to the type data that is in the payload.

In CMD_Addr you will see all the “Packet Types” and what their purposes are and the Index Map will give the locations of where each Command is stored.

So when a MQTT packet arrives, the uC knows where to store the data that arrives.

The MySQL DB  will have a table that will hold all data that is suppose to go out to the Devices.

The Devices in the field will have a heartbeat that checks into the Web every 5 minutes or so and should there be data there to that Device, it must be sent to that device. If that Device has data for the Server, then all that that data will be put it a MySQL table and the important fields will be the Datetime, SiteID, Command (packet Type) and then the Packet and a Status (bool).

 

On the Web we need to be able register many devices, this will involve registering the SITEID with a 16 Byte ID code.

 

 

Regards

Neal