# Architecture Guide mDash is a framework for developing and maintaining connected products. The main design goal was to provide a robust and easy to use device management capabilities. mDash's remote control is unprecedented: one-clike firmware updates (OTA), file management, device shadow/twin, remote function calls, access level control, notifications, database, and more. mDash could be used either stand-alone, or together with other cloud solutions - like AWS IoT, Azure, Google Cloud, etcetera. ![mDash Architecture](media/arch.png) In order to launch a product on mDash, a business needs to implement the hardware part - a physical device itself. mDash's reference firmware and mobile app are simple to modify, thus usually there is no need to outsource the development. If, however, you need help - do not hesitate to [contacts us](/home/contact.html). ---
## Authentication mDash uses industry-standard TLS1.2 transport protocol for protecting network traffic. mDash clients use key-based authentication: - **Connected devices**. Protocol: Websocket over TLS1.2. Authentication key: a "device password", generated for eash registered device. A device connects to the mDash using that key, can receive management commands, and send notification (e.g. reporting data). - **Management API**. Protocol: HTTPS/REST. Authentication key: a "master" API key. Gives root access to everything. You, as a vendor, can create/delete multiple keys. Keep them safe. - **Customer API**. Protocol: HTTPS/REST. Authentication key: a "public" API key, generated by a device. Gives a restricted access to that device. The level of access is controlled by per-device ACL settings. > NOTE: mDash also supports MQTT as a transport protocol for devices. > However, we would discourage from using it because of a security reasons. > A single leaked device password can give an access to the entire fleet. > A malicious client can suscribe to the `#` wildcard topic to sniff all > traffic, and also can send commands to any device connected over MQTT. ## ACL mDash provides an ability to access any device by external users, your customers. In order to accomplish that, mDash has an access control list (ACL) associated with every device. ACL defines a access key for the external customers, which values in the devices shadow they can see and modify, which RPC functions they can call, and whether they can access device data. The ACL of any device could be modified via the REST API, by sending a POST request with the `acl` set. Alternatively, that can be done via the mDash GUI - click on a device management link and choose "ACL" tab. Here is an ACL reference: ```javascript { "shadow_read": "...", // Regexp for a publicly visible shadow keys "shadow_write": "...", // Regexp for a publicly writable shadow keys "rpc": "Func1,Func2", // Comma-separated list of accessible RPC methods "public_key": "...", // Public access key "data_read": { // Endpoints in /api/v2/devices/:id/data/QUERY "query1": "...", // In every query, DEVICE_ID is substituted with "query2": "..." // an actual device ID }, "data_write": { // Database queries that this device can call "query1": "...", // via the mDashStore() function - see library API "query2": "..." // for more details } } ``` By default, end-customer device access is disabled. In order to enable end-customer device access: 1. Set `public_key` to some unique, hard-to-guess value - like SHA1 hash. mDash client library does that automatically. With Mongoose OS, there are two ways of setting `public_key`: by directly editing the ACL on mDash, or by setting `device.public_key` in a device firmware if it is empty. 2. Set `shadow_read` and `shadow_write` properties in order to expose device shadow. 3. Set `rpc` property in order to expose certain RPC functions. 4. Set `data_read` property in order to expose device's data to an external customer. The `data_read` is an object, where keys become URL endpoints, and values are SQL queries to your database. In a query, `DEVICE_ID` is automatically replaced with and actual device ID. 5. Set `data_write` property in order to to allow devices store data into the database. The `data_write` is an object, where keys are names and values are SQL queries. Queries can take parameters. Devices should call `mDashStore(name, ...)` API function in order to store data. See [mDash library API::mDashStore](/docs/api/lib.md#mdashstore-store-data-into-the-database) for more details. This is an example of the ACL that enables shadow, RPC and data for the end customers: ```javascript { "shadow_read": "^.state.reported.(online|ota.id|config.*)", "shadow_write": "^.state.desired.(foo|config.*)", "rpc": "Sys.Reboot,MyDevice.*", "data_read": { "query1": "SELECT COUNT(*) FROM data WHERE device_id='DEVICE_ID'", "query2": "SELECT * FROM data WHERE device_id='DEVICE_ID'", "query3": "SELECT * FROM data WHERE topic=?" }, "data_write": { "query1": "INSERT INTO data (timestamp,device_id,topic,message) VALUES (?,?,?,?)" }, "public_key": "dch0bBsVvB8lumlXR0C7Tw" } ``` Get device shadow (note that only keys set in `shadow_read` are shown): ```bash curl https://mdash.net/api/v2/m/device?access_token=PUBLIC_KEY ``` Modify device shadow (note that shadow value matches `shadow_write`): ```bash curl -H 'Content-Type: application/json' \ -d '{"shadow": {"state": {"desired": {"foo": 42}}}}' \ https://mdash.net/api/v2/m/device?access_token=PUBLIC_KEY ``` Call RPC function without parameters (note that `Sys.Reboot` is listed in `rpc`): ```bash curl https://mdash.net/api/v2/m/device/rpc/Sys.Reboot?access_token=PUBLIC_KEY ``` Call RPC function with parameters (note that `MyDevice.Act` is listed in `rpc`): ```bash curl -H 'Content-Type: application/json' -d '{"foo": 42}' \ https://mdash.net/api/v2/m/device/rpc/MyDevice.Act?access_token=PUBLIC_KEY ``` Get device data (note that `query2` is a key in `data_read`): ```bash curl https://mdash.net/api/v2/m/device/data/query2?access_token=PUBLIC_KEY ``` If an SQL query contains `?` placeholders, then it is possible to pass respective SQL query parameters in the request body. Query parameters should be an array and their types should correspond expected query types. Run `query3` pass parameters `["foo"]` to that query: ```bash curl -H 'Content-Type: application/json' -d '["foo"]' \ \ https://mdash.net/api/v2/m/device/data/query3?access_token=PUBLIC_KEY ``` ## Notifications mDash provides a special secure Websocket endpoint `wss://mdash.net/api/v2/notify`. This is a read-only notifications endpoint. Each notification is a JSON object with three keys: - `name`: notification name, e.g. "online", "offline", "rpc.in.GetInfo", "rpc.out.Log" - `id`: an ID of a device that generated the event - `data`: optional notification-specific data. The `online` and `offline` events are generated by the dashboard. The `rpc.out.*` events are generated by the device: these are JSON-RPC requests without an ID (notifications). For example, `dash` library forwards all device logs to the dashboard as `Log` RPC calls, thus generating `rpc.out.Log` events. RPC call to the device generate `rpc.in.*` events. The dashboard UI uses `/api/v2/notify` endpoint in order to catch state changes. Login to the dashboard and open the developer tools / network / WS panel to see it in action. > NOTE: by bringing up a custom notification catcher, you can integrate with > any other 3rd party service - for example, store data into a Google, AWS, > or Azure cloud. See next section for an example. ### Catching notifications You can implement your own service that attaches to the `/api/v2/notify`, for example in Node JS (don't forget to substitute `API_TOKEN` with your real API access token). This is a simple Node.js program that can show all incoming notifications: ```javascript const Websocket = require('ws'); // npm install -g ws const addr = 'wss://mdash.net/api/v2/notify?access_token=API_TOKEN'; // <- Modify! const ws = new Websocket(addr, { origin: addr }); ws.on('message', msg => console.log('Got message:', msg.toString())); ``` To see your mDash API tokens, login to mDash and click on "Keys" tab. ### Sending notifications Notifications are sent by devices, and some notifications (like online/offline) are generated by mDash. The device API is documented at [mDash library API](../api/lib.md). Here is a quick example: ```c mDashNotify("MyStat", "{%Q: %f}", "temperature", 12.34); ``` Note that the format of data is arbitrary - it could be anything: a number, a string, an array, or a complex nested object. It's up to you. In this example, a simple object `{"temperature": 12.34}` is used. The example generates the following notification: ```javascript { "id": "DEVICE_ID", // This is the device ID that generated notification "name": "rpc.out.MyStat", // "rpc.out.EVENT_NAME", in our case rpc.out.MyStat "data": {"temperature": 12.34} // Your event payload - arbitrary } ``` If you start your own notification catcher, you cat perform any custom action depending on a received event. For example, do some analysis, send SMS alert, etc. This is the way to extend mDash functionality. ## Life of a device connection This section explains what happens when a device connects to mDash. 1. **TCP connection**. Device performs TCP connection to mdash.net 2. **TLS handshake**. Device performs TLS1.2 handshake 3. **Websocket handshake**. Device performs Websocket handshake - Device sends `Authorization: Bearer DEVICE_PASSWORD` header to authenticate - If mDash already has alive connection from the device with the same password, that old connection is considered stale and gets closed 4. **Online event**. mDash sends `online` notification 5. **Who are you?** mDash sends `{"id":1,"method":"Sys.GetInfo","params":{"utc_time":1558870586},"src":"$sys/d1"}` message to a device, thus calling the `Sys.GetInfo` RPC method 6. **Setup local time**. A device sets the local time to `utc_time` if it is not already set 7. **Sys.GetInfo reply**. A device replies with the message similar to this: ```javascript { "fw_version": "1.0.18-arduino-10809-pico32", "arch": "esp32", "fw_id": "20190526-102743", "app": "sketch_huzzah32.ino", "status": 0, "uptime": 6, "reboot_reason": "power-on" } ``` 8. **Store Sys.GetInfo reply in shadow state.reported.ota**. mDash stores the received `Sys.GetInfo` reply to the `state.reported.ota` device shadow section, in order to provide essential information about the device even if it is offline 9. **Idle - normal operation**. Device works according to the firmware-specific logic. 10. **Disconnect**. When a device disconnects, this usually happens in a non-clean way (e.g. device power-off). mDash sends TCP keep-alive every 20 seconds, and closes the connection after 3 failed keep-alives. When a connection is closed, mDash sends `DEVICE_ID` message to the `db/offline` topic. Disconnected device automatically goes to step 1 (reconnects) ## Supported matrix | Hardware | Software Framework | | - | - | | ESP32 | Arduino | | ESP32 | Mongoose OS | | ESP32 | ESP-IDF | | ESP8266 | Mongoose OS | | STM32 (L4,M4) | Mongoose OS | | TI CC3220 | Mongoose OS | ## Service limitations | Name | Limit | | --- | --- | | Max message size | 8K | | Max device shadow size | 8K | | RPC call timeout | 30 seconds | | Customer metadata size | 32K | | Data storage size per device | from 100K to 10M and more, see [Database](db.md) |