Today we'll install an update for the commands and setting engine, to provide new features:

  • device-level commands: now you can send a command to the device without direct access to the channel where the device is connected. Added a new REST API interface for the device-level commands queue management, similar to the channel-level commands: you can POST command into the queue, examine queue content and check command execution results. The set of the available commands is the same as for the channel.

  • instant device-level commands sending: if your device is connected right now, you can send device-level command by HTTP REST request, which will return its response only when the command was executed on the device. To do this you can specify the optional "timeout" parameter and your request will be suspended until command completion.

Please be noted that during the update installation process you may face an insignificant delay in commands execution.

7 days later
  • Edited

We modified the update with device commands and split various device remote control patterns into multiple methods.
Full listing of remote control methods you may find in the device REST API documentation.

There are four types of remote control methods over device now:

  • device settings - that can be used mostly for persistent device state configuration
  • device commands queue - used for all types of remote commands, where the caller want to asynchronously deliver command to the device once it is connected, commands in the queue are executed in serial mode, one by one
  • real-time command over connected device - REST API method that will check if device is connected at the moment and return false if not. If the device is connected it will deliver the command to the device within specified timeout and wait for response. Once command will be executed on the device, REST API method will return to the caller its actual response. It is also to post multiple commands in one single method and they will be executed one by one.
  • SMS command - this method will just send SMS to the device via available modem and return text of SMS.

All these commands report their status into device logs and all remote operations over device can be retrieved with GET /gw/devices/{device-selector}/commands-result method.

In addition to kial and shal updates, we've added a command field to the GET
/gw/channel-protocols/{protocol-selector}/device-types/{devtypes-selector}
request.
That field contains an array of all commands available to execute directly through the device (real-time, sms commands, and device-level commands queue).
The array of available commands are combined from the channel-level commands and get- and/or set-command per each setting defined for the device.
Now it's possible to execute one device setting several times with different parameters using the device commands queue, and all of the PUT requests will be delivered to the connected tracker.

21 days later
  • Edited

The general purpose streaming protocols http, mqtt, aws_iot, azure_iot, and google_iot have been leveraged with the new convert_message configuration parameter. This parameter is intended for flexible configuration of outgoing messages. This currently makes it possible to change the parameter names generated by flespi, such as 'position.latitude', using one of the available schemes:

  • CamelCase - PositionLatitude
  • lowCamelCase - positionLatitude
  • snake_case - position_latitude
  • {own_pattern} - position{own_pattern}latitude
5 days later

We enhanced channels with the possibility to post-process the message. The list of post-processing channels that are operating on special protocol type is available in the channel configuration:

It works such a way that message from normal device first being parsed by the original protocol which output standard message (JSON) for the channel. After that message is used as a source for input for any post-processing channel that may add or remove any field there or perform some special parsing for fields.

This feature open for us a whole world of new possibilities:

  • add or remove any parameters from the message
  • perform special parsing of 3rd party devices attached to serial port or CAN bus including SigFox and LoRaWAN protocols
  • attach geocoded position, from Google Maps for example
  • attach position detected via LBS service
  • perform any other logic we may need

Right now we are developing few protocols to parse data from peripheral devices attached to the serial port of standard GPS device. These devices are using standard GPS device just as a transport layer to deliver its data.

As usual we are open for our users to request to implement such protocols by our team. These protocols with custom logic can be installed in private mode, available only for particular flespi user.

We are reporting minor changes to the Azure IOT stream configuration.
The previously introduced parameter convert_message has its own rules for working with periods in the names of message parameters. Therefore the special parameter replace_dots is deprecated and should be replaced with the convert_message parameter. Please uncheck replace_dots option, enable convert_message option and select snake_case for replace_dots option.

On September 16, the deprecated stream configuration will be fixed automatically.

6 days later

If you change a non-existent device setting, you will not receive any errors. This is a bug, not a feature. We're going to change this behavior. Thus, any PUT/DELETE request to gw/devices/{devices-selector}/settings/{setting-name} to change a non-existent device setting will return 400 Bad Request.

The update will be installed on October 8th.

  • adsa replied to this.

    We've installed an update extending digital inputs and outputs (din and dout) parameters handling for the protocols, which sends them as a separated bits, not as a combined bitmask.
    In that protocols in addition to parameters din and dout you will get the parameters din.N/dout.N where N is a 1-based bit index, and the parameter value will be true/false according to that bit state.

    SMS gateway updated: + symbol excluded from destination phone number of SMPP client when sending a message

      • Edited

      shal Tomorrow on September 14th we will start the migration of devices messages to a new microseconds format.

      We currently have 40 data clusters with device messages data independent of each other (we call it shares). Each data cluster is serviced by 3 mirrors located on different servers.

      Initially, we will add 4th mirror with a new data format (in microseconds). Once synchronized all read operations from the database will be primarily serviced by these new mirrors and thus return messages with microseconds granularity. Once we ensure that everything is OK, we will stabilize each data cluster (share) with 1 data mirror in a new format and 2 data mirrors will remain as is in the old data format (seconds granularity), mostly maintained for fail-over and backup purposes.

      After a day or two in such mode, we will move further and replace one old mirror with a new one resulting in 2 mirrors in microseconds granularity and 1 fail-over mirror in seconds granularity. In this mode we will operate few more days, up to a week, ensuring that everything is fine and smooth.

      Once we make sure that everything is operating completely as expected, we will make the final transition and replace the last 3rd mirror with a new data format.

      Basically for all users, the database will act on a microseconds granularity immediately once we synchronize the 1st mirror to a new data format, as this mirror will always be chosen as preferable for all read operations. Please be prepared for this. For more information about the changes you should expect please take a look at the quoted post above (posted on July 20).

      We will indicate here in the changelog once the whole transitioning process is completed.

      • Edited

      We've added two useful methods to the flespi REST API.

      The first method is POST gw/devices/{selector}/messages. It is designed to inject your own messages into the device storage. This method is useful for marking up device messages. For example, to mark up driver assignments, you need to register a message with a driver code and create a calculator based on this parameter. It is also suitable for simulating the message flow of a device.

      The registered message is different from the message sent by the device in the following:

      • has a rest.timestamp parameter
      • has no server.timestamp and channel.id parameters.

      The second method is GET gw/devices/{selector}/packets. Along with GET gw/channels/{selector}/packets, this method is useful for viewing the bi-directional traffic flow between a device and a gateway while debugging unexpected device behavior.

      shal We completed 1st and 2nd steps described above and now for all 40 data clusters(shares) there are two mirrors providing messages with microseconds granularity.

      If everything will be OK next week we will replace last 3rd mirror and finish transitioning to microseconds data format, but already starting you can enjoy this feature in flespi database as read operations are automatically routed only to new mirrors.

      8 days later
      • Edited

      Two more useful features have been added:

      1. Now you can configure the interval for polling the device settings using the options:

        • once, by default, query device setting values only once
        • never, never request settings from the device. This is useful for reducing device traffic.
        • daily, weekly, monthly - periodically update the settings
      2. For the convenience of accessing available device commands, we have added a new device field - commands. This field contains an array with the definitions of all available commands.

      We have created a new streaming protocol to integrate flespi with the Polish E-Toll system https://etoll.gov.pl/.
      Stream requirements:

      1. You must be registered as a ZSL provider at https://test.puesc.gov.pl/en/web/puesc/zsl-105.
      2. You must have a valid certificate, private_key and access point URI.
      3. Due to the need for E-Toll, you must provide your own original IP address. So your stream URI should point to the TCP proxy running on your server.
      4. Before you can access the production environment, you must pass the E-Toll certification.

      If you have any questions, please write to us in the Helpbox.

      shal We have migrated all our storage systems from seconds to microseconds granularity according to initially defined schedule.

      We advice all our users using REST API to fetch devices messages to check for correctness of to field in their REST API queries. For example your message is the following:

      {
      "timestamp": 1490347944.893743,
      "din": 5,
      "param1": 1,
      ...
      },

      Legacy implementation of GET /gw/devices/XXX/messages for the query:

      {
      "from": 1490347940,
      "to": 1490347944,
      }

      will correctly return this message, as the storage granularity was one second.
      New (current) implementation will not return this message, because it's timestamp (1490347944.893743) is larger then right selection border (1490347944).
      To avoid any discrepancy please either request 1490347944.999999 as right border (in from parameter) or just select up to next second: 1490347945 and filter out unsuitable values from result.

      shal We've added a new public post-processing protocol parameters-filter that can be used to remove any unneeded parameters (except system ones) from the messages of any channel. For example, it can be used to reduce messages storage amount or reduce traffic when receiving a lot of messages through MQTT or a stream.

      Another example to remove position information (latitude and longitude) from all messages:

      screenshot

      5 days later

      Added functions related to the channel:

      • You can now change the protocol when updating an existing channel.

      • Moving a channel between subaccounts. This action is not integrated into the panel, but the action is available through the PUT gw/channels/{selector}/cid method.

      21 days later

      adsa The fix was released today.

        14 days later
        • Edited

        This week we released a rather interesting platform update related to device plugins. The plugin can insert its own fields into the registered device message. But the limitation was in the parameter name, which was fixed to plugin.{plugin-alias}.{field-alias}.

        We have completely revised this concept and decided not to restrict our users to this naming scheme. We have made the plugin alias optional. Such plugins store their fields as they are, without a prefix.

        The main advantage of this function is to replace any parameter value from the original message with your predefined or calculated using an expression.

        This concept no longer requires a plugin alias. It will be removed in a week. All plugins with non-empty alias and assigned devices will be patched. This will not change the logic of existing plugins.

        8 days later
        • Edited

        We just installed a massive update to the telematics hub for plugins.

        Among all other new features, it is now possible to code any processing of device message using our PVM language, instructions on its usage will be published later, now you may just ask our team for assistance to code something.

        Also now it is possible to specify in plugin list of parameters for removal and also specify timezone for using date/time functions in all expressions/validators. For example, you may activate the removal of coordinates on non-business days to conform to GDPR private driving.

        This is just an intermediate step on the expansion of plugins functionality. A lot of things are still in our TODO list to make them more usable and simpler and we will implement these things one by one. But this step opens a huge list of possibilities to perform over device message whatever you may want - any arbitrary logic and manipulations over fields. And everything can be done by the user with no requirement for the flespi team to perform any development. We are just developing the technology and supporting it which is an extremely scalable approach.

        Remember plugins REST API is still in an experimental mode and we may change it quite quickly so it is better to inform us if you are using something there.