Interval selector of type geofence now is able to control corridors - path of connected points of certain width.
With flespi analytics now it is possible to control next types of geofences:
New function can be used in expressions from now: exists('param-name') will check if parameter named param-name is exists in the message/interval and return 1 or 0.
Typical usage can be the following. For example to detect movement state of the vehicle you can use engine ignition sensor if available or just check if it is moving over the time:
if(exists('engine.ignition.status'), $engine.ignition.status, mileage()>0.005)
Currently analytics expressions support only double precision numbers as values. Even when you manipulate strings, we take their 32-bit hash as a value of string parameter.
But we are moving closer and closer to introduce various value types in analytics expressions. Already new expressions library is used in device plugins and sooner or later we will introduce it and in analytics expressions as well. Once introduced all values during expression evalution will have type attribute.
We will initially support next types: number (double precision floating point), boolean (true or false), string and null. It is not allowed to perform operations on the values of different type. For example you can not add true to 5 or 10 to "abc".
To simplify the transition period already in existing expression evaluations engine we introduced (dummy now) functions that will help you later. First of all this is type checks: isnumber(X), isboolean(X), isstring(X) and isnull(X) that will check if value is of given type. Also we introduced value cast functions: tonumber(X), toboolean(X), tostring(X) to convert value to different type.
And of course it is now also possible to use special keywords: null, true and false in expressions that in a new engine will have correct meaning and type. They are dummy now and true evaluates to one, while false and null evaluates to zero.
We have changed implementation of mileage() function:
Expressions engine in flespi analytics is ready to be upgraded to the next version. Current version of expressions processing engine is called legacy and to be replaced with next version on the next Friday, April 2nd.
The key difference is that we introduce different types of values except for simple number: nulls, strings, and booleans.
This will make different values for some expression evaluations between legacy and next versions. Take for example counter with type=expression and expression="engine.ignition.state". In legacy version it will store 0 or 1 in the interval message and in next version it will store false or true.
Another key difference is that reference to always-valid message parameter ($some.parameter.name) for parameters that are not present in the message will return 0 in legacy version and null in next.
To simplify the upgrade we did our best to synchronize type casts as much as possible and in the current version and for all comparison operators we do automatic type casts to numbers. null and false are cast to zero, true castes to one.
A special set of functions can be used to wrap value and convert it to another type: tonumber, toboolean, tostring.
So what you should do in order to continue to use analytics seamlessly starting next Friday, April 2nd?
We are now evaluating expressions by both versions of libraries and report any differences in internal logs. The only change for the last 48 hours of such operation that we detected is related to different types of counter values like on example above with engine.ignition.status or with expressions that contain tests like "position.speed>0" will evaluate to 1/0 in legacy version and to true/false in next.
We recommend to wrap all such counters with tonumber(X) function: "tonumber(engine.ignition.status)" and "tonumber(position.speed>0)" so that both legacy and next library versions will evaluate expression to the number. And once the expressions engine is upgraded (will be reported in its changelog) you may update your implementation to work with real booleans, strings, or nulls if required.
More information about the next version of the expressions library you may read in the Knowledge Base.
With a new expressions engine a few new functions are now possible to use in expressions:
More details about expressions, functions, and operators available in them can be read here.
We enhanced json(X, Y) function. Now it is possible to use json path as a second argument to access values inside complex structures.
Imagine you have the following JSON as a message:
{"f1":[{"x":"z", "z":"p"}, 102, {"e":[1, 2, 3]}]}
You may access some fields with expressions:
Analytics was enhanced with the new error() function. It will generate an error for expression evaluation when encountered.
And more significant change is the way we now handle if function: if(CONDITION, WHEN-TRUE, WHEN-FALSE)
Before this update both WHEN-TRUE and WHEN-FALSE were evaluated which was the result of such expressions like listed below to fail.
Now depending on the CONDITION only WHEN-TRUE or WHEN-FALSE will be evaluated further.
And expressions samples that are affected by the current update:
REST API requests for calculator management are served with a new internal framework. This shouldn't cause any problems, but if you encounter unexpected behavior, please let us know.
This update allowed any device fields to be used as a selector when assigned to a calculator, i.e.
POST gw/calcs/all/devices/connected=true - assign all connected devices to all available calculators.
The following fields are available as a selector of already assigned devices: name, connected, configuration.
GET gw/calcs/all/devices/name=test* - get all assigned devices whose name matches test * mask.
Be aware that tomorrow we will start upgrade process and move devices messages store from seconds to microseconds granularity. At the same time flespi analytics system will continue to operate with devices on a seconds granularity and thus may not react to some state changes that may be visible in the source messages.
Please read more information here.
We plan to move analytics it to a microseconds granularity a little bit later (October). Please prepare your receivers for MQTT events and REST intervals from analytics system to receive double precision timestamps in begin, end and duration properties for each interval. Also it may be so that within one second there may be generated multiple intervals.
Sample JSON for the old interval:
{
"begin": 1631516960,
"end": 1631516963,
"duration": 3,
"timestamp": 1631516970.987203,
...
}and same JSON for the new interval may look like this:
{
"begin": 1631516960.700201,
"end": 1631516963.874323,
"duration": 3.174122,
"timestamp": 1631516970.987203,
...
}We introduced the change in analytics behavior at the moment when you are assigning a device to the calculator.
Previously only intervals within update_period were calculated. And from now all intervals from time_begin (by default - unlimited) are being generated. But then, when the calculator is changed, the assigned device is disabled/enabled or any new message from the device arrives - only intervals within the update_period are being modified.
Thus with new behavior, once you assign a device to the calculator you end up with the whole device history analyzed and split into intervals. If this is not a behavior you want please use the time_begin attribute during assigning a device to the calculator specifying the exact time from which to calculate intervals.
New function is now available to use in expressions (selectors, counters, plugins, etc) - distance(lat1, lon1, lat2, lon2).
It will calculate distance between two points with specified latitude and longitude coordinates or return zero if unable to calculate or if one of the input parameters is zero or out of bounds.
And a new REST API method GET /gw/calcs/XXX/devices/YYY/intervals/last is available to quickly access the last interval for the device YYY assigned to calculator XXX.
We installed major update to the analytics engine.
There are two new features available for you now.
First of all - analytics, calculators, intervals and all events are now much easier to debug with toolbox. You can find in the logs & messages section of the corresponding calculator next log entries about corresponding events:
The second new feature is about two new events that should make application of flespi analytics in notification handling systems much easier. Initially flespi analytics was designed as a reports engine that provides updates in realtime. That's why we have information about created, updated, deleted intervals and can access the last or currently active interval. But our users often asked us about a simple system that will allow them to catch baisc events like enter/exit geofence, input activation/deactivation, speed control, and so on that are happening to the device right now. For such use cases analytics was a little cumbersome and didn't provide convenient tools. Until now.
So now there are two super-simple MQTT topics that you may use to receive update for the current state of the device:
These topics will reflect the state of the device currently known to flespi. If you receive activated event, it means the interval configured in calculator's selector is active right now. If it was activated somewhere in the past (by uploading blackbox for example) and currently last interval is not active, you will not receive this event.
Some more information you can read here: https://flespi.com/kb/working-with-devices-assigned-to-calculator
If you want to catch these events in your solution you can wrap them as a flespi message in the MQTT channel and forward to your system with standard flespi streams. Here is a good guide for this: https://flespi.com/kb/how-to-stream-intervals
If you have questions, please contact our team via HelpBox - we will be glad to answer them.
We implemented the new interval selector type="inactive". It can be used to detect connection/reporting problems with the device in realtime and historic mode.
In contrast to all other selector types, in the "inactive" selector interval is created even if there is no data from the device. When device does not send the data for the specified period of time, a new "active" interval is created and its "end" time is automatically incremented every 60 seconds if there is still no data. Once a device is back online and delivers messages to the platform, the interval will be marked as non-active with updated and correct "begin" and "end" times of the inactivity.
New feature is now available in calculators configuration - special boolean property update_onchange.
This property controls the behavior of analytics system when calculator configuration is changed.
With TRUE value (default) the system will synchronize all devices intervals according to a new calculator configuration within update_period (10 days by default). So if you change selectors, counters, or validation fields - you will immediately see the effect with intervals created, updated, deleted events. This is standard behavior of flespi analytics from the very beginning.
While FALSE value will prevent any automatic synchronization to currently existent intervals and will only use the new configuration for new device messages processing. This is convenient to use in production level calculators with thousands of devices assigned to prevent flooding of your handlers with intervals update events.
In order to provide our users with precise control we also added new method that will force analytics system to apply current configuration of calculator to a specific range of device messages: POST /gw/calcs/XXXX/devices/XXXX/recalculate. This can be convenient to force intervals synchronization for historical device messages beyond standard update_period.
We tuned analytics system to use update_delay calculator's property to prevent the creation of false-positive short-living intervals.
The system will delay intervals generation which begin time is less then current time minus update_delay.
This can be effectively used in systems where data delivery can be delayed due to network issues or internal device's logic - for example to correctly delay processing of crash events from Teltonika (which are delivered out of ordinary data stream) or prevent false-positive notifications with only partial data sent by the device.
Two new features now available in analytics.
And as a reminder, for those who works with plugins and calculators - we have nice expression testing tool that simplifies a lot expressions construction and evaluation.
