[Changelog] flespi analytics

shal

Here we will post information about flespi analytics new features and bugfixes.

shal

Fixed bug with last interval sometimes reset to null in flespi/state MQTT. Bug may occurred when last interval was inactive.

shal

Again modified trigger event for 'last' interval change. Previous fix wasn't complete.

Modified detection of recalculation time ranges when receiving new message. Previous implementation used too large ranges. The detection of time ranges for recalculation upon receiving of new messages is primary for analytics. In worse case this can lead to problems in intervals detections or splitting them into multiple parts.

adsa

There was a bug in the latest update that affected event triggering system. Thus interval 'created' event triggered for each message. Bug was fixed and update was installed.

shal

adsa By the way, we recommend to disable/enable all devices assigned to calculators in order to correctly recalculate intervals that may be wrong due to bug installed this morning. You can do it with simple REST API calls:

curl -X PUT --header 'Authorization: FlespiToken XXX' -d '{"enabled":false}' 'https://flespi.io/gw/calcs/all/devices/all'

curl -X PUT --header 'Authorization: FlespiToken XXX' -d '{"enabled":true}' 'https://flespi.io/gw/calcs/all/devices/all'

shal

After a long pause we have more updates to the flespi analytics engine.

First of all it is possible to use date/time functions in expressions - month(X), day(X), hour(X), minute(X). This can be used for intervals validation e.g. hour(begin)>8 or just to filter some messages with hour(X)>12 if you need to detect intervals for some shifts only. Any kind of usage of course.

And additionally we have added active MQTT topic for device assigned to calculator. It will contain the same value as last whenever last detected interval is active at the moment. Once it is no more active null will be published into MQTT. active topic can be used as a foundation for analytics-based own notifications system.

shal

Additional new feature: now it is possible to test actual textual parameters values in expressions. For example you can test for non-empty driver identifier with expression ibutton.code!="" or generate intervals when it is changed with expression like ibutton.code == $ibutton.code.

shal

For those who used merge_unknown=true option in calculators and in general in flespi analytics in selectors configuration - we detected a bug that in some very special cases may lead to generation of false intervals contain messages with unspecified by selector values.

In general the logic of merge unknown option has been rewritten and this may lead to different results comparing to previous logic for messages which ON or OFF state according to interval selectors configuration is unknown.

So the bug has been fixed and we recommend you to recalculate currently generated intervals. This is easy to achieve by issuing via REST API any change to calculator configuration or by disabling and re-enabling assigned device.

shal

We have added possibility to access JSON objects and arrays in expressions.

Imagine that you have next message:
{"key.s":{"key2":2, "key3":3}, "key.p":"key2", "key.a": [10, 9]}

In order to access value under "key2" it is possible to use any of next formats (by index, by name or using name from other message parameter):
json('key.s', 0) == json('key.s', 'key2') == json('key.s', key.p) == 2

To access values inside arrays use zero-based index specification:
json('key.a', 0) == 10

shal

Calculators received new parameter - update_period.

It specifies validity period for the generated intervals and covers any update to the intervals - due to new messages or manual parameter changes in calculator. Any interval which end time lays outside of specified update_period (in seconds from current moment of time) will be ignored.

The default value for update_period is 365 days and we recommend to keep in balance next formula:
update_period < messages_ttl (in devices) < intervals_ttl.

For example you may set update_period to 30 days, messages_ttl to 90 days (to keep original messages from devices up to 90 days) and intervals_ttl to 10 years - to keep lightweighed report results for up to 10 years.

One important notice is that if you want to change calculator settings and want system to resync all current intervals according to new settings you should set update_interval >= intervals_ttl to cover whole range.

You can read more details in calculators knowledge base.