• omol

  • Post #2
  • Edited

We are updating the ACL API. The goal is to make it up to date with the current REST API and remove "GET method inheritance" constraint for submodules. All the changes will be done in several steps.

Step 1 (13 Feb 2020)

ACL entry for gw/protocols becomes deprecated. Since this time access to protocols list is always open for ACL tokens. You may still use ACL entries for gw/protocols but you will receive an error message for every API call with such entry in addition to data reply. In 4 weeks (11 Mar 2020) gw/protocols entry will be removed from API.

We will:

  • disable GET method inheritance for submodules and allow specifying GET method for submodules
  • add new submodules for ACL entries to correspond with the current REST API
  • drop gw/protocols support for ACL entries

Here is an example with gw/devices entry to show the scope of changes.
Old API:

  • specified GET method was inherited by submodules
  • supported submodules:
    • "settings", with PUT and DELETE methods (no GET)

New API:

  • specified GET method is NOT inherited by submodules
  • supports the following submodules (the exact mapping of the current REST API):
    • NEW "calculate", with POST method support
    • NEW "logs", with GET method support
    • NEW "messages", with GET method support
    • "settings", with GET NEW, PUT and DELETE methods support
    • NEW "snapshots", with GET method support
    • NEW "telemetry", with GET method support

If your ACL contains entries with enabled GET method and enabled submodules, these entries are expanded to keep backward compatibility. Here is an example.

Old version:

{
   "uri": "gw/devices",
   "ids": "all",
   "methods": ["GET"],
   "submodules": [
      {
         "name": "settings",
          "methods": []
      }
   ]
}

Patched version:

{
   "uri": "gw/devices",
   "ids": "all",
   "methods": ["GET"],
   "submodules": [
      {
         "name": "logs",
         "methods": ["GET"]
      },
      {
         "name": "messages",
         "methods": ["GET"]
      },
      {
         "name": "settings",
         "methods": ["GET"]
      },
      {
         "name": "snapshots",
         "methods": ["GET"]
      },
      {
         "name": "telemetry",
         "methods": ["GET"]
      }
   ]
}

Explicitly mentioned or omitted GET method for the specified submodule also opens or closes access to the corresponding MQTT flespi topics:

  • every "log" submodule regulates corresponding "flespi/log/..." topic
  • gw/channels and gw/devices "messages" submodule regulates corresponding "flespi/message/..." topic
  • gw/channels "commands-queue", "connections" submodules regulate corresponding "flespi/state/gw/channels/{channel-selector}/{submodule}/#" topics
  • gw/devices "telemetry", "settings" submodules regulate corresponding "flespi/state/gw/devices/{device-selector}/{submodule}/#" topics
  • gw/streams "channels", "devices" submodules regulate corresponding "flespi/state/gw/streams/{stream-selector}/{submodule}/#" topics
  • gw/calcs "devices" submodule regulates corresponding "flespi/state/gw/calcs/{calc-selector}/devices/#" topic

We will also enable sending error messages for the following cases:

  • ACL REST API usage in an old manner: i.e. won't specify GET method for a submodule when there is GET method specified for an entry
  • sending GET queries to the resource with implicitly enabled GET access (i.e. GET method is inherited for the corresponding submodule in ACL)
    This may trigger false positive errors, but we believe it's better to notify as many users as possible before the final API change.

Step 2 (11 Mar 2020)

We stop sending error messages in response to the REST API calls and disable the GET method inheritance for submodules. From this moment all GET requests are denied unless ACL explicitly allows them. All the specified gw/protocols entries will be removed from ACL as unnecessary.

8 days later

  • shal

  • Post #3

flespi REST API gateway has been enhanced with additional feature: whenever we have a REST request from token that specified subaccount with valid cid (e.g. server process simulates itself as a subaccount with x-flespi-cid HTTP header) we log REST API call quantity and traffic for given subaccount. Previously the call has been always logged exactly under the account its token belong to. This update is related to users of subaccounts management via API and can be used for limiting amount of calls or traffic subaccounts can use.

    13 days later

    We have moved warnings about deprecated REST API usage from "errors" array into the new "warnings" array. If you are performing REST API queries in deprecated format then you will start receiving responses like the following one:
    { "result": [...], "warnings": [ { "reason": "Sample warning message" } ] }.

      7 days later

      Today we have installed update for tokens ACL according to the schedule mentioned in this thread (omol). Now all the GET method inheritance for submodules is disabled and you have to open it explicitly if it is necessary for your workflow.
      All your tokens created and modified before the first update (13 February 2020) will continue work as expected. This change affects only tokens created/modified during this four-week period (13 February - 11 March).

          15 days later

          In order to optimize logs storage we will remove some parameters from a generic log message. Parameters list are below:
          event_origin - parameters origin_id and origin_type allow you to restore it
          event_text - textual representation of event_code

          Changes will be applied at march 30th.

            4 days later

            adsa installed today. The internal log subsystem has also been reorganized to reduce the log selection delay.
            If you notice incorrect log behavior, please contact us.

                3 months later

                We introduced slight non-blocking change to the tokens configuration.

                First of all we added created field which shows their creation time. For all existing tokens for which we do not know the time they were created, we specified last accessed time as creation time.

                Second - we do not update expire field anymore automatically for tokens with ttl specified. We just softly calculate validity state for the token by using both explicitly specified expire field and dynamically calculated created/accessed + ttl fields.

                For more information about time control please refer to the knowledge base.

                3 months later

                • omol

                • Post #10
                • Edited

                We are announcing some changes in token ACLs:

                1. ACL entries "gw" and "storage" will be removed. You should use item-specific entries (gw/channels, storage/containers, etc)
                2. When you specify access for an item and also specify access for its submodules, then access for non-mentioned submodules is denied. If you don't specify access for submodules then ACL behavior will remain unchanged.
                3. All the ACLs created before 8 October 2020 will be updated to new format automatically.
                4. Almost all the flespi users will not be affected by these changes since their usage pattern suits for old and new approaches.

                When?

                Starting from today (24 September 2020) you will receive warning messages if your ACL usage is deprecated. On 8 October 2020 we will update all the tokens and apply changes mentioned above.

                Details

                Abandoning "gw" and "storage" items

                99% users do not use this global items. These items have very wide scope and are not applicable for real life use cases. It is decided to remove them.

                Moving from implicit submodules access to explicit only approach

                How does it work now?

                When you specify access for an item and do not specify access for its submodules, then submodules access is defined by their item (i.e. access is implicitly inherited).

                Example ACL entry:

                {
    "uri": "gw/channels",
    "ids": "all",
    "methods": ["GET", "PUT"],
    "submodules": [
        {
            "name": "messages", 
            "methods": ["GET", "DELETE"]
        }
    ]
}
                • explicitly allows to GET, PUT /gw/channels/{selector}
                • explicitly allows to GET, DELETE /gw/channels/{selector}/messages
                • implicitly allows to GET, PUT any non-specified submodules /gw/channels/{selector}/{logs, connections, etc}
                How will it work?

                When you specify access for an item and also specify access for its submodules, then access for non-mentioned submodules is denied.

                Example ACL entry.

                {
    "uri": "gw/channels",
    "ids": "all",
    "methods": ["GET", "PUT"],
    "submodules": [
        {
            "name": "messages",
            "methods": ["GET", "DELETE"]
        }
    ]
}
                • explicitly allows to GET, PUT /gw/channels/{selector}
                • explicitly allows to GET, DELETE /gw/channels/{selector}/messages
                • denies all the request to non-mentioned modules (i.e. does not allow what is not mentioned)
                How to avoid warnings?

                If you need to use some submodules in ACL and you want to avoid warnings, then you may explicitly deny access for all unused submodules.
                Let's assume you need to GET, PUT all streams and assign devices to them. Then you might have the following ACL entry:

                {
    "uri": "gw/streams",
    "ids": "all",
    "methods": ["GET", "PUT"],
    "submodules": [
        {
            "name": "devices", 
            "methods": ["GET", "POST", "DELETE"]
        }
    ]
}

                To avoid receiving warnings you may explicitly deny access to "channels" and "logs" submodules:

                {
     "uri": "gw/streams",
     "ids": "all",
     "methods": ["GET", "PUT"],
     "submodules": [
        {
            "name": "devices",
            "methods": ["GET", "POST", "DELETE"]
        },
        {
            "name": "channels",
            "methods": []
        },
        {
            "name": "logs",
            "methods": []
        }
    ]
}
                15 days later

                omol We are deferring the final update to 12 October 2020

                    5 months later

                    Some REST API methods like PUT gw/devices or GET gw/devices/messages, etc., multiplied the API call count by the number of items affected. Thus, we tried to protect the platform from overload.
                    But this method was not obvious to platform consumers, so we decided to simplify it - now any REST API call increases the total counter by 1.

                    8 months later

                    • shal

                    • Post #14
                    • Edited

                    Now all REST API call logs (e.g. HTTP access logs) are available for your information in the Platform section of the Toolbox.
                    We log the source of the request, its parameters, content, and response code, size, and duration it took to perform the request.

                    8 months later

                    You can now set "x-flespi-app" header in all your REST API requests to specify any application details you may need. The content of this header (truncated to 64 characters length) is saved in corresponding platform logs messages.


                    2 months later

                    A new feature has been added to the Subaccount's Limits API. You can specify how long the item will be blocked.

                    2 months later

                    • shal

                    • Post #17

                    We implemented REST API to access billing information: https://flespi.io/docs/#/platform/billing
                    Now via API it is possible to:

                    • access to current commercial client billing configuration;
                    • change email recipients for billing messages;
                    • activate/deactivate auto charging using default Credit Card and control data stored in the payment provider (Stripe);
                    • download all issued invoices in PDF and JSON (meta-data) formats;

                    We will soon provide UI in https://flespi.io panel that utilizes that API. This announcement is for such users that want to automate flespi invoicing process non-standard way.

                    4 months later

                    • shal

                    • Post #18
                    • Edited

                    In order to standardize tokens and oauth REST API services together with other platform entities we moved them to a different REST API path.
                    Old: /platform/customer/tokens and /platform/customer/oauth
                    New: /platform/tokens and /platform/oauth

                    Syntax and call parameters remained the same, just "/customer" part was removed from the API call path.

                    Old REST API methods will be available until May 15, 2023. On March 1, 2023 we will mark them as deprecated and they will be highlighted in the platform logs with warnings about deprecated method usage on each API call. We will also detect and personally contact all flespi users that will use old REST API methods after April 1, 2023 to ensure their smooth operations.

                    MQTT tokens and oauth state topics also changed to reflect updated paths:
                    Old: flespi/state/platform/customer/tokens and flespi/state/platform/customer/oauth
                    New: flespi/state/platform/tokens and flespi/state/platform/oauth

                    Both topic paths are synchronously updated now and old topics will be maintained until May 15, 2023.

                    Please update your software accordingly until May 15, 2023. If you have any question do not hesitate to contact us in flespi chat.

                    3 months later

                    shal

                    The methods /platform/customer/tokens and /platform/customer/oauth have been removed as they were deprecated.

                      13 days later

                      We started to publish the unique idents received by the channel to its retained MQTT storage. The topic is flespi/state/gw/channels/+/idents/+

                      2 months later

                      We introduced certain limits for traffic generated by CDNs which may lead to temporary suspension of particular CDN operation. And we activated webhooks limits as was initially published (were not active during experimental period).
                      You may find latest restrictions per each tariff by this link: https://flespi.com/en/docs/restrictions

                      10 days later

                      When an account hits certain limitations, we no longer block the entire account. Instead, we block specific active tokens that have exceeded the limit. For example, if your API token starts consuming too many requests or generating excessive traffic, it will be temporarily blocked for a certain period of time. However, all of your other systems will continue to function as usual. Additionally, you will still be able to log in to the panel to check logs and perform other tasks.

                      When you perform an HTTP request using a blocked token, you will receive an HTTP error 429: Too Many Requests.

                      12 days later