'rest' capability for WebSocket User API

Overview

Provides an interface for CRUD operations on entities of static and dynamic classes. To work, the user must have a role allowing access to the route /ws#rest with the WEBSOCKET method.

When processing requests, it performs additional role authorization for access to specific REST API endpoints for a specific operation. Collection endpoints, specific collection element endpoints, and property endpoints for attachment access are distinguished. Thus, if the user is allowed only the GET method to a class collection by role routes, they can read the collection but cannot read a specific entity or modify and create entities.

Every non-empty change results in notifications being generated to subscribers.

This rest capability does not generate any events.

Accessible classes include:

  • Static collections of the domain center;

  • Dynamic data model collections;

  • Special collections.

Request construction

General request form:
[
  "rest",
  {
    "qid":"16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "...",
    "classpath": "/rest/v1/...",
    ...
  }
]

Required request fields:

  • operation - class operation type (read, create, replace, update, delete, clear, lookup, hc). The hc operation returns in the content field a control value of the collection state that changes on every modification; used to check the relevance of the client cache.

  • classpath - full path to the class collection in the REST API: "/rest/v1/model/builder/packages/Packages",

Specific request fields:

  • interval - list of two dates (period start and end). Required in selection operations from dynamic class collections with storage_mode = history or transactionlog.

  • id - specific entity identifier. When performing collection operations (element selection, lookup, collection clearing), the field must be excluded from the request.

  • content - request content. Required when creating, replacing, or modifying an entity (JSON object), and also for lookup by key (the value to search for — a string or number).

  • hrkey - sets the key for hashring selection of the modification operation handler process. By default the entity identifier is used. Used to parallelize processing of modification and read operations for a specific entity (for classes with integrity_mode: async, sync_fast_read, sync_fast_notify). Binding to a specific handler process preserves the operation order important for the client system during processing and in notifications. A separate key can be used to preserve guaranteed ordering of operations on several different class entities within one super-operation known to the client system.

  • dry_run - true: check without saving for create, replace, update, delete operations of dynamic (dms) and static (dc) class collections. A successful response is marked with the dry_run field. For other operations and collections the request is rejected. More in the article Check without saving.

  • timeout - response wait time in milliseconds. For dynamic class (dms) collections — the collection read request execution timeout (default 60000), after which DMS will not start executing the request if it is still waiting in the queue. For static class (dc) collections — the time during which the web server waits for a response and domain center availability (by default the 'rest_reattempts_sec' parameter value, no more than 120000). After it expires, an error response is sent, but a request already passed to the domain center is not canceled by it: the modification result is unknown in this case. See Retries instead of 5xx.

Selection condition fields - all optional:

  • filter - filter condition. See general-purpose functions.

  • order - sorting order.

  • mask - output fields.

  • offset - offset in the selection.

  • limit - limit on the number of elements in the selection.

  • aggr - aggregation fields. An object with returned fields as keys and aggregation functions as values. As an argument, a superposition of functions can be used, constructed similarly to filter functions. See (aggregation functions).

  • groupby - aggregation base. An object with returned fields as keys and object field names or a list representing a superposition of functions as values. Functions are constructed similarly to filter functions. See (general-purpose functions).

Supported operations:

  • read - selection from the collection, including aggregate.

  • read - if the id field is specified, the response returns data of the specific entity with the specified identifier. Masking (mask) is optionally allowed.

  • lookup - searching for collection element identifiers by key fields. The value to search for is passed in the content field.

  • clear - clearing the collection. Supported only in dynamic class collections.

  • create - creating an entity in the collection with automatic identifier generation.

  • replace - creating or completely replacing an entity with the specified identifier. Supported only in dynamic class collections.

  • update - modifying an existing entity with the specified identifier. Unspecified fields remain unchanged. In dynamic class (dms) collections, field values are replaced entirely. In static class (dc) collections, composite fields allow partial update one level deep.

  • delete - deleting an existing entity with the specified identifier.

  • setup - runtime collection configuration (for example, temporary suspension of change notifications to subscribers, reading and deleting partitions). The operation type and its possible parameters are passed as an object in the content field.

Examples of requests to static and dynamic collections

Seleting elements from a collection:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "read",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "filter": ["or",["like",["property","pstr"],"asdf*"],["==",["property","pint"],8640000]],
    "order": ["pstr",{"pint","desc"},{"id","asc"}],
    "mask": ["id","pstr"],
    "offset": 0,
    "limit": 2
  }
]
Aggregate selection from a collection:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "read",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "filter": ["like",["property","pstr"],"a*"],
    "groupby":{"a":["%",["property","pint"],10]},
    "aggr":{"cnt":["sum",["-",["property","pint"],1]]}
    "offset": 0,
    "limit": 2
  }
]
Searching for elements in a collection by key fields:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "lookup",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "content": "abc"
  }
]
Clearing a collection:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "clear",
    "classpath": "/rest/v1/model/builder/packages/Packages"
  }
]
Reading an entity:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "read",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "id": "c87a8ca6-0f08-91b7-9d19-2fa86f019916"
  }
]
Creating an entity:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "create",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "content": { ... }
  }
]
Replacing an entity:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "replace",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "id": "c87a8ca6-0f08-91b7-9d19-2fa86f019916",
    "content": { ... }
  }
]
Modifying an entity:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "update",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "id": "c87a8ca6-0f08-91b7-9d19-2fa86f019916",
    "content": { ... }
  }
]
Deleting an entity:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "delete",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "id": "c87a8ca6-0f08-91b7-9d19-2fa86f019916"
  }
]
Searching for an entity by key:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "lookup",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "content": "asdfasdf"
  }
]
Runtime collection configuration:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09db22",
    "operation": "setup",
    "classpath": "/rest/v1/model/builder/packages/Packages",
    "content": {
      "cmd": "suspend_notification"
    }
  }
]

Special collections

/rest/v1/domain/nservices

Executing requests to product layer nanoservices.

Getting the list of active nanoservices:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda0915f7",
    "operation": "read",
    "classpath": "/rest/v1/domain/nservices"
  }
]
Response:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda0915f7",
    "data": [
      {"name":"builder.DataService"},
      {"name":"builder.GeneratorService"}
    ]
  }
]
Executing a request to a nanoservice:
[
  "rest",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09472e",
    "operation": "create",
    "classpath": "/rest/v1/domain/nservices",
    "name": "platform.PerfmonService",
    "method": "test",
    "request": {"a":1,"b":"2","cc":[{"d":100,"e":"200"},{"f":300}]}
  }
]
Response to the request:
[
  "rest_result",
  {
    "qid": "16066361-dfc0-49f6-b734-9b3dda09472e",
    "data": {
      "result": "ok",
      "code": "success",
      "response": {"a":"zxcvzxcvzxcv"}
    }
  }
]

Connection setup for executing REST requests

  1. General request form:

[
  "setup_rest",
  {
    "qid": 0.1234,
    "skip_response_entity": true,
    ...
  }
]

Possible options - request fields:

  • skip_response_entity - allows setting, for a specific WebSocket connection, a prohibition on sending the full entity in response to create, replace, update operations. In this case create still sends back the entity with a single "id" field; other requests simply return 'ok'.

Retries instead of 5xx

Requests that cannot be processed (microservice relocation, timeouts, queue overload, etc.) are answered with an error with code 5xx, equivalent to HTTP codes.

The web server configuration parameter 'rest_reattempts_sec' controls the retry mode. Until the configured interval expires, the web server periodically attempts to resend the request to the target microservice. Works only for WebSocket connections, in the DMS and domain center (DC) direction. This allows product microservices interacting with the platform data model not to reinitialize but to wait.

DMS direction

Retries are performed no more than once every 5 seconds. For 5 seconds after sending the request, delivery confirmation is awaited. For 30 seconds after sending the request, a response is awaited.

Side effects:

  • During retries, a number of timeouts may fire in the request initiator logic.

  • If the request was delivered but not processed within 30 seconds (the target microservice has a non-empty DB queue where the request waited for execution), then:

    • if the request has not yet started executing from the queue, it is canceled.

    • if the request is already executing, repeated attempts a) cause additional DB load, making the queue grow even more, b) create and delete requests during repeated executions fail with error 404.

DC direction

Requests to static class collections wait for a response and domain center availability during the request timeout (by default the 'rest_reattempts_sec' parameter value, no more than 120 seconds; value 0 — one attempt without retries). The response is sent no later than this time expires, but not earlier than after 5 seconds of waiting for an attempt. Unlike DMS, the domain center does not cancel the request after timeout expires: the time limits only the web server wait. If no response to the modification operation is received during this time, error 503 is returned with a message about an unknown result; the entity state should be re-read. The response is sent as soon as the domain center responds.

The domain center is considered unavailable if it is not found (restart, switch to a standby instance), not loaded, or has no DB connection. In these cases the request is repeated with a growing pause from 0.25 to 3 seconds. If the domain center process terminated while waiting for a response (restart), only read operations (read, lookup, hc) are repeated. Modification operations are not repeated to avoid executing them twice and are answered with error 503. If the domain center did not respond within the call wait time, the request is not repeated and is answered with error 503: the center is overloaded and is probably still executing the request. Errors returned by the domain center are returned immediately, without retries.

The number of simultaneously executing requests to the domain center of a domain from one web server instance is limited by the 'dc_rest_limit' parameter (default 10). Other requests wait in the queue. A request that did not get its turn within the wait time (but not less than 5 seconds) is answered with error 503 without being sent to the domain center.

The number of simultaneously executing requests to the domain center from one WebSocket connection is additionally limited by the 'dc_rest_conn_limit' parameter (default 5). Other connection requests wait in its own queue and do not delay requests of other connections. The wait time in the connection queue is included in the request wait time. Requests of one connection to one entity (with id specified) are executed strictly in turn: the next is sent to the domain center after the response to the previous one. The order of requests from different connections is not guaranteed.

When the domain center is unavailable, the queue is suspended: availability is checked by a single service request at intervals of up to 3 seconds, and waiting requests are not sent to the domain center. As soon as the domain center is available again, the queue resumes.

Side effects:

  • During the wait, a number of timeouts may fire in the request initiator logic.

  • A request already sent to the domain center is executed by it even after the WebSocket connection is closed.

  • A timeout value of 0 disables retries, but the request may wait in the queue for up to 5 seconds.