Data Model Service (dms)
Description
Domain data model service.
Works on the basis of the domain center entity class `classes`, interpreting each instance as metadata of a new user/project collection of information objects.
Provides a CRUD API for other roles and services to manage instances of user/project collections. Supports collection methods `create`, `update`, `replace`, `read`, collection element `read`, collection `clear`, and `lookup`. Depending on the collection metadata, supports an in-memory dictionary, a distributed object DB, a PostgreSQL server, KAFKA, and the ClickHouse columnar DB. Supports single-threaded, parallel, and asynchronous modes of working with collections. Can guarantee integrity of modification/read/notification and guarantees preservation of the modification sequence in storage and in notifications. Allows temporary or permanent caching for collection elements. Uses a connection pool to the PostgreSQL DB server. By default, up to 10 connections are used for each unique connection point, but the pool size is controlled by the microservice configuration parameter. Pool connections are closed after one minute of non-use.
Works together with wssubscr to notify subscribers about changes (`modelevents.data_changed` class/event type). Supports subscription with filters and masking. Notifies with command adjustment taking the filter into account (for example, create instead of update). Data selection from a collection supports complex filtering, masking, sorting, aggregation, offset, and limiting.
Access via HTTP(S) REST API `/rest/v1/model/…`, followed by the class name (or path). Management is also possible via the WebSocket API, both with authorization under a user session and via a token.
Access to collections and methods is determined by web server routes. By default, the administrator has access to all paths and methods; the rest is determined by roles (class roles) assigned to the user (user.opts.roles) or to the integration channel (integration_point.opts.roles).
Supports attaching file attachments to entities with `attachment` properties of single or list type.
Attachments are placed in the storages specified in the class, connections to which are configured using entities of the storage class with types related to file storages.
Split into microservices by domains.
Made redundant in Active-Passive mode. Scaled by splitting the domain tree.
Code |
|
Operation mode |
|
Redundancy mode |
|
Site types |
|
Layer |
|
Placement |
|
State preservation and recovery on restart |
|
Application |
|
Own storage |
|
Limitations
-
Available within the 'incoplax' information and communication platform.
-
Performance depends on the selected collection storage type. Collection throughput up to 10 thousand requests per second is achievable.
-
Data is not synchronized between sites by the role. Merging through external storage is possible, subject to possible integrity problems and the need to disable the cache.
Parameters
| Name | Type | Default | Description |
|---|---|---|---|
|
|
required |
Name. May consist of Latin letters and digits and must start with a letter. |
|
|
required |
Role type. Possible values: |
|
|
empty |
Additional role options. Contains a JSON object or list. |
|
|
empty |
Role activity flag. When set to false, the role does not participate in validation and is not started. |
|
|
empty |
Limit on the number of CPU logical cores used. Default: 0 - unlimited. |
|
|
empty |
Default logging level set specifically for the role instance (microservice type). More |
|
|
|
Flag allowing parameter differences in roles of the same group. Default: false - not allowed. |
|
required |
Connection strings to the PostgreSQL server farm of domain DBs, where domain databases with the entity storage of product models are created. Each domain uses a pool of 10 connections to each DB (unique connection string). A new connection is created only when busy. In idle mode, there are no DB connections. Connection string format: Listing several connection strings provides standby access to the DB server farm: if connection via the first string fails, connection is made via the second, and so on. On the central site where the MDC role is located, if this parameter is absent, the connection strings specified in the mdc role parameters are used for classes with 'auto' mode. Each connection string defined in the aliases section lists the following parameters separated by commas:
For example:
|
|
|
required |
Group number. |
|
|
required |
Order of the role instance within the group. |
|
|
empty |
List of domain names served by this group. |
|
|
undefined |
Domain tree service group number. |
|
|
2 |
Limit on the number of simultaneously loaded domains. |
|
|
1 |
Limit on the number of simultaneously loaded classes within one domain. |
|
|
300 |
Interval for clearing the temporary cache for classes with the corresponding caching mode. |
|
|
10 |
Number of processes serving read operations in parallel within one class, if parallel reading is enabled by the class mode. It is the default value for classes where opts.worker_r_count = 0. |
|
|
4 |
Number of processes serving write operations in parallel within one class, if parallel writing is enabled by the class mode. It is the default value for classes where opts.worker_w_count = 0. |
|
|
true |
Switch for per-minute statistics logging for classes that do not have an individually configured per-minute statistics logging level. |
|
|
10 |
Number of threads in the connection pool to the PostgreSQL server used by default. There are separate pools for server services and for client connections. Each expands up to the specified number of connections. |
|
|
4 |
Number of processes serving dmsquery operations in parallel on the server. |
|
|
List of sections in which the instance participates. |
||
|
|
|
Default object storage operation mode.
|
|
|
|
Operational context of the object storage.
The default dirty option does not create problems for system operation compared to the transaction option because: |
|
|
global |
Takeover mode switch.
|
|
|
Special mode for dmsquery primarily through the PostgreSQL recovery instance. |
|
|
|
Default value for the property of the same name of classes stored in the PostgreSQL DB (category, history). Possible values:
The 'load_only' and 'read_allow' modes do not give preference specifically to the master instance; they actually connect to the first available PostgreSQL instance regardless of its status. Classes for which the property of the same name is explicitly set do not use the configuration setting. Also, when the configuration value changes, applying the new value requires reloading the active dms microservice instance or the corresponding classes for which the default value applies. |
|
|
60000 |
Timeout for a read query operation in the PostgreSQL DB (category, history). |
|
|
0 |
Response sending mode switch. The criterion for setting a non-zero value may be detection of high network load between servers close to the limit, together with the appearance in logs of information about delays in class request processing. |
|
|
false |
Switch for validating different settings of DMS instances in one group. When set to 'true', the validator stops blocking different instance settings, which may lead to loss of integrity. |
|
|
true |
Start class reader and writer processes as needed. |
|
|
200 |
Estimated operation wait in the class process queue (queue length multiplied by average operation time); if it exceeds this value for longer than class_workers_grow_sustain_ms, class processes are added. Short bursts of fast operations do not cause growth. |
|
|
1000 |
Queue wait estimate above which class processes are added immediately in the number that brings total wait within class_workers_grow_wait_ms (a burst of long operations). |
|
|
1000 |
How long the wait estimate must exceed class_workers_grow_wait_ms before class processes are added. |
|
|
10 |
An observation window is considered quiet if the wait estimate within it did not exceed this value. |
|
|
300 |
Observation window duration for reducing class processes. |
|
|
3 |
Number of consecutive quiet windows required for reduction: writer processes are halved, readers are reduced by one; if there were no operations in the window, the last ones are stopped. |
|
|
1800 |
Class processes are not reduced during this time after they are added. |
Configuration example
Configuration management is performed in an application available to master domain administrators. The application hides the full configuration content, but it is still available through the API.
The configuration contains a section describing all instances of all roles. Parameters are defined for each specific role instance.
{
"name": "dms1",
"roletype": "dms",
"group": 11310,
"order": 1
}
See also
-
WebSocket API for CRUD operations through an integration channel
-
WebSocket API for subscribing to class entity changes through an integration channel
-
WebSocket API for CRUD operations through a user-authorized connection
-
WebSocket API for subscribing to class entity changes through a user-authorized connection