Data Model Service (dms)

Description

Domain data model service.
Works on the basis of the domain center entity class `classes`, interpreting each instance as metadata of a new user/project collection of information objects.
Provides a CRUD API for other roles and services to manage instances of user/project collections. Supports collection methods `create`, `update`, `replace`, `read`, collection element `read`, collection `clear`, and `lookup`. Depending on the collection metadata, supports an in-memory dictionary, a distributed object DB, a PostgreSQL server, KAFKA, and the ClickHouse columnar DB. Supports single-threaded, parallel, and asynchronous modes of working with collections. Can guarantee integrity of modification/read/notification and guarantees preservation of the modification sequence in storage and in notifications. Allows temporary or permanent caching for collection elements. Uses a connection pool to the PostgreSQL DB server. By default, up to 10 connections are used for each unique connection point, but the pool size is controlled by the microservice configuration parameter. Pool connections are closed after one minute of non-use.

Works together with wssubscr to notify subscribers about changes (`modelevents.data_changed` class/event type). Supports subscription with filters and masking. Notifies with command adjustment taking the filter into account (for example, create instead of update). Data selection from a collection supports complex filtering, masking, sorting, aggregation, offset, and limiting.

Access via HTTP(S) REST API `/rest/v1/model/…​`, followed by the class name (or path). Management is also possible via the WebSocket API, both with authorization under a user session and via a token.
Access to collections and methods is determined by web server routes. By default, the administrator has access to all paths and methods; the rest is determined by roles (class roles) assigned to the user (user.opts.roles) or to the integration channel (integration_point.opts.roles).

Supports attaching file attachments to entities with `attachment` properties of single or list type.
Attachments are placed in the storages specified in the class, connections to which are configured using entities of the storage class with types related to file storages.

Split into microservices by domains.
Made redundant in Active-Passive mode. Scaled by splitting the domain tree.

Table 1. System characteristics

Code

dms

Operation mode

Storage, service

Redundancy mode

Active-Passive

Site types

Any

Layer

Business logic

Placement

Internal

State preservation and recovery on restart

Yes, except RAM-type classes.

Application

era_dms

Own storage

RAM, Mnesia, PostgreSQL, KAFKA, Clickhouse

Limitations

  • Available within the 'incoplax' information and communication platform.

  • Performance depends on the selected collection storage type. Collection throughput up to 10 thousand requests per second is achievable.

  • Data is not synchronized between sites by the role. Merging through external storage is possible, subject to possible integrity problems and the need to disable the cache.

Parameters

Table 2. Parameters
Name Type Default Description

name

str

required

Name. May consist of Latin letters and digits and must start with a letter.

roletype

str

required

Role type. Possible values: "dms".

ext

json

empty

Additional role options. Contains a JSON object or list.

enabled

bool

empty

Role activity flag. When set to false, the role does not participate in validation and is not started.

limit_of_cores

int

empty

Limit on the number of CPU logical cores used. Default: 0 - unlimited.

logging_default_level

int

empty

Default logging level set specifically for the role instance (microservice type). More

allow_divergence

bool

false

Flag allowing parameter differences in roles of the same group. Default: false - not allowed.

dbconnstrings

array<str>

required

Connection strings to the PostgreSQL server farm of domain DBs, where domain databases with the entity storage of product models are created.
These connection strings are used for classes with storage_mode: 'category' or 'history' that do not have specific PostgreSQL connection parameters and have the 'auto' mode set (the default).

Each domain uses a pool of 10 connections to each DB (unique connection string). A new connection is created only when busy. In idle mode, there are no DB connections.
Each domain uses the original specified string only to create its own DB with the unique name 'era_model_%DOMAIN%', and for operation uses a modified connection string where its own DB name is substituted as the database.

Connection string format: "alias://pgdb_strings/<ALIAS_NAME>", where <ALIAS_NAME> is a specific configuration alias from the pgdb_strings section.

Listing several connection strings provides standby access to the DB server farm: if connection via the first string fails, connection is made via the second, and so on.

On the central site where the MDC role is located, if this parameter is absent, the connection strings specified in the mdc role parameters are used for classes with 'auto' mode.

Each connection string defined in the aliases section lists the following parameters separated by commas:

  • host — host (required);

  • port — port (required);

  • login — login (required);

  • pwd — password (required). Secret: ciphertext when read; the plaintext value is not returned via API and is decrypted by the service when used; a plaintext value can be supplied on input (Encryption and keys);

  • database — database name (required);

  • ssl — (optional) secure connection flag: false | true. If ssl is not used, omit the parameter.

  • auth_method — (optional) authorization mode when using ssl: cleartext, md5, scram-sha-256.

For example:

  • host:192.168.1.101,port:5432,login:erapgadmin,pwd:StrongPassword,database:postgres;

  • host:172.17.0.102,port:5441,login:User,pwd:Pwd,database:postgres,ssl:true;

  • host:10.0.1.103,port:5441,login:User,pwd:Pwd,database:postgres,ssl:true,auth_method:scram-sha-256.

group

int

required

Group number.
As part of horizontal scaling, the role can be split into several groups on a site, in each of which only one instance is active and the rest are standby.
Responsibility is divided between groups both by domain sets and by individual classes of specific domains.
Together all groups serve the full domain set and full class coverage.
All roles in one group must have the same responsibility.
If the peak load on the active instance exceeds the normal limit, it is enough to distribute the load across several instances by separating loaded classes by some criterion, or by splitting into several domain groups, or both, and distributing them across different servers.
An integer from 1 to 9999999.

order

int

required

Order of the role instance within the group.
Determines the order in which takeover occurs in Active-Passive mode.

include_domains

array<str>

empty

List of domain names served by this group.
Any domain of any level can be specified. Specifying a domain causes its subtree to be served on this group and not on other role groups, except for branches mentioned in other groups.
Together, all role groups on the site serve the full domain tree.

domain_group

int

undefined

Domain tree service group number.
Used when service must be split across several instances by domains and by sections at the same time.
To create several different groups serving the same incomplete domain tree but different sections, these instances must have the same value in include_domains and the same value in domain_group.
Thus, the mechanism for distributing subtrees across groups while excluding subtrees of other groups uses the domain_group value with priority over group.
An integer from 1 to 9999999.

max_loading_domains

int

2

Limit on the number of simultaneously loaded domains.
Useful in systems with a large number of domains.
Used to limit CPU load during microservice restart.
The parameter is applied at startup, on microservice restart, and during mass creation of new domains.

max_loading_classes

int

1

Limit on the number of simultaneously loaded classes within one domain.
Useful in systems with a large number of domains.
Used to limit CPU load during microservice restart.
The parameter is applied at startup, on microservice restart, and during mass creation of new classes in a domain.

cache_filter_interval_sec

int

300

Interval for clearing the temporary cache for classes with the corresponding caching mode.
The more often clearing occurs, the less memory the cache occupies and the fewer resources one iteration takes, but the more CPU time is spent on the operation as a whole.
Recommended values from 30 to 600 seconds.

class_readers_count

int

10

Number of processes serving read operations in parallel within one class, if parallel reading is enabled by the class mode.
This includes filter queries.
Minimum value is 2, maximum value is 32.
Useful in systems with a large number of domains.
If the system has many lightly loaded domains, recommended values are 4 or lower.
If the number of domains is small (up to 100), recommended values are above 10.
A microservice restart is required to apply the value.

It is the default value for classes where opts.worker_r_count = 0.

class_writers_count

int

4

Number of processes serving write operations in parallel within one class, if parallel writing is enabled by the class mode.
A write operation involves preparing the object and validating values.
Minimum value is 2, maximum value is 32.
Useful in systems with a large number of domains.
If the system has many lightly loaded domains, recommended values are 4 or lower.
If the number of domains is small (up to 100), recommended values are above 4.
A microservice restart is required to apply the value.

It is the default value for classes where opts.worker_w_count = 0.

stat_enabled

bool

true

Switch for per-minute statistics logging for classes that do not have an individually configured per-minute statistics logging level.
Every minute, operation statistics are collected and logged as several lines with general performance metrics: the number of requests that passed under different conditions, and the minimum, maximum, and average request processing time.
To increase the logging level for a specific class, set the option stat_mode = 'high'. Then statistics on minimum, maximum, and average processing time are output by operation.
To reduce the per-minute statistics logging level for classes, disable this mode and configure the required level for specific classes.

pg_max_connections_default

int

10

Number of threads in the connection pool to the PostgreSQL server used by default.
The pool is determined by the connection string: a separate pool is organized for each server and each database.
For individual classes, the number of threads in the connection pool can be set individually in the class options.

There are separate pools for server services and for client connections. Each expands up to the specified number of connections.
If a connection is unused for more than 60 seconds, it is closed.

query_workers_count

int

4

Number of processes serving dmsquery operations in parallel on the server.
The operation involves executing an arbitrary DB query and then preparing the result, whose size may be arbitrary.
Minimum value is 1, maximum value is 32.
Serves to limit peak loads created by parallel construction of custom reports using the dmsquery request.
It should be commensurate with the project logic, the number of cores on the server where the microservice instance runs, and the composition of other microservices on that server.
A microservice restart is required to apply the value.

sections

array<str>

List of sections in which the instance participates.
A section can be specified in the properties of the class entity, and then only instances that have the specified section in this field are involved in its execution.
The parameter must be identical on instances of the same group (active-passive).

mnesia_storage_mode

str

disc

Default object storage operation mode.
Used by classes of type 'runtime' for which the storage mode is not set individually.
Possible values:

  • disc - full mode, in RAM with disk persistence on all group instances.

  • ram - RAM only (with distributed and transactional support). If all group instances restart or shut down simultaneously, data is lost.

  • disc_only - disk only (noticeably less performant but saves RAM).

mnesia_activity_context

str

dirty

Operational context of the object storage.
Possible values:

  • dirty - each operation is performed locally, and data is asynchronously (without delays) sent to the storage of the passive group instance. Write operation time is about 30 microseconds.

  • transaction - each write operation creates a transaction. Execution time grows to an interval equal to two pings between servers of one group. If this context is set, the dirty_write_enabled option can be enabled for individual classes to increase processing speed many times over.

The default dirty option does not create problems for system operation compared to the transaction option because:
- Simultaneous writing from several instances is not used;
- Operations on one entity are performed strictly in one handler thread;
- Parallel data reading does not create logical conflicts;
- During takeovers and failovers, the last loaded instance is synchronized from the one loaded earlier, thus giving priority to the current data of the active instance;
- When a group splits into islands and two instances are active simultaneously, priority is given to the leader’s data.

takeover

bool

global

Takeover mode switch.

Failover - activation of a lower-priority service instance when the higher-priority service instance (leader of the active-passive group) is detected as unavailable.
Takeover - activation of the higher-priority service instance (leader of the active-passive group) during its recovery, accompanied by stopping the service on the lower-priority service instance.

  • If the mode is enabled, takeover is performed automatically.

  • If the mode is disabled, takeover is not performed; the recovered group leader temporarily becomes a follower and waits for the service to be unloaded on the lower-priority instance.

  • If the option is not set, its value from the current site parameters is used. If it is not set on the site, the value from the general section is used. If it is not set anywhere, takeover is enabled for more accurate load distribution according to the configured priorities.

dmsquery_prefer_recovery

bool

false

Special mode for dmsquery primarily through the PostgreSQL recovery instance.
Serves to offload the current active DB server and reduce the impact of reports on current service processes.

pg_allow_recovery_mode

str

none

Default value for the property of the same name of classes stored in the PostgreSQL DB (category, history).

Possible values:

  • none - load and read only from the PostgreSQL master instance.

  • load_only - class loading at startup and restart, cache loading, and maximum increment calculation can be performed from a connection to the DB recovery instance. Allows loading a class when restarting or migrating the DMS microservice when it is impossible to connect to the PostgreSQL master instance.

  • read_allow - class loading at startup, cache loading, maximum increment calculation, key lookup, and collection reading with filters and aggregations can be performed from a connection to the DB recovery instance. Allows reading collection data when it is impossible to connect to the PostgreSQL master instance, for example during the period when the decision to convert the available recovery instance to master has not yet been made or implemented.

  • read_prefer - class loading at startup, cache loading, maximum increment calculation, key lookup, and collection reading with filters and aggregations are primarily performed from a connection to the DB recovery instance if one is found. Allows offloading the master DB instance by redirecting read requests with filters and aggregations to a less loaded server with a PostgreSQL recovery instance.

The 'load_only' and 'read_allow' modes do not give preference specifically to the master instance; they actually connect to the first available PostgreSQL instance regardless of its status.

Classes for which the property of the same name is explicitly set do not use the configuration setting.

Also, when the configuration value changes, applying the new value requires reloading the active dms microservice instance or the corresponding classes for which the default value applies.

pg_read_timeout

int

60000

Timeout for a read query operation in the PostgreSQL DB (category, history).
Can be used for debugging to read data from tables with a large amount of data where execution time exceeds the default timeout.

reply_worker_count

int

0

Response sending mode switch.
If the value is 0, the response is sent to the request initiator directly by the request handler thread, which, when located on different servers under peak network loads, can cause short-term delays in request processing at random times due to TCP buffer overflow.
Any value greater than zero sets the number of parallel threads for sending requests, thereby freeing handler threads from short-term blocking at the network layer.

The criterion for setting a non-zero value may be detection of high network load between servers close to the limit, together with the appearance in logs of information about delays in class request processing.
If statistical logging is enabled for a class (stat_mode=high), the per-minute statistical report by each operation type shows the minimum, maximum, and average processing time.
A significant increase in maximum (for example, by 4 orders of magnitude) and average (for example, 2-3 orders of magnitude) values relative to the minimums may be caused precisely by periodic waiting for TCP send buffer relief.
At the same time, a slight increase in averages is acceptable because garbage collection occasionally occurs (adding up to several tens of milliseconds to processing time, while weakly affecting averages during the period).

allow_divergence

bool

false

Switch for validating different settings of DMS instances in one group.
Can be used at the configuration administrator’s responsibility, for example, to set different PostgreSQL connection strings for the active and passive instances of a group if the configuration needs to be adapted to the specifics of the PG cluster node layout.

When set to 'true', the validator stops blocking different instance settings, which may lead to loss of integrity.

class_workers_lazy

bool

true

Start class reader and writer processes as needed.
If true, class processes are created on the first operation, and their number grows up to class_readers_count and class_writers_count (or the class opts.worker_r_count and opts.worker_w_count values) as the operation queue grows. Idle classes do not keep processes.
If false, all processes are created when the class is loaded.
A microservice restart is required to apply the value.

class_workers_grow_wait_ms

int

200

Estimated operation wait in the class process queue (queue length multiplied by average operation time); if it exceeds this value for longer than class_workers_grow_sustain_ms, class processes are added. Short bursts of fast operations do not cause growth.
Applies when class_workers_lazy = true.

class_workers_grow_wait_max_ms

int

1000

Queue wait estimate above which class processes are added immediately in the number that brings total wait within class_workers_grow_wait_ms (a burst of long operations).
Applies when class_workers_lazy = true.

class_workers_grow_sustain_ms

int

1000

How long the wait estimate must exceed class_workers_grow_wait_ms before class processes are added.
Applies when class_workers_lazy = true.

class_workers_shrink_wait_ms

int

10

An observation window is considered quiet if the wait estimate within it did not exceed this value.
Applies when class_workers_lazy = true.

class_workers_shrink_window_sec

int

300

Observation window duration for reducing class processes.
Applies when class_workers_lazy = true.

class_workers_shrink_windows

int

3

Number of consecutive quiet windows required for reduction: writer processes are halved, readers are reduced by one; if there were no operations in the window, the last ones are stopped.
Applies when class_workers_lazy = true.

class_workers_shrink_cooldown_sec

int

1800

Class processes are not reduced during this time after they are added.
Applies when class_workers_lazy = true.

Configuration example

Configuration management is performed in an application available to master domain administrators. The application hides the full configuration content, but it is still available through the API.

The configuration contains a section describing all instances of all roles. Parameters are defined for each specific role instance.

Node example
{
  "name": "dms1",
  "roletype": "dms",

  "group": 11310,
  "order": 1
}