Step 4: Changing the configuration
You need to make changes in the configuration, for example, add a server, redistribute roles, or change their parameters. It would seem that it’s easier to put a new configuration file on all servers and restart the system. Then ServerShell will do everything as it did at the boot step (step 3). You could even choose not to restart, counting on ServerShell to keep track of file system changes on a regular basis. However, there are three significant drawbacks to this approach:
-
If there are a lot of servers, there’s a lot of work, especially considering the possible difficulties of accessing remote sites.
-
If any servers are unavailable, the operation will have to be postponed, as otherwise there is a risk of significant and harmful misalignment of individual system clusters.
-
If there is an error in the configuration file, the configuration cannot be applied. This refers to both static errors and dynamic errors - not every change can be applied immediately.
That is why there is an API for working with configuration files. This API is not a domain level API, it is a system level API, so only administrators of the master domain can use it.
Case: an administrator in the master domain downloads and applies a new configuration file with customization changes, and the system acquires a new holistic state as a result of its application.
The solution to this case study is a set of roles jointly responsible for configuration: MIC (Master Infrastructure Configurator) and IC (Site Infrastructure Configurator).
First, similar to step 1, the configuration file uploaded by the administrator gets through WS to MIC. There it is saved and awaits further administrator activity. The administrator verifies that the uploaded file is correct, validation is performed by the same MIC role. Then, if successful, the administrator applies the file with an API request
/api/admin/v1/cfg/apply
. The MIC initiates the complex process of synchronizing the file
configuration file on all sites, servers, and
_The process reaches every working role so that they have the opportunity to apply the new settings in a hot way. This process reaches every running role so that it can apply the new settings in a hot way.
Configuration application is a tree branching process.
-
MIC sends a command to synchronize the configuration file to all sites - roles IC.
-
IC sends a command to synchronize the configuration to all servers - the ServerShell service process in the root node.
-
ServerShell calculates the difference and applies the changes, unloading unnecessary nodes and launching the necessary ones. It may have changed the names of nodes - they are formed automatically on the basis of the roles executed in them. It sends a command to the other nodes of its server to apply the configuration, namely to the BasedRole service process in each node.
-
BasedRole calculates the difference and applies the changes. The name of nodes depends on the composition of roles, so the composition could not have changed, therefore BasedRole sends a command to apply the new parameters to all roles running on the current working node.
-
Each role independently analyzes the changes and applies its new startup parameters, either hot or by rebooting.
Each of the nodes of the considered process of configuration synchronization sends a command to all its subscribers, and itself accordingly subscribes to the parent node. Thus, the responsibility of the child node is to detect the parent node by means of the configuration and perform the subscription (see figure).
MIC and IC roles like the others, which means their basic behavior is subordinate to their local ServerShell and BasedRole. And the target functionality keeps them at the top of the hierarchy of the configuration management process. (see pic above). There are two points of view on the connection of configuration services, two levels of connection: From the point of view of the process of subscription and configuration update the main MIC, and further MIC → IC → ServerShell → BasedRole, and from the point of view of system startup the main MIC → IC → ServerShell → BasedRole, and from the point of view of system startup the main MIC → IC → ServerShell → BasedRole ServerShell, and then ServerShell → BasedRole → Role, including MIC and IC. It is non-trivial to adequately draw a flat compact diagram reflecting both views simultaneously, but it is more than realistic to abstractly represent this model:
ServerShell starts BasedRole, BasedRole starts MIC among other roles, it starts working and provides configuration management functionality. Similarly IC on its server. If suddenly the configuration changes, MIC notifies all available ICs, IC notifies all available ServerShells on the site (including its own), ServerShell manages working nodes and notifies BasedRole (including on the MIC node). The MIC role can then change its location and behavior. As long as the IC is not running as a role, all ServerShells make unsuccessful attempts to connect to it based on the current configuration. These attempts are asynchronous and do not interfere with the intended process of starting working nodes and the roles in them.
MIC – special role, it must be present on exactly one site. The site that hosts the MIC is called the master-site. And on the other hand, if a site is a master site, it must have MIC. Other sites are called working sites. That said, if a site (whether working or master) has all the composition of the required roles, then it is called a communication site. Thus a communication site is a sign of a product site’s full functionality. Any working site must be a communication site_.
The diagram shows the hierarchy of subscription to configuration changes using the example of a binary branching tree (two sites, two servers, two roles each):
In the bottom layer of the diagram components are BasedRole, but we remember that they in turn control working nodes with functional roles. Thus, one of the BasedRole shown in the diagram controls the startup of a node with the MIC role, and two more control the startup of nodes with IC roles. You should think carefully about this model before moving on.
Disconnected servers and sites
If any node in the system is unavailable at the time the configuration is applied, this will not prevent the configuration from being applied. All available nodes will switch to the new configuration, and those that are unavailable at the time of connection to the cluster will connect based on the previous configuration file to the parent node (in particular, the connected site to the MIC from the previous configuration, the connected server to the IC from the previous configuration). The parent node will immediately give the new configuration and the newly connected node will receive the new configuration of itself and the rest of the infrastructure. The same thing will happen if a new site or server is connected to the system.
First configuration
What is the configuration originally? After all, the configuration change subscription tree must exist and be active.
When installing the program on each new server, the parameters specify nothing but the address of the server on which the MIC role (as the name of the node). Thus, the server with the MIC role is started first in order to further connect to it. It automatically has a basic configuration of three roles: MIC, DC, WS. Each newly installed server acts as an analog of the IC in the sense that it is a direct subscriber of the MIC to configuration changes. And only at the moment when it discovers itself in the configuration, it will start up fully and reorient itself to the IC of the current site.
When the current configuration already contains sites and active IC roles, either MIC or any available instance can be specified as the connection point for new servers IC.
About the difficulties in changing the configuration
There are some minor difficulties with reconfiguration, so that not every configuration can be applied to the current one. For example,
-
It is not possible to move MIC to another server in one turn. However, it can be done in two turns within a master site.
-
There are difficulties in moving MIC to another site.
-
There are difficulties with changing server IP addresses.
-
There are a number of roles that, similar to MIC, can only be moved to other servers in two moves so that no data is lost.
-
There are difficulties with changing site names.
-
There are difficulties with changing role identifiers.
-
There are difficulties in moving roles to which domain entities are explicitly attached.
-
It is not possible to change the name of the master domain.
"Apply in two steps" means: create an intermediate transitional configuration based on the current and target configuration, apply it, wait for the required servers to complete its application (all roles will start and march - this process may be slow due to the need for data synchronization), and then apply the target configuration.
In order not to face these difficulties it is necessary to think over the future state of the system before the start of implementation. It is recommended to make a project passport, where to approve the master domain, the master site, the names of all sites, to fix the range of IP-addresses for the servers of the future system and to model the distribution of roles on the servers. Expanding the system is easy, narrowing is more difficult, moving and changing some configuration parameters is not just difficult, but sometimes impossible without a complete reconnection of one server at a time. However, even in this case, entities in the domains will be preserved and can be loaded from the DomainDB.
term | Determination |
---|---|
|
! |
|
! |
|
! |
|
! |
|
! |
-
What prevents configuration changes from being made directly on the servers?
-
In which domains is configuration management possible?
-
Which role, is responsible for configuration management?
-
What roles are involved in applying the configuration?
-
What services/roles are present in both the configuration management tree and the system startup tree?
-
Under what condition will disconnected servers be able to connect to the system with the new configuration?
-
At what point does the first (base) configuration occur?
-
Next step: Step 5: Synchronize data in domains between sites
-
Previous step: Step 3. multi-server system