Conversation recovery
Initial task
Servicing each call proceeds through several microservices. Microservices serving media traffic and SIP signaling are scaled and reserved in active-active mode. Each conversation (connection) is bound to specific media gateway instances and specific instances of SIP signaling servicing services.
If the system has several servers, each microservice type is represented by several instances. Thus, when a server or node/microservice is lost, new calls are established bound to other servers and active microservice instances.
The task is to preserve current conversations when various instances of media traffic and SIP signaling servicing microservices are lost.
Under normal conditions system behavior is as follows:
-
When active-passive microservices are lost, their tasks are taken over by secondary instances. For example mgc, mdc, store, callstore, broker, mware, wssubscr, etc.
-
When media gateways are lost, including their restart, the call media context is lost. However, if the SIP user agent server owning the media context remains in operation, it initiates creation of a new context on one of the available media gateways based on the media gateway loss information and substitutes media sessions for both dialog legs. At the same time:
-
the IVR script restores the media context from the beginning of the current component, i.e., background playback is lost;
-
some providers/devices do not support session substitution, in particular RTP ports, which makes it impossible to restore full traffic exchange.
-
-
When any of the SIP signaling services servicing a specific conversation between subscribers is lost, further signaling servicing becomes impossible. At the same time:
-
if the media gateway remains active and available, the media context continues serving media traffic. Thus subscribers can bring the conversation to an end;
-
hold, transfer, whisper and monitor connection interrupt the conversation.
-
The main problem is that when servicing a subscriber in an IVR script or while they are in a waiting queue, loss of the signaling server is fatal for the conversation.
Thus the task is to restore the conversation and some current point in the servicing context when signaling microservices are lost.
On a single-server system, microservice loss is associated with either manual operations, insufficient server RAM, or critical errors in microservice code — all unlikely. On a multi-server system, microservice loss may be associated with loss of individual servers or connectivity to them. At the same time, planned decommissioning of individual servers is provided by Deservice tools (Master domain → Settings → System → Decommission) without involving the conversation recovery service.
How it works
The system stores information about the nodes involved in SIP signaling servicing of each conversation. This context is stored in distributed storage under the maintenance of the callstore microservice, reserved in active-passive mode. The system monitors the availability of all nodes.
When node loss is detected, the coupling among dialogs is calculated. A recovery task is formed for each coupled dialog. Depending on the coupling configuration (which nodes are lost in which legs), a recovery algorithm is selected.
In any case a new dialog is formed, and the end user agents are called by substitution (INVITE+Replaces). Beforehand the system waits for all required nodes, registrations, states, and connects to appear.
The context of the recovered call in the call center is preserved by assigning the same session to the new call.
IVR scripts are user agents, so when they are lost the scripts start either from the beginning or from the last saved point/context.
Configuration
Configuration must be preceded by a configuration project:
-
Duplication and placement of the system SIP microservices across servers.
-
Configuring Virtual Ip or another method of reserving internal subscriber connections (alternative outbound proxies).
-
Reserving provider connections specifically by transferring responsibility to another esg instance. The option of reserving via alternative routing to another account does not provide the ability to further recover the conversation.
After preliminary preparation, the system must ensure the ability to connect and operate all subscribers in all possible variants of partial server unavailability that fall within the threat model. Such configuration ensures the system is ready to continue servicing new calls after detecting changes in the server and microservice composition and adjusting the configuration to the new state.
Now you can proceed to configuring the recovery service.
-
Mandatory. In the configuration enable the option in the 'roles..callstore.sip_restore_enabled' section for all callstore microservice instances.
-
Optional. It is better to leave the default value (1) for the 'roles..callstore.sip_restore_mode' parameter.
-
Optional. The 'roles..callstore.sip_restore_limit_sec' parameter default is 30 seconds; change if necessary.
-
Recommended. Organize routing in the call center queue through dedicated feature code numbers instead of placing the component deep in a nested script relative to the initial call.
-
Recommended. Place context save points in IVR scripts.
-
Recommended. Create several instances of each SIP microservice and distribute them across different servers.
-
Configuration for external leg recovery. For each provider determine the recovery mode and enter it into the provider account settings (opts.sip_restore_mode). If the uplink supports Replaces (RFC-3891), set the 'replaces' mode in opts.sip_restore_mode. Otherwise decide whether to create a new call or refuse recovery.
-
Configuration for internal leg recovery. If connection reservation is configured not via Virtual Ip but via alternative outbound proxies or a DNS name with several IP addresses, enable 'roles..sg.reregister' in the sg instance configuration parameters.
The log of this service is in the active callstore node (/callstore/erl_*.log). The conversation recovery process is accompanied by intensive traffic, which is placed on a common basis in the /sip/trn_*.log logs of the SIP microservices.