CallerId and CalledId transformation rule (provider_callerid)

Description

CallerId and CalledId transformation rule for SIP telephony provider accounts.
Used as an adapter between external and internal number plans. In particular, when modifying the CallerId of internal subscribers for sending outside, adaptation to the external number plan occurs, and when modifying the CallerId of external subscribers for sending into the system, adaptation to the internal number plan occurs, that is, preparation of data for the routing process.

The discovered modification rule is applied to modify both From username (number) and To username (number). If modifications need to be separated to reduce the number of rules, a special "priority" indicator can be used as the modifier for To (first the rule for FromNumber is applied, then a rule for ToNumber is searched and applied, but with priority greater than or equal to the original rule and with a filter by the already changed FromNumber).

The configured set of CallerId transformation rules can be tested through the diagnostic API. However, keep in mind that application differs between the test and a call because in a call the To number is taken from Request-Uri, and also when the value matches the entity account username, an empty string is substituted as the number for modification.

Limitations

  • The collection is unavailable in the master domain.

Fields

Entity structure
{
  "id": uuid,
  "priority": int,
  "dir": str,
  "idprovider": uuid,
  "providercode": str,
  "fromdomain": str,
  "fromnumber": str,
  "tonumber": str,
  "modfromdisplay": str,
  "modfromnumber": str,
  "modtonumber": str,
  "ext": {
    "ct": date,
    "lwt": date
  },
  "opts": {
    "title": str,
    "comment": str
  }
}
Table 1. Fields
Specification Description

Field: id
Mode: inout
Type: uuid
Default: generated

Identifier. Can be specified at creation, otherwise generated by the system.

Field: priority
Mode: in
Type: int
Default: required

Rule priority. A lower value means higher priority.

Field: dir
Mode: in
Type: str
Default: required

Filter by change direction.
The rule is applied either to internal subscriber data or to external subscriber data regardless of the call direction.
A call between external subscribers effectively has two dialogs on the esg role, each associating an external subscriber as one dialog leg on the B2BUA role.

Field: idprovider
Mode: in
Type: uuid | str_empty
Default: str_empty

Filter by SIP telephony provider identifier.
Applying this filter significantly speeds up rule search.

To specify a particular provider, first clear the mask in the 'providercode' field.

Field: providercode
Mode: in
Type: str
Default: empty

Filter mask for the SIP telephony provider code.
Filter operation modes.

Field: fromdomain
Mode: in
Type: str
Default: "*"

Filter mask for the call initiator address/domain (From header of the SIP INVITE request).
Filter operation modes.

Field: fromnumber
Mode: in
Type: str
Default: "*"

Filter mask for the call initiator number/username (From header of the SIP INVITE request).
Filter operation modes.

Field: tonumber
Mode: in
Type: str
Default: "*"

Filter mask for the destination number/username (To header of the SIP INVITE request).
Filter operation modes.

Field: modifier_mode
Mode: in
Type: str
Default: "mask"

Operation mode of the from_username, from_displayname, and to_username modifiers.
Allows switching from the normal modifier operation (each field separately with a string modifier) to a service script.

Possible variants:

  • mask - normal operation with separate sequential application of string modifiers to the corresponding values.

  • svcscript - modification of the three fields occurs in one service script whose code is specified in the mod_svcscriptcode field.

Field: modfromdisplay
Mode: in
Type: str
Default: "*"

Call initiator name modifier. The result is passed to the representation process.
Modifier operation modes.

Field: modfromnumber
Mode: in
Type: str
Default: "*"

Call initiator number modifier. The result is passed to the call routing process.
Modifier operation modes.

Field: modtonumber
Mode: in
Type: str
Default: "*"

Call destination number modifier. The application result is passed to the call routing process.
Modifier operation modes.

Field: mod_svcscriptcode
Mode: in
Type: str
Default: empty

Service script code for modifying CallerId and CalledId numbers.

The service script runs once and immediately returns several values using variables with special names:

  • from_username - initiator subscriber number (From header, username field in the URI);

  • from_displayname - initiator subscriber name (From header, displayname field);

  • to_username - dialed number (To header, username field in the URI).

  • add_headers - headers added to the outgoing INVITE request. Optional. Applied only to the first INVITE sent in the new leg.

The variables must be created in the script with the specified names.

Possible value formats of the 'add_headers' variable:

  • each header on a new line.

  • JSON object, keys are header names, values are header values.

  • JSON array of objects. Each object is a separate header. Object keys:

    • key - required, header name;

    • value - required, header value;

    • mode - optional (default 'add'), where to place the header:

      • replace - replaces a header with the same name, so that only one header with the specified name remains as a result of the operation.

      • add - adds after one or several existing headers with the same name;

      • insert - adds before an existing header with the same name.

The script runs synchronously in the routing process and is allotted a maximum of 3 seconds.

According to the ReInvite request pass-through mode selected for the provider account, the script, like string modifiers, can be applied when subscriber data changes on one side.

The following start parameters are passed to the script:

  • 1 - provider account code.

  • 2 - call direction ('inside' or 'outside').

  • 3 - From username.

  • 4 - From domain.

  • 5 - From display_name.

  • 6 - To username.

  • 7 - To domain.

  • 8 - Diversion username. If the Diversion header is absent, an empty string; if there are several, the first (last assigned).

  • 9 - the entire SIP message.

  • 10 - message type ('request' or 'response').

  • 11 - transaction method (e.g. 'INVITE' or 'REFER').

  • 12 - numeric SIP response code (e.g. 200).

Field: opts
Mode: in
Type: object
Compound field

Field: opts.title
Mode: in
Type: str
Default: empty

Arbitrary title

Field: opts.comment
Mode: in
Type: str
Default: empty

Arbitrary comment

Field: opts.tab
Mode: in
Type: array<object>
Default: []

Tabular data for using filters (and modifiers) within one rule search session.
Each object in the list contains arbitrary fields that can be used in the mask as the string {tab:some_field}.
More in Filter operation modes, item 'Table'.

Field: opts.pre_modifier
Mode: in
Type: bool
Default: false

Switch for preliminary number modification mode (to apply the result at an intermediate stage of interaction with an external billing system).
When enabled, normalization rules are executed again, but the already modified request is used as input.

Field: opts.substitute_diversion
Mode: in
Type: bool
Default: false

Switch for substituting the destination number with the number from the Diversion header when searching for and applying a suitable normalization rule.

Field: ext
Mode: inout
Type: object
Compound field

Allows extending the compound with arbitrary keys and values

Field: ext.ct
Mode: out
Type: date
Default: generated

Object creation time

Field: ext.lwt
Mode: out
Type: date
Default: generated

Time of last modification of the object

Rule application direction

Table 2. Rule application direction
Value Description

"inner"

Toward the external party from the internal subscriber. The rule is applied to internal subscriber data when passing it toward the external subscriber.

"outer"

Toward the internal party from the external subscriber. The rule is applied to external subscriber data when passing it toward the internal subscriber.

Filter operation modes

Table 3. Filter operation modes
Mode Description

Per-character

The value being checked is passed through the filter character by character.
The following special characters can be used:

  • X – any character;

  • * – all remaining characters;

  • ? – any character (when used for a domain name mask, any character except the dot);

  • $ – any number of characters up to the nearest dot.

If one of the service characters must be specified as a target, it should be enclosed in square brackets, e.g. [X].

Table

The substring {tab:…​} can be used. It allows selecting a group of characters in the checked value and matching them against the table embedded in the rule (opts.tab).
For example, {tab:a} selects a substring and matches it against the a field of all objects/rows in the table.
The table links several filter and modifier fields. Based on sequential filter checks, fewer rows remain in the table, taking detected matches into account.
The first remaining row is used in modifiers. If no rows remain in the table, the rule is rejected.
Character capture from the checked value based on the mask can occur automatically or with strict specification of the substring length:

  • {tab:KEY} - Automatic capture from the checked value.

  • {tab:KEY:LENGTH} - Capture the specified number of characters from the checked value.

The following can be used as table field values:
* constants;
* regular expressions (to do this, set the value in the format /reg/EXPRESSION, e.g. /reg/^1[1-4]$);
* mappings to other table fields (to do this, set the value in the format /tab/FIELD, e.g. /tab/a). The mode is used to link values of different fields. For example, to allow calls only to one’s own voicemail box. The linking order must be strictly observed: if the referenced table field has not yet been checked and filled with the real value of another entity field, the match will not succeed. Entity field check order: fromnumber, fromdomain, tonumber.
* any value applied without a regular expression (to do this, set /any). The mode is useful exclusively for direct mappings to values of other fields without additional filters.

Using the table modifier is allowed in combination with other control commands of per-character mode.
Repeated specification of fields in one modifier, repetition, and specification of nonexistent fields are allowed.

For example, {tab:a:3}XXXXX{tab:b} allows selecting the 3-character city code in the number and matching it with field a in the table, and simultaneously selecting the tail starting from the 9th character and matching it with table field b.

Regex

The Pattern template is applied to the source value.

Structure of the regex template value:
/reg/Pattern1.

For example,- value: "302"
- mask: "/reg/0"
- result: true.

  • value: "302"

  • mask: "/reg/^0$"

  • result: false.

  • value: "302"

  • mask: "/reg/^302$"

  • result: true.

  • value: "302"

  • mask: "/reg/^(301|302|305)$"

  • result: true.

All standard regular expression rules can be used when forming Pattern templates.

Range

The value being checked is an integer and falls within the specified range of numeric values.

Structure of the dia template value:
/dia/FromValue+N - matches N+1 values from FromValue to FromValue+N.

For example,- value: "302"
- mask: "/dia/300+10"
- result: true.

Modifier operation modes

Table 4. Modifier operation modes
Mode Description

Per-character

The source value passes through the specified modifier character by character with accumulation.
The following special characters and combinations are allowed:

  • ? – capture the current character;

  • * – capture all remaining characters;

  • X – capture the current character (unavailable when modifying DisplayName - the 'modfromdisplay' field);

  • / combined with ? or X – exclude the current character from the result, e.g. /XXX/ excludes 3 current characters;

  • [ combined with ] – include the service character located between the brackets in the result, e.g. [X];

  • {C} – include the SIP telephony provider account code in the result;

  • {c} – include the lowercase SIP telephony provider account code in the result;

  • {U} – include the SIP telephony provider account username in the result;

  • {u} – include the lowercase SIP telephony provider account username in the result;

  • {D} – include the SIP telephony provider account displayname in the result;

  • {d} – include the lowercase SIP telephony provider account displayname in the result;

  • {F} – include the username from the From header in the result;

  • {f} – include the lowercase username from the From header in the result;

  • {T} – include the username from the To header in the result;

  • {t} – include the lowercase username from the To header in the result;

  • {DIV} – include the username from the Diversion header in the result (empty string if absent);

  • {div} – include the lowercase username from the Diversion header in the result (empty string if absent);

  • {E} and {e} – include an empty value in the result;

  • any other character is captured into the result at its corresponding position.

For example,- value: "123456"
- From displayname: "Abonent"
- Code: "R1"
- modifier: "00/X/XX5[*]67{E}8?*{d}{C}"
- result: "00235*678456abonentR1".

Table

The substring {tab:…​} can be used. It allows substituting a substring from the specified table field (opts.tab) into the value being modified.
The source is the first row remaining after filtering in the table.
For example, {tab:ccc} substitutes the value from field ccc of the first remaining table row into the corresponding position of the final result.
If the specified field is absent in the first row/object, an empty value is substituted.

The table links several filter and modifier fields. Based on sequential filter checks, fewer rows remain in the table, taking detected matches into account.
The first remaining row is used in modifiers. If no rows remain in the table, the rule is rejected at the filter check stage.

Using the table modifier is allowed in combination with other control commands of per-character mode.
Repeated specification of fields in one modifier, repetitions, and specification of nonexistent fields are allowed.

Regex (to SIP message text)

The substring {reg:ext.KEY} can be used. Here KEY is an arbitrary key in the 'ext' section of the number normalization rule entity.
This key must be found and must contain a string of the form '/reg/PATTERN/REPLACEMENT/OPTS' or '/reg/PATTERN/REPLACEMENT'; otherwise the string '{invalid}' is substituted as the result.
The PATTERN regular expression with OPTS options is applied to the SIP message body. The result must be found; otherwise the string '{undefined}' is substituted as the value.
The result is taken as the base, and a replacement operation is applied to it using the same PATTERN regular expression and the REPLACEMENT command.

OPTS is a concatenated string of allowed characters:

  • i — case-insensitive search;

  • f — search only in the first line;

  • m — multiline search;

  • s — search where '.' covers all special characters, including end of line (dotall).

For example, let the source number modifier value be: "sample1_{reg:ext.regex1}", and the value of the ext.regex1 key in the rule entity be: "/reg/cont(act):.<sip:(.+)@./\\g{1}_sample2/i".
The first Contact header with its value is extracted from the SIP message. The operation is performed case-insensitively. Then the replacement operation is applied to the obtained header value, that is, the selected field (the username part of the URI) is taken instead of the header, and '_sample2' is appended to it. The result is substituted into its place in the modifier value, and the overall result 'sample1_CONTACTUSERNAME_sample2' is placed as the From URI username in the outgoing leg.

If, as in the example, the From number must be substituted on re-INVITEs, enable From-URI update on re-INVITEs for the provider account.

Substitution of values from the SIP message can also be combined with SIP message modification rules at the border. For example, with full-text SIP message processing in a more flexible mode, a set of values can be collected in one place and a new header formed, which is then used in the normalization rule.

Regex (to source value)

The Pattern template with Opts options is applied to the source field value, and the found block(s) are replaced with the Replace template.
The result can again be passed to the next Regex modification operation, and so on a finite number of times.

General structure of the regex modifier value:
/reg/Pattern1/Replace1/Opts1 /reg/Pattern2/Replace2/Opts2 …​.

Options can be omitted or contain any combination of the characters:

  • i – case-insensitive

  • g – global.

For example,- value: "qwerty,qwerty"
- modifier: "/reg/t/E/g /reg/qwer/a/"
- result: "aEy,qwerEy".

When forming the Pattern and Replace templates, all standard regular expression rules can be used, including capture groups, backreferences, named group substitution, etc.

"priority=…​"

The mode is implemented in task #192.
For To: username modifiers, the value priority=X may be specified as the modifier, where X is any non-negative number.
Thus the CallerId and CalledId modification process becomes:

(1) Rule R1 is searched by the initial From and To.

(2) Its From modifier is applied.

(3) If the To modifier is 'priority=X', where X is any non-negative number, then

(3.1.1) a new rule R2 is searched by the new From and initial To, with priority starting from the specified X.

(3.1.2) the To modifier from rule R2 is applied to the initial To.

(3.2) otherwise the To modifier of rule R1 is applied to the initial To.

The algorithm for building a compact set of CallerId and CalledId transformation rules can be as follows:

(1) Act as usual.

(2) If there is a need to reduce M*N rules to M+N, then

(2.1) Select M From modification rules with high priorities, and set their To modifier values to 'priority=X'.

(2.2) Create N more rules where From filters correspond to the From modification result or *, and set To modifiers.

(2.3) When certain values do not require From modification, either modify To immediately or specify priority=X where X is greater than the current rule priority.