Codecs and media negotiation
Summary
-
Call codecs are negotiated by SDP exchange using the offer/answer model (RFC 3264): one side offers a list of formats (offer), the other answers with those selected from it (answer).
-
The platform tries not to transcode: when passing the offer to the other leg, formats known to the platform from the incoming SDP remain first in their order, and additional codecs are added after them (except
--*and the esg offer into the platform — see opts.payloads_audio). -
The composition and order of additional codecs is set by the payloads_audio_offer role parameter. The base codecs
PCMU/8000,PCMA/8000, andtelephone-event/8000are always offered unless explicitly excluded (except the esg offer into the platform, the provider leg withopts.payloads_audio, andaudio_formatsof recording servers). -
For a provider the opts.payloads_audio codec list is optional, and if set — exact: only the specified codecs in the specified order are offered on the provider leg. Applied only when servicing media through the bgmg media gateway.
-
The answer sent by the platform contains a set of codecs or one codec depending on the allow_ac_set parameter.
-
Video in SIP calls is generally passed through by the platform without decoding.
Terms
- Leg
-
A call segment between two nodes. Each signaling microservice on the call path services two legs — incoming and outgoing. A call of an internal subscriber to a provider (subscriber → sg → b2b → esg → provider) has 4 signaling legs. There can be as many media legs as there are media gateways in the processing chain — from 0 to 3, depending on tasks and settings.
- Format (fmt, payload type, PT)
-
The number under which the codec is specified in the SDP
m=line. Numbers 0-95 are static and assigned to codecs (0 — PCMU, 8 — PCMA, 18 — G729, etc.), 96-127 are dynamic and linked to a codec by thea=rtpmapattribute. - Termination
-
The connection point of a leg stream to the media gateway. Each media leg has its own termination with its own local and remote SDP.
- Transcoding
-
Re-encoding audio from one codec to another by the media gateway. Depending on the codec combination it may require additional resources: for example, repacketization between PCMU and PCMA is table-based and almost free, while encoding to G729 or opus is noticeably more expensive.
Supported codecs
Codec names in parameters are specified in Name/Freq format; case does not matter.
| Codec | Default PT | Note |
|---|---|---|
|
0 |
G.711 μ-law. Base |
|
8 |
G.711 A-law. Base |
|
3 |
|
|
9 |
Wideband |
|
13 |
Comfort noise. Not used as the main codec |
|
18 |
|
|
97, 98, 99 |
Dynamic |
|
101 |
DTMF signals per RFC 2833/4733. Base. Not used as the main codec |
|
102–105 |
Dynamic |
|
106 |
Dynamic |
|
107 |
Dynamic |
Video: H263/90000 (34), H263-1998/90000, H264/90000, VP8/90000, VP9/90000 (dynamic).
Unknown audio formats are discarded when SDP is processed for a media gateway with transcoding; unknown names in parameters are ignored. Unknown video formats are not discarded: video is passed through, see Video.
Base codecs
PCMU/8000, PCMA/8000, and telephone-event/8000 are always offered, even if not specified in payloads_audio_offer (except the esg offer into the platform, the provider leg with opts.payloads_audio, and audio_formats of recording servers). Thus different roles of one platform instance with different settings (for example, ivr3 and b2b2) always have a common codec and do not answer each other with the final unsuccessful 415 Unsupported Media Type response.
A base codec can be explicitly excluded: -PCMU/8000 — do not add PCMU, -* — do not add any unspecified base codec.
Parameters
payloads_audio_offer, payloads_video_offer (roles)
Parameters of the b2b, conf, esg, ivr, prompt, sg roles. Applied when a role instance forms an offer: creates a new outgoing call or passes an incoming call offer to the other leg. sg applies the parameter only to the SDP of the OPTIONS answer.
| Element | Meaning |
|---|---|
|
The codec is offered; its position in SDP follows its position in the list |
|
The codec is not added (usually a base one) |
|
No unspecified base codec is added |
|
The codec is not added and is excluded from the incoming call SDP passed to the other leg |
|
Like |
Final list: first the specified codecs in list order, then unspecified and non-excluded base ones. CN/8000 and telephone-event/8000 are placed at the end. If the parameter is not set or empty, only base codecs are offered.
For video there are no base codecs: the list defines composition and order.
opts.payloads_audio (provider)
The provider opts.payloads_audio field.
-
Media is not serviced (the provider
mediafield is off, bgmg is not used): codecs are not applied to provider calls, SDP passes through esg unchanged. The field must be empty. -
Media is serviced through bgmg, the field is set: only the specified codecs in the specified order are offered on the provider leg, and the media gateway transcodes if necessary;
telephone-event/8000is added at the end unless excluded by the-telephone-event/8000or-*element. The list applies only to the provider leg. A call, re-INVITE, or INVITE with Replaces from the provider is accepted if its SDP contains a codec from the list, otherwise esg answers 488 Not Acceptable Here (re-INVITE is rejected without dropping the call). The answer to the provider contains the list codecs from its offer in list order; withopts.allow_ac_set = false— one: the one already used by the other leg, otherwise the first in the list. If the INVITE arrived without SDP, the inward offer is built from the list. The offer into the platform contains the provider codecs from the list in provider order (telephone-event/8000— if not excluded), so transcoding to devices is performed by the b2b media gateway, not bgmg. The list codecs must be supported by the mg media gateways (codecsparameter) through which b2b calls are serviced. Internal legs are not restricted by the list: if the inward offer contains no list codecs, bgmg transcodes. -
Media is serviced through bgmg, the field is not set: esg passes SDP unchanged, the media gateway passes media without transcoding — this reduces bgmg load. If the INVITE, re-INVITE, or INVITE with Replaces contains no main codec known to the platform, esg answers 488 Not Acceptable Here (re-INVITE is rejected without dropping the call). The
payloads_audio_offerof the esg instance is applied only in two cases:-
the provider reinvite field has value
0: re-INVITEs are not forwarded, esg answers them itself, and the media gateway can transcode. Takingpayloads_audio_offerinto account, INVITEs to the provider and INVITEs to the new target on provider-side transfer (REFER), which esg services, are built; the provider offer into the platform is passed with its codecs in its order (formats unknown to the platform are discarded). The esg answer to re-INVITE and INVITE with Replaces contains only the offer codecs; if there are no common ones — 488 Not Acceptable Here, and the re-INVITE is rejected without dropping the call; -
the INVITE arrived without SDP: esg itself forms the offer to the other leg according to
payloads_audio_offer.
-
The provider list order is intentionally not adjusted to the incoming SDP order: a list of several codecs may express a preference (for example, G729/8000 to save bandwidth, PCMA/8000 as a fallback). Therefore transcoding is possible: the internal device offered PCMA, and the provider chose the first G729 in the list. If transcoding on the media gateway is undesirable, put first the codec used by internal devices (for example, ["PCMA/8000", "G729/8000"]), or specify only it.
allow_ac_set
The b2b role parameter (default true) and the provider opts.allow_ac_set field (default false, applied with bgmg). Determines how many codecs the platform specifies in its answer to the offer:
-
true— a subset of the offer codecs known to the platform; first are the codecs selected by the opposite (called) side in its answer, if they are in the offer (the esg answer to a provider with a list — in list order); -
false— one codec: the first in offer order that is also in the opposite side answer; if there are none — the first main codec of the offer.telephone-event/8000andCN/8000from the offer are added to it (the esg answer to a provider with a list — see opts.payloads_audio).
More — RFC 3264 and allow_ac_set.
How SDP is built
New outgoing call
The role forms the offer itself, without incoming SDP: an outgoing call from an IVR script, a conference participant call, service calls. The list is built according to the role instance payloads_audio_offer.
| payloads_audio_offer | Formats in SDP |
|---|---|
not set |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For calls to SIPREC and streamed recording servers the codec composition can be set separately — by the server audio_formats field in the domain settings siprec_options, mixed_call_rec_options, streamed_call_rec_options. If the field is set, exactly the specified codecs are offered in list order (telephone-event/8000 and CN/8000 — at the end if specified), and payloads_audio_offer is not applied; if not set — the offer is built according to the b2b instance payloads_audio_offer.
Passing the offer to the other leg
The role receives an INVITE with SDP and passes the offer to the other leg (b2b; esg — see opts.payloads_audio). Order:
-
formats of the incoming SDP known to the platform — in their order (except excluded
--); -
then missing codecs according to
payloads_audio_offer— in final list order; -
CN/8000andtelephone-event/8000— at the end.
Incoming formats remain first because the called side usually selects the first suitable format. If it selects any format offered by the calling side, the media gateway does not need to transcode: streams in this format are passed between legs unchanged.
| payloads_audio_offer of the b2b instance | Formats in the offer to the other leg |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Answer
The answer contains only formats from the offer it answers: static — by number, dynamic — by codec name. The count is per allow_ac_set.
| Leg B answer | b2b answer to leg A with allow_ac_set = true | with allow_ac_set = false |
|---|---|---|
|
|
|
|
|
|
|
|
|
If the codec selected by leg B is in the A offer, there is no transcoding. Otherwise the media gateway transcodes.
Dynamic format numbers
In the answer dynamic formats are indicated under the numbers from the offer it answers (RFC 3264, section 6.1). In the offer passed to the other leg, the dynamic format numbers of the incoming SDP are preserved; added dynamic codecs receive free numbers not occupied in the incoming SDP. If the sides declared different numbers for one codec, the media gateway matches formats by codec name: each side receives a stream with numbers from its own SDP.
Incoming offer: m=audio 30000 RTP/AVP 8 96 101 a=rtpmap:96 opus/48000/2 a=rtpmap:101 telephone-event/8000 Offer to the other leg (payloads_audio_offer = ["G722/8000", "speex/16000"]): m=audio 40000 RTP/AVP 8 96 9 98 0 101 a=rtpmap:8 PCMA/8000 a=rtpmap:96 opus/48000/2 a=rtpmap:9 G722/8000 a=rtpmap:98 speex/16000 a=rtpmap:0 PCMU/8000 a=rtpmap:101 telephone-event/8000
Call scenarios
Outgoing from IVR via provider
IVR → b2b → esg → provider.
-
IVR forms a new offer according to its
payloads_audio_offer. -
b2b passes it to the esg leg, adding codecs per its own
payloads_audio_offer. -
esg on the provider leg: if the provider has
opts.payloads_audioset (and bgmg is used) — only its codecs in its order; otherwise SDP is passed unchanged, and withreinvite = 0— supplemented per esgpayloads_audio_offer(see opts.payloads_audio). -
The provider answers; the answer on each leg contains codecs from that leg offer, and the media gateway connects the streams, transcoding if necessary.
Call between subscribers
Device → sg → b2b → sg → device (sg may be the same instance or different ones).
sg using bgmg does not change codecs, passing the INVITE through, and works without transcoding. b2b passes the calling device offer to the called one, adding codecs per payloads_audio_offer; incoming formats remain first. If the called device selects any of the calling formats, there is no transcoding.
From device to provider
Device → sg → b2b → esg → provider. Example: the device does not support telephone-event/8000 (sends DTMF via SIP INFO), and the provider has opts.payloads_audio not set.
b2b supplements the offer with base telephone-event/8000, and esg passes SDP to the provider unchanged (with reinvite = 0 — supplements per esg payloads_audio_offer). The b2b answer to the device contains only the codecs of its offer, without telephone-event/8000. Provider DTMF signals per RFC 2833 are passed by b2b to the device via SIP INFO and back (see DTMF without telephone-event).
Incoming from provider
Provider → esg → b2b → …
-
The provider has
opts.payloads_audioset (bgmg): esg accepts the call only if the provider SDP contains a codec from the list (otherwise — 488 Not Acceptable Here), and answers with the intersection codecs: a set or one per opts.allow_ac_set. The offer further into the platform contains the provider codecs from the list in provider order, and b2b supplements it per its ownpayloads_audio_offer. -
Not set: esg passes the provider offer into the platform unchanged (with
reinvite = 0— without formats unknown to the platform); b2b supplements it per its ownpayloads_audio_offer. The answer to the provider contains codecs from its offer.
Different roles of one platform instance
Roles of one platform instance may be configured differently, for example ivr3 with ["G722/8000"] and b2b2 with ["G729/8000"]. Thanks to the base codecs they always have a common codec, and the call does not end with the final unsuccessful 415 response. When excluding base codecs (-PCMU/8000, -*), it is also worth making sure all instance roles keep a common codec.
sg and esg when using bgmg
-
sg does not change codecs: the INVITE passes through sg with its SDP, and there is no transcoding on sg.
-
esg acts per the provider opts.payloads_audio: if the list is set, it is applied to the provider leg SDP, the provider codecs from the list are passed inward, and transcoding is performed if necessary; if not set — the provider offer is passed inward with its codecs, and the esg instance
payloads_audio_offeris applied to offers to the provider withreinvite = 0and to INVITEs without SDP.
DTMF without telephone-event
DTMF signals are transmitted in RTP per RFC 2833 (format telephone-event/8000) or by SIP INFO messages (application/dtmf-relay). b2b repacketizes them between legs:
-
SIP INFO from one leg is passed to the other as RFC 2833 if the other leg SDP contains
telephone-event/8000, otherwise — as SIP INFO; -
an RFC 2833 event from one leg is passed to the other as SIP INFO if the other leg SDP does not contain
telephone-event/8000.
Therefore excluding telephone-event/8000 (-telephone-event/8000, -*) does not deprive the call of DTMF if the device or provider accepts SIP INFO.
RFC 3264 and allow_ac_set
RFC 3264 allows exchanging codec sets in both the offer and the answer:
-
the offer lists formats in preference order (section 5.1); the offering side must be ready to receive any of them;
-
the answer must contain at least one format from the offer; it is recommended to answer with a subset of the offer formats, in the same relative order, with the same dynamic format numbers (section 6.1);
-
the answer may also include formats not present in the offer, but they are not used in the current session until they appear in a new offer;
-
during a session the sides can switch between negotiated formats without a new SDP exchange.
In practice most devices and providers answer with one codec from the offer. There are implementations that incorrectly handle an answer with a set of codecs: for example, a device plays silence when the stream arrives in a negotiated but not the first codec of the answer. To maintain operability of the platform with such devices and providers, the platform supports answering with one codec: allow_ac_set = false (for a provider this is the default). The parameter is set for the b2b role instance and for the provider; it is not set for an individual subscriber account.
Media gateway: tubes
The media gateway receives local and remote SDP for the termination of each media leg. local arrives as a template: addresses, ports, and specific attributes are substituted by the media gateway itself.
Between terminations of opposite legs, tubes are built — paths from each format of one leg to the formats of the other, in both directions. An RTP packet finds its entry by format number (PT) and passes through the tube into the opposite stream: without re-encoding if the format matches (format number from the receiving side SDP), or with re-encoding. Tubes are built so that transcoding is minimal: matching formats are connected directly.
Therefore a side changing codec within the negotiated set, as allowed by RFC 3264, is serviced without a new SDP exchange: packets of the other format simply go through another tube.
Video
The platform does not record video in SIP calls, so as a rule it passes it between legs without decoding: video formats, including those unknown to the platform, are passed as is. Video transcoding is off by default (use_video_transcoding parameter). payloads_video_offer sets the composition and order of video formats in formed offers; there are no base video codecs.
Currently video and web conferencing services are mainly used for video communication, including the meet service on board the platform (Video conferencing), so development emphasis has shifted there.
FAQ
- Why are codecs in the INVITE in a different order than in payloads_audio_offer?
-
In the offer passed to the other leg, incoming SDP formats remain first in their order; the configuration list sets the order of added codecs. For a new call (IVR, conference) the order matches the configuration, with
CN/8000andtelephone-event/8000at the end. - How to offer only PCMA?
-
["PCMA/8000", "-*"]. When passing the offer to the other leg incoming formats are preserved; to leave only PCMA there too, use["PCMA/8000", "--*"]. - How to remove telephone-event?
-
-telephone-event/8000inpayloads_audio_offeror in the provideropts.payloads_audio. DTMF is then transmitted by SIP INFO, see DTMF without telephone-event. - Why was the call rejected with final unsuccessful 415 or 488?
-
The sides have no common codec: for example, base codecs were excluded and the offer has no codec from the list. esg answers 488 Not Acceptable Here if the provider SDP has no codec from opts.payloads_audio or the offer has no codec known to the platform.
- Where to see the call SDP?
-
In trn logs and on the call sequence diagram (/api/monitor/v1/trace).