Real-time communication with WebRTC

 

Customized web-based audio & amp; video solutions Real-time communication with WebRTC

Latencies, jittering, disconnections: Who hasn’t had problems with audio and video transmission over the Internet? The WebRTC standard is intended to remedy this situation, this article deals with the possibilities and peculiarities of browser-based “web Real Time Communication”.

Company about the topic

Media stream call inserted into the JavaScript console of the Chrome browser.Media stream call inserted into the JavaScript console of the Chrome browser.

(Image: Boele / Google)

In the past 12 months, network-based communication via audio and video has received a huge boost in the area of digitalization as a result of the COVID pandemic-also in Germany, which is more reserved in this regard. According to a study by Twilio, digitalization initiatives in some sectors have been accelerated by up to 7.2 years compared to the usual pace of development.

The massive success of the providers of collaboration solutions is therefore not surprising. The downside of most solutions on the market is that you have to install additional software to use audio and video conferencing as well as desktop sharing and embedded chat functionality. In addition, appropriate server capacities must be maintained in order to enable the use of these technologies.

This also makes sense for conferences with more than three participants, for security reasons alone: for example, with regard to authentication, authorization and accounting. For interactions between two participants, i.e. in peer-to-peer setups, you do not need a setup via a virtual conference room as long as no additional functions such as logging, debugging or recording are required.

Ideally, the media data here runs between the two participants with far less negative effects due to latency and jitter. In the field of telemedicine, direct media traffic between doctor and patient is a core requirement for use in the sphere of the KBV (National Association of Statutory Health Insurance Physicians).

Today, the web browser is an essential component of any stationary and portable computer, smartphone and tablet. Therefore, it was already some time ago offered to expand the browsers with functionalities for real-time communication. This is where WebRTC – Web Real Time Communications – comes in.

Video or voice communication easy with the web browser

WebRTC is an open-source project consisting of a set of standards controlled by the W3C (World Wide Web Consortium) and the IETF (Internet Engineering Task Force) that define a media stack for real-time communication for web browsers and mobile devices on the Internet. The functionalities are accessible via various APIs, i.e. application programming interfaces.

Without additional plug-ins or applications, this enables peer-to-peer video and voice communication as well as data exchange within websites. This greatly improves the user experience and enables communication on closed systems. The project is supported – among others – by Apple, Google, Microsoft, Mozilla and Opera.

In terms of design, WebRTC includes several APIs. The essential elements are:

  • getUserMedia allocates the audio and video media of the system (for example, by accessing the camera and microphone of the computer or smartphone used)
  • RTCPeerConnection sets up audio and video communication between peers. This includes signal processing, handling of the codecs, peer-to-peer communication, security and bandwidth management. WebRTC supports DCSP marking, which can be taken into account by downstream network elements in traffic handling.
  • RTCDataChannel enables bidirectional exchange of any data between peers. It uses the same APIs as websockets and is characterized by very low latency.

For example, if you run the following code in the JavaScript console of the Chrome browser, you get a media stream with audio and video track, provided that you allow access to the camera and microphone (and these are also available):

const constraints = {
'video': true,
'audio': true
}
navigator.mediaDevices.getUserMedia(constraints)
.then(stream => {
console.log('Got MediaStream:', stream);
})
.catch(error => {
console.error('Error accessing media devices.', error);
});

The code inserted into the JavaScript console of the Chrome browser, including the response.The code inserted into the JavaScript console of the Chrome browser, including the response.

(Image: Boele / Google)

For example, the answer looks like in the preceding image. A quick test of the getUserMedia API call can also be done via the website https://simpl.info/getusermedia / make. A data transfer can be made via https://simpl.info/rtcdatachannel / try it out. The link to the code on GitHub is also stored there. An additional benefit of using WebRTC is the use of the security mechanisms implemented in the browser, including secure communication via validated strong cryptographic protocols, including DTLS and SRTP.

The support of various audio and video codecs is integrated in the web browsers, which greatly simplifies handling. Supported video codecs include VP8 and VP9 and AVC / H. 264 (with restrictions). AV1 (for Chrome) and HEVC (for Safari) are currently in the experimental stage. In the field of audio, Opus, G. 711 PCM (a-Law and u-Law) are supported. Different web browsers at the endpoints can agree on codecs that are supported by both sides.

Server components for various applications

As already mentioned, applications for WebRTC range from peer-to-peer voice and video communication (e.g. for telemedicine), event and collaboration platforms to use in call centers where the browser can completely replace the classic handsets.

Depending on the application, WevRTC server components are required for the following tasks in order to enable these services:

  • NAT Traversal Server for WebRTC

The application server hosts the website through which the WebRTC API calls are triggered. The signaling servers ensure the establishment and dismantling of connections and use protocols such as SIP, XMPP or MQTT. Also are various proprietary implementations of signaling servers on NODE.JS basis available. Signaling is not part of the WebRTC standards and must therefore be implemented accordingly, for which websockets are suitable as a protocol due to the low overhead and the data exchange in real time.

In the majority of cases, the WebRTC endpoints that want to communicate with each other will be behind NAT firewalls. NAT traversal servers help communication partners to communicate with each other. A distinction is made between STUN and TURN servers.

  • The STUN (Session Traversal Utilities for NAT)-Server determines the public IP address of a client, which is then used to exchange the media streams between the peers.
  • The TURN CARD (Traversal Using Relays around NAT) server is used to forward the media streams. This is used when the peers cannot communicate directly with each other.

The WebRTC Media Servers act as server-side clients and act, among other things, as MCU (Multipoint Control Unit) to provide group rooms. Furthermore, they can serve as a protocol converter and gateway to other networks, enabling recording and broadcasting/streaming functionalities and more.

WebRTC closed a critical gap in the web platform with its release several years ago. It offers cost-effective audio, video and data communication via the web browser, without the need to use proprietary and stand – alone applications off-the-shelf – real-time communication via a web page. “Voice is just another JS application”, as the former CTO of the FCC, Henning Schulzrinne, has put it in a nutshell.

Thomas Boele

Thomas Boele (Picture: Twilio)

If you like the examples listed above, you can also go deeper into the matter and create a peer-to-peer react web app with very little effort, a guide can be found here: https://github.com/twilio/twilio-video-app-react. Corresponding examples are also available for Android and iOS.

* Thomas Boele is Director Solutions Engineering at Twilio.

(ID:47117249)

Ready to see us in action:

More To Explore

IWanta.tech
Logo
Enable registration in settings - general
Have any project in mind?

Contact us:

small_c_popup.png