JavaScript

WebRTC 101: How Real-time Video Communication is Facilitated

Giorgi Beruashvili — Fri, 27 Jan 2023 09:51:38 +0000

WebRTC 101: How Real-time Video Communication is Facilitated Giorgi Beruashvili Fri, 01/27/2023 - 13:51

Real-time video communication has become an essential part of our daily lives, whether it's for work or socializing. The ability to have face-to-face conversations with anyone, anywhere, has been made possible by the development of WebRTC, a technology that allows for real-time video and audio communication in the browser.

The process of connecting two users starts with locating each other using their public IP addresses, but this can be challenging due to the limited number of public IP addresses available. Private IP addresses were introduced to solve this problem, and NAT and STUN/TURN servers are used to assign public IP addresses and establish direct connections. Additionally, ICE and SDP protocols are used to standardize the exchange of information about media streams, inputs, and outputs.

Let's start with the basics. Consider the scenario in which person A wants to call person B. The first step is to locate each other. If we imagine an empty Cartesian coordinate system, we need to locate two dots (A and B) in order to connect them with a line. In the internet world, their locations are their public IP addresses, which determine their public presence.

When public IP addresses were first introduced, they were formatted using a 32-bit system, which mathematically means that there could only be 2 to the power of 32 instances of public IP addresses, approximately 4.2 billion possible public IP addresses. However, nowadays there are more internet users and connected devices than the number of possible public IP addresses. As a result, not every device can have its own public IP address and therefore cannot have a public presence, meaning no dots on the Cartesian system.

How do we solve this issue?

To solve this problem, private IP addresses were introduced. Generally, only routers have public IP addresses, and all devices connected to this particular router have their own private IP address. These private IP addresses cannot communicate with the external world; they cannot send a request to the internet or receive a response.

When they send a request to the router, they are given a public presence (public IP address) through a process called NAT (Network Address Translation). This request is then sent to the destination and a response is returned, which is translated back to the private IP address that sent the request.

There are four types of NAT:

Full-cone NAT
IP address restricted NAT
Port restricted NAT
Symmetrical NAT.

Most routers operate with the full-cone NAT system, which allows responses from all IP addresses and ports to come through. Other types, however, check the trustworthiness of the response-issuer's IP address, port, or both. This will be important later on in the process, but for now we know that we send a request from a private IP address which is wrapped by a public IP address using NAT. The request receives a response from the destination and either passes through (full-cone) or is checked for trustworthiness (e.g. has the destination been visited before? If so, it is considered trustworthy) before being allowed to pass or be blocked.

Now we know that even though each device does not have its own public IP address, we can still communicate with the external world through our routers using NAT. However, how do we determine what our public IP address is?

We can use STUN (Session Traversal Utilities for NAT) to help with this.

STUN servers are cheap and easy to maintain, and they are mostly public servers that respond to us with our public IP address, port, and connectivity status. We will package this data and send it to our Peer as an ICE candidate, who will do the same (Interactive Connectivity Establishment is a protocol used to transfer data provided by the STUN server in a standardized way). This process allows us to locate each other and connect directly.

However, STUN servers can sometimes fail. This is because some NAT methods, such as symmetrical NAT, block direct connections after exchanging information about each other.

For example:

A has a full-cone NAT while B has a symmetrical NAT. They exchange their information and attempt to connect directly browser-to-browser, as this is what video calls are about - direct connection without delays. A's router, which uses full-cone NAT, allows the signal from B to come through, but B's router encounters A's signal for the first time and does not trust it, blocking it from passing through. This is where TURN (Traversal Using Relays around NAT) servers come into play.

They provide both peers with information about their IP addresses, ports, and other details, and then act as a mediator between them, allowing the signal to pass through them and eliminating the need for trust between the caller's and receiver's endpoints. TURN servers are more expensive and difficult to maintain, and are often privately owned. They are used as a relay if STUN servers fail.

We have reached the point where A has determined their public IP address using STUN or TURN and created an object containing this information as an ICE candidate.

As mentioned above, ICE is a protocol that is used to standardize the construction of this information. On the other end of the line, B also did the same.

What should we do next? Do we need to gather any more information, and if so, how should we send it? Do we need more information in general?

Yes, we need to gather information about our media streams, inputs, and outputs, among other things. We collect this information through WebRTC and create an offer containing it, also called an SDP (Session Description Protocol), which is essentially a string of information.

How do we exchange this information with each other? That part is left to your imagination. You can use paper, pigeons, or servers. Mostly, servers are chosen. We implement a web socket that fires when Peer A sends his SDP and ICE candidate to the server. This allows Peer B to react to the event and use this information, while also sending his own SDP. The same process happens with Peer A, as the web socket informs him about Peer B firing an event and he can respond to it.

The following code examples represent some of the concepts explained in the article. For example, getting a local video stream and rendering it in an HTML element.

In the following code snippet, we write a function called requestMediaDevices, which accesses the navigator object (The Navigator interface represents the state and identity of the user agent). Then, in the mediaDevices object, which contains information about user media devices, we call a method. This method prompts the user for permission to use a media input, which produces a MediaStream with tracks containing the requested types of media.

private async requestMediaDevices(): Promise<void> {
  try {
    this.localMediaStream = await navigator.mediaDevices.getUserMedia();
  } catch (error: any) {
    alert(`An error occured while getting user media ${error.name}`);
  }
}

In order to attach a video stream to an HTML element, you need to map through the tracks of the localMediaStream and enable them. These tracks are video and audio. The localVideoStream is just a reference to the <video> HTML element and you bind this flowing stream of media to this element's srcObject.

this.localMediaStream.getTracks().forEach(track => {
	  track.enabled = true;
});

this.localVideoStream.nativeElement.srcObject = this.localMediaStream;

Now we can skim through the call process:

async startCall(): Promise<void> {
  this.peerConnection = new RTCPeerConnection(RTCConfiguration);
  // in RTC configuration we have information about websocket that we are using, stun servers
  // Here you should write handlers for onicecandidate (retirieve ice candidates and send them to websocket)
  // Also handlers for oniceconnectionstatechange, onsignalingstatechange and ontrack - to properly handle call failures, retries and etc.   
 
  //adding local video stream into the mediaStream of the offer
  this.localMediaStream.getTracks().forEach(
    track => this.peerConnection.addTrack(track, this.localVideoStream)
  );
  
  try {
    const offer: RTCSessionDescriptionInit = await this.peerConnection.createOffer();
    //setting local sdp offer into the localdescription of peerconnection
    await this.peerConnection.setLocalDescription(offer);
    this.RTCconnectionWebsocket.sendMessage({type: 'offer', data: offer});
  } catch (err: any) {
    //handle error here
  }
}

Now some moments from handling Peer A’s offer from Peer B’s side:

private handlePeerOffer(offer: RTCSessionDescriptionInit): void { 
 // An sdp from A, which is an offer is a remote description for B and visce versa
 this.peerConnection.setRemoteDescription(new RTCSessionDescription(offer))
    .then(() => {
      // add media stream to local video html element reference srcObject just like in Peer A
      this.localVideoStream.nativeElement.srcObject = this.localMediaStream;
      // add media tracks to remote connection just like in Peer A's case
      this.localMediaStream.getTracks().forEach(
        track => this.peerConnection.addTrack(track, this.localVideoStream)
      );
    }).then(() => {
    // Build SDP for Peer B which in peerConnection would be an answer to the offer
    return this.peerConnection.createAnswer();
  }).then((answer) => {
    // Created answer would be B's SDP which should be set as his local description
    return this.peerConnection.setLocalDescription(answer);
  }).then(() => {
    this.RTCconnectionWebsocket.sendMessage({type: 'peerAnswer', data: this.peerConnection.localDescription});
  }).catch(//catch error here);
}

As the code snippet shows, B sets A's offer as a remote description while creating its own answer and setting it as a local description of his own. Then, it alerts the web socket that the answer was set so that the further event handlers which depend on the web socket getting this type of message can act upon it.

Hopefully, this roundup can shed some light on the process of real-time video communication and contribute to enhancing the low-level knowledge of building such applications.

Language English

Unleashing the Power of SVG Templates in Angular

Besarion Satno — Fri, 20 Jan 2023 12:34:45 +0000

Unleashing the Power of SVG Templates in Angular Besarion Satno Fri, 01/20/2023 - 16:34

As a developer, you are probably familiar with the importance of creating high-quality and visually appealing graphics for your web applications. One of the best ways to achieve this is by using SVG. SVG is a powerful and versatile format that allows you to create and display vector graphics on the web. Since SVG is using XML to describe the graphics, generating dynamic SVGs can be quite useful and simple to implement.

Apparently, in Angular, we can use .svg templates for components instead of the .html files we are accustomed to. This opens the door to using Angular's syntactic sugar, like property binding and event binding, with SVGs to create dynamic and interactive graphic content.

In this blog post, we will take a closer look at how and why to use SVG templates in Angular. Using SVG templates in Angular allows you to create dynamic, responsive graphics that can change based on dynamic data or user interactions. This makes it a great choice for creating complex graphics, visualizations, and animations for your web applications.

To use SVG as an Angular template, first, you need an Angular project. Let’s create one:

ng new svg-demo

Now delete everything from app.component.html and create a new component:

ng g c svg-demo

I usually manually change the .html suffix to .svg, delete its contents, and update the .ts file with the new templateUrl in the @Component decorator.

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-svg-demo',
  templateUrl: './svg-demo.component.svg',
  styleUrls: ['./svg-demo.component.scss'],
})
export class SvgDemoComponent implements OnInit {
  constructor() {}

  ngOnInit(): void {}
}

CLI takes care of many things, such as adding component declarations to the @NgModule decorator, and we can freely use it in the app.component.html with the <app-svg-demo></app-svg-demo> tag.

In svg-demo.component.svg I’ll paste the SVG code for this isometric cube which I got from draw.io

Here is the contents of cube.svg:

<svg  width="91px" height="101px" viewBox="-0.5 -0.5 91 101" style="background-color: rgb(255, 255, 255);">
	<g>
		<path d="M 45 0 L 90 21.61 L 90 78.39 L 45 100 L 0 78.39 L 0 21.61 Z" fill="rgb(255, 255, 255)" stroke="rgb(0, 0, 0)"/>
		<path d="M 0 21.61 L 45 43.21 L 90 21.61 M 45 43.21 L 45 100" fill="none" stroke="rgb(0, 0, 0)"/>
	</g>
</svg>

To break this code down, the surrounding <svg> tag contains one <g> tag, which stands for "group" and groups two <path> tags. Those two paths represent these two parts of our shape:

Now, let's make this markup more Angular-y and decide what we want to do with it.

One of the most basic things we can do in Angular Components is to bind .html and .ts files. In our case, we can bind values from .ts to the values in .svg. Let's consider the colors of those shapes and make the outline of the same color.

Below, we have just an array of colors, a strokeColor property, and a function that returns a randomly chosen color from the array.

import { Component, OnInit } from '@angular/core';

@Component({
  selector: 'app-svg-demo',
  templateUrl: './svg-demo.component.svg',
  styleUrls: ['./svg-demo.component.scss'],
})
export class SvgDemoComponent implements OnInit {
  colors: string[] = [
    'red',
    'orange',
    'yellow',
    'green',
    'blue',
    'purple',
    'black',
  ];

  strokeColor: string | undefined;

  constructor() {}

  getRandomColor() {
    const randomNumber = Math.floor(Math.random() * this.colors.length);
    return this.colors[randomNumber];
  }

  ngOnInit(): void {
    this.strokeColor = this.getRandomColor();
  }
}

I want to bind the randomly chosen color to the stroke attribute of those two shapes, as we want to change both of them.

<svg  width="91px" height="101px" viewBox="-0.5 -0.5 91 101" style="background-color: rgb(255, 255, 255);">
	<g>
		<path d="M 45 0 L 90 21.61 L 90 78.39 L 45 100 L 0 78.39 L 0 21.61 Z" fill="rgb(255, 255, 255)" [attr.stroke]="strokeColor" />
		<path d="M 0 21.61 L 45 43.21 L 90 21.61 M 45 43.21 L 45 100" fill="none" [attr.stroke]="strokeColor" />
	</g>
</svg>

Now the color of the stroke changes on every refresh. Let's also change the fill color of the first <path> tag. I will modify the .ts file for my purposes:

strokeColor: string | undefined;

// add new property
fillColor: string | undefined;

ngOnInit(): void {
  this.strokeColor = this.getRandomColor();

	// assign random color to this as well
  this.fillColor = this.getRandomColor();
}

And I'll update the SVG template as well:

<svg  width="91px" height="101px" viewBox="-0.5 -0.5 91 101" style="background-color: rgb(255, 255, 255);">
	<g>
		<path d="M 45 0 L 90 21.61 L 90 78.39 L 45 100 L 0 78.39 L 0 21.61 Z" [attr.fill]="fillColor" [attr.stroke]="strokeColor" />
		<path d="M 0 21.61 L 45 43.21 L 90 21.61 M 45 43.21 L 45 100" [attr.fill]="fillColor" [attr.stroke]="strokeColor" />
	</g>
</svg>

Here are the results of multiple reloads, all with different (random) colors:

Let's change the behavior and make the colors change on mouse click, instead of relying on the refresh button. We can use Angular's event binding syntax for this. I will modify the files once again:

changeColors() {
  this.strokeColor = this.getRandomColor();
  this.fillColor = this.getRandomColor();
}

ngOnInit(): void {
  this.changeColors();
}

<svg  width="91px" height="101px" viewBox="-0.5 -0.5 91 101" style="background-color: rgb(255, 255, 255);">
	<g (click)="changeColors()">
		<path d="M 45 0 L 90 21.61 L 90 78.39 L 45 100 L 0 78.39 L 0 21.61 Z" [attr.fill]="fillColor" [attr.stroke]="strokeColor" />
		<path d="M 0 21.61 L 45 43.21 L 90 21.61 M 45 43.21 L 45 100" [attr.fill]="fillColor" [attr.stroke]="strokeColor" />
	</g>
</svg>

That's basically it. If you want to move the shape, you can use the x and y attributes for each element, or for all of them at once with the parent tag.

Few things to keep in mind:

1. When Angular renders the SVG Component in the app-root, it is usually rendered inside a <div>-like element. And if you want to nest multiple SVG components, you should change the component selector and use the directive's syntax like this:

@Component({
	selector: '[app-svg-demo]',
	...
});

<svg app-svg-demo></svg>

2. We need to use [attr.X] syntax to not get the following error:

error NG8002: Can't bind to 'fill' since it isn't a known property of ':svg:path'.

According to Angular documentation:

This error arises when attempting to bind to a property that does not exist. Any property binding must correspond to either:

A native property on the HTML element, or

An @Input() property of a component or directive applied to the element.

So by using [attr.X] syntax, we make sure that Angular knows that we’re trying to bind the value to the native property on the HTML (in our case SVG) element.

3. We can as easily use the @Input and @Output data / event binding between a parent and child components.

4. We can also use structural directives *ngIf / *ngFor with the SVG elements (don’t forget to add CommonModule to the AppModule imports).

5. SVG paths and shapes follow their order. The next element will be drawn on top of the previous one.

Language English

Getting started with Prettier — Modern JavaScript Tooling (Part 1)

Giorgi Kapanadze — Mon, 27 Jul 2020 14:36:23 +0000

Getting started with Prettier — Modern JavaScript Tooling (Part 1) Giorgi Kapanadze Mon, 07/27/2020 - 18:36

JavaScript has grown into an enormous ecosystem for the past couple of years, which makes learning this language exciting and challenging at the same time. The vast array of available tools and frameworks might be a roadblock for some junior developers who just can’t figure out where to start and what to use.

The general rule of thumb is to get the basics of the language first and move to nuts and bolts down the road. For sure, this is a valid approach. In my experience, though, some tools can be essential even for the absolute beginners, and knowing how to set up and use those can make the learning curve smoother.

So in this series of posts, I’ll be covering a couple of must-have (or, must-take-a-look) tooling of the modern JS ecosystem, as well as tips on how to use and integrate them in our projects.

Before we begin, let’s set up our environment. We won’t need much for the tools discussed in this post — just get the latest Node.js on your machine, if you don’t have it installed already. Node is compatible with all three operating systems, so you’re free to use your OS of choice for the development machine.

Coding style

Imagine that you’re working on a project with a couple of fellow developers. All those guys and gals will have their preferred coding styles — from indentation preferences to bracket placements, line wrapping, and commenting methods. The resulting code of a team without a strict coding style guidelines will be a mess to read and hard to follow through.

This is why the coding style and standards have to be agreed upon at the start of the project and followed by all members without any exceptions. But we’re all humans and we have this beautiful (but sometimes — annoying) thing called muscle memory. Muscle memory in our fingers helps us perform routine tasks without even thinking (like creating a code block for IF statement) but also prevents us from “switching” between those behaviors quickly. This is why it’s pretty hard to switch your coding mannerisms from project to project and it’s unreliable to hope that the coding styles will be followed by every team member all the time.

Prettier solves this problem by automating the code formatting.

Let’s create a new directory and initialize NPM in it. Open terminal, jump to the project folder and run npm init -y command. This will create the following project.json file for us:

Now we can add Prettier to our project. Use this command to install it in our project:

npm install --save-dev prettier

The --save-dev flag tells NPM to install this package only for the development environment (we won’t need it elsewhere since the code will be formatted during the development.) After the command is finished, we’ll have the node_modules folder in our project directory and the package.json will be updated as well (you’ll see the devDependencies group since the Prettier was installed for the dev environment only):

Now let’s make some mess for the Prettier to do its job. Create a new .js file and type some code without any formatting consistencies. Something like this:

This code has several formatting issues: for one, the strings are written with both types of quotes — single and double; the function block is indented with 4 spaces, while the if-statement block is using 2 spaces for the same job; some lines are too long; and so on.

Let’s put Prettier to work. Create a new script called format that runs the Prettier for us. Our package.json should look like this:

{
  "name": "modern-js",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
   "format": "prettier --write"
  },
  "keywords": [],
  "license": "ISC",
  "devDependencies": {
   "prettier": "^1.19.1"
  }
}

The scripts described in package.json lets us run complex commands from the node_modules/.bin directory in a shortcut-like manner. Let’s take a look inside this folder:

prettier is a regular executable JavaScript file. Scripts in package.json help us create a “shortcut” for a command like this:

./node_modules/.bin/prettier --write src/index.js

To run the same command with NPM script, we can just run this command from our project root directory (the same directory that holds package.json):

npm run format src/index.js

The last part of both commands — src/index.js — tells Prettier which file to format. After the command is finished, our messy file will look like this:

I think we can agree that this code is, indeed, prettier. But most importantly, it’s easier to read or quickly scan to understand its structure. The code is more consistent, line indentations are the same, if-else blocks look much better and chained methods are broken into lines.

But this style might not be the same as what your team has agreed on. For example, the team might consider 2-space indentation as their default, or to use single quotes for strings, instead of double ones. This is why Prettier rules are configurable and you can change the defaults with the .prettierrc file. Let’s create it in our project’s root directory and put this piece of JSON in it:

{
  "tabWidth": 4,
  "semi": true,
  "singleQuote": true
}

With this we’re letting Prettier know that we prefer 4 spaces for indentation (tabWidth); also that we want semicolons to be compulsory (semi) and to force single-quotes for writing string literals instead of double ones (singleQuote).

After running the format script again, we’ll get the src/index.js file formatted with our configured rules:

The configuration options we used are just a small part of the set prettier configuration options you can use to match its behavior with your project style guidelines.

Before we wrap up, let’s extend our format command a bit and make it more flexible to use. Most probably, we’ll be needing the formatting guidelines forced on all files in our project, not just on a single one. Instead of running the script on all files one by one, we can specify a file name pattern for Prettier that will pick up all .js files inside our src/ directory and it’s child directories as well.

Open the package.json file and add the following script to the scripts object:

"format:all": "prettier --write \"src/**/*.js\""

To see the new script in action, create some more .js files in the src/ directory; or create some directories in src/ and put the .js files inside those.

Now, when you run npm run format:all all your .js files in src/ directory will be (recursively) picked up by Prettier and re-formatted with rules specified in the .prettierrc configuration file.

In this small introduction to Prettier you’ve learned how to use it in terminal and with NPM scripts. It’s very easy to integrate this tool in your project workflow, configure it, and forget about code formatting altogether. You can never think about the formatting guidelines of a specific project, just focus on your code and Prettier will do the rest.

Prettier can also be integrated into your favorite code editor (for example, see extensions for VS Code, Atom, or Sublime editors). With integration like this, the code formatting will be performed every time you save the file, which makes using this tool even more comfortable and seamless.

Language English