< Back to Package Overview
Working with Video
This article discusses concepts and features relating to working with video on an active call. Unlike audio, video participation can change dynamically. This imposes requirements on the application that must be understood and managed by the calling application.
The following article is a recommended to get you started:
Adding video to a call
Once an audio call is established between two endpoints, either endpoint can request the addition of video media to the call. Such a request could be started in response to a UI action by the user on the endpoint's application, and the client SDK sends the request as signaling to the remote party.
The addition of video is not guaranteed, however, just because one side requests it. The other side must explicitly accept it, and the other side may accept it only in a modified way. For example, the requesting side may ask for bi-directional video, but the accepting side may only provide video reception.
In order to add video to an existing audio call, the requesting side follows a process similar to the one used for making a video call:
- Obtain resources to offer video
- Assign offered video to the call
- Activate the video resources after remote acceptance
The accepting side has a flow which is distinct from that for answering a video call, and the accepting side can elect to accept the video stream bi-directionally, uni-directionally, or not at all.
For the case where video is to be accepted, the application needs to perform the following activities:
- Detect the remote video offer
- Obtain resources for accepting video
- Accept the video offer
- Activate video resources after local acceptance
For the case where video is to be declined, the application needs to perform the following activities:
- Detect the remote video offer
- Decline the remote video offer
Obtain camera
The application needs to set video camera when making a call with video. Camera availability is verified through the CameraVideoSource class. This class manages most aspects of camera operation, including verification of available hardware.
CameraVideoSource cameraVideoSource = new CameraVideoSource();
CameraDevice selectedCamera = cameraVideoSource.Cameras[0];
Assign video to the call
To add a video stream, the application sets video mode to the existing call. For this case of escalation after answer, a completion handler is used to signal that the operation failed or was declined by the remote endpoint. There may be a several second delay in execution of the completion handler since network signaling needs to complete. The application should use the completion handler to update UI elements to indicate success or failure of the signaling operation.
call.SetVideoMode(VideoMode.SendReceive, (error) =>
{
if (error != null)
{
// Update UI elements to indicate that video was not activated
}
else
{
// Update UI elements that video was activated (in some way)
}
});
Activating the video resources
Successfully requesting the addition of video doesn't imply that remote endpoint accepted the request with the desired video direction. Although bi-directional video is requested by the local application, the remote endpoint may support only uni-directional video, or it may not support video at all. The usual VideoChannelsUpdated callback of Call, tells the requesting endpoint how the remote side accepted the video addition.
public void VideoChannelsUpdated(object sender, VideoChannelsEventArgs e)
{
if (e.VideoChannels.Count > 0 && e.VideoChannels[0].IsEnabled)
{
int videoChannelId = e.VideoChannels[0].ChannelId;
VideoSink sink = videoInterfaceInstance
.getLocalVideoSink(videoChannelId);
cameraVideoSource.setVideoSink(sink);
VideoRenderer2 video = remoteVideo[0];
try
{
VideoSource videoSource = videoInterfaceInstance
.getRemoteVideoSource(videoChannelId);
videoSource.setVideoSink(video);
Console.WriteLine("Remote rendering success");
}
catch (Exception ex)
{
Console.WriteLine("Remote rendering failed");
}
}
Detecting the remote video offer
The accepting endpoint receives a notification of the remote video offer from the IncomingVideoAddRequestReceived event on the Call. The arrival of the event should trigger the application to apply a response policy. For example, the application could accept the video escalation as reception-only, or the application could explicitly deny the accepting the additional video stream. If the application does not respond to the event, the
Client SDK will automatically decline on behalf of the application.
public void IncomingVideoAddRequestReceived(Call call)
{
// Implement a policy to handle the remote request for video
}
Accepting the remote video offer
The application accepts the addition of video by setting video mode, initializing it with the desired video direction.
// Note that there's no guarantee that it will
// start with the expected video direction (or any) video. A completion handler
// parameter is available, but not too useful as the App should react only to
// the VideoChannelsUpdated event.
call.AcceptVideo(VideoMode.SendReceive, (error) =>
{
if (error != null)
{
// Update UI elements to indicate that video was not activated
}
else
{
// Update UI elements that video was activated (in some way)
}
Decline the remote video offer
When remote video addition is detected, the local application need not accept it; the application can use the denyVideo method on the call to decline the request. The decline will be signaled back to the originator via the completionHandler of the remote's invocation of setVideoMode.
Removing video from a call
Once a video call is established between two endpoints, either endpoint can downgrade the call to audio-only; this stops both video reception and transmission. After downgrading to an audio-only call, the endpoint may be able to add video back again if desired.
Note: Some endpoints will not allow video to be restored after downgrade.
To end video on the local endpoint, the application sets VideoMode.DISABLE video mode. This will deallocate the video channel id in use for the call, release the camera, and send signaling to end video on the remote endpoint.
In response to the local removal of video, the Client SDK will send up VideoChannelsUpdated with an empty list of video channels. Such an event is the key to the application to remove its render surfaces.The remote client will also see VideoChannelsUpdated event relating the the call's loss of video indicates that the video channel has been deleted and the camera (if running) has been stopped. The application should clean up the corresponding render surfaces.
Incoming call
The application can use the hasVideo method on the call to indicate if the incoming call has video.
Turning a camera on or off
Once a video call is established between two endpoints, either endpoint can turn their video camera off or on. Turning the local camera on or off means new invocations of SetVideoMode with handling of the subsequent VideoChannelsUpdated events. Such an action results in a signalling update to allow the remote side to react accordingly. The result of such an action may, in fact, result in video cessation, but not video removal.
Handling remote video cessation or resumption
Once a video call is established between two endpoints, it's possible for network services to cease video temporarily. Examples of such services are hold, standard conferencing, or transfer. In such cases, the application will receive VideoChannelsUpdated events that video has ceased. The cessation may be temporary (one or two seconds) or may be substantive (several minutes). The application will need define policies on what to display on the UI in such cases.
