< Back to Package Overview
Working with Video
This article discusses concepts and features relating to working with video on an active call. Unlike audio, video participation can change dynamically. This imposes requirements on the application that must be understood and managed by the calling application.
The following article is a recommended to get you started:
Adding video to a call
Once an audio call is established between two endpoints, either endpoint can request the addition of video media to the call. Such a request could be started in response to a UI action by the user on the endpoint's application, and the client SDK sends the request as signaling to the remote party.
The addition of video is not guaranteed, however, just because one side requests it. The other side must explicitly accept it, and the other side may accept it only in a modified way. For example, the requesting side may ask for bi-directional video, but the accepting side may only provide video reception.
In order to add video to an existing audio call, the requesting side follows a process similar to the one used for making a video call:
- Obtain resources to offer video
- Assign offered video to the call
- Activate the video resources after remote acceptance
The accepting side has a flow which is distinct from that for answering a video call, and the accepting side can elect to accept the video stream bi-directionally, uni-directionally, or not at all.
For the case where video is to be accepted, the application needs to perform the following activities:
- Detect the remote video offer
- Obtain resources for accepting video
- Accept the video offer
- Activate video resources after local acceptance
For the case where video is to be declined, the application needs to perform the following activities:
- Detect the remote video offer
- Decline the remote video offer
Obtain camera
The application needs to set video camera when making a call with video. Camera availability is verified through the CSVideoCapturerIOS class. This class manages most aspects of camera operation, including verification of available hardware.
self.videoCapturer = [[CSVideoCapturerIOS alloc] init];
if ([self.videoCapturer hasVideoCameraAtPosition:CSVideoCameraPositionFront]) {
[self.videoCapturer useVideoCameraAtPosition:CSVideoCameraPositionFront
completion:nil];
}
else if ([self.videoCapturer hasVideoCameraAtPosition
:CSVideoCameraPositionBack]){
[self.videoCapturer useVideoCameraAtPosition:CSVideoCameraPositionBack
completion:nil];
}
else {
// This could happen for iPhone simulator. Video will not work for iPhone
// simulator, since CSVIideoRendererIOS doesn't support camera selecetion
// (which is supported by CSVIDEORenderOSX)
// Video will always fail for iPhone simulator.
}
Assign video to the call
To add a video stream, the application sets video mode to the existing call. For this case of escalation after answer, a completion handler is used to signal that the operation failed or was declined by the remote endpoint. There may be a several second delay in execution of the completion handler since network signaling needs to complete. The application should use the completion handler to update UI elements to indicate success or failure of the signaling operation.
[call setVideoMode:CSVideoModeSendReceive completionHandler:^(NSError *error) {
if (error) {
// Update UI elements that video was activated (in some way)
} else {
// Update UI elements to indicate that video was not activated
}
}];
Activating the video resources
Successfully requesting the addition of video doesn't imply that remote endpoint accepted the request with the desired video direction. Although bi-directional video is requested by the local application, the remote endpoint may support only uni-directional video, or it may not support video at all. The usual didUpdateVideoChannels callback of CSCall, tells the requesting endpoint how the remote side accepted the video addition.
- (void)call:(CSCall *)call didUpdateVideoChannels:(NSArray *)videoChannels {
CSVideoChannel *videoChannel = videoChannels[0];
if (videoChannel.enabled == YES) {
if (self.viewController.remoteVideoView.hidden == YES) {
self.viewController.remoteVideoView.hidden = NO;
}
// Start video rendering if the negotiated video direction is
//send-receive or receive-only
// and video rendering is not in progress
if (videoChannel.negotiatedDirection == CSMediaDirectionSendReceive
|| videoChannel.negotiatedDirection == CSMediaDirectionReceiveOnly) {
[[videoInterface getRemoteVideoSource:videoChannel.channelId]
setVideoSink:
self.remoteVideoSink];
NSLog(@"%s Started video rendering", __PRETTY_FUNCTION__);
}
// for a held call the media direction is inactive
if (videoChannel.negotiatedDirection == CSMediaDirectionInactive){
[[videoInterface getRemoteVideoSource:videoChannel.channelId]
setVideoSink:nil];
NSLog(@"%s Stopped video rendering", __PRETTY_FUNCTION__);
}
// Start video transmission if the negotiated video direction is
//send-receive or send-only
//and video transmission is not in progress
if (videoChannel.negotiatedDirection == CSMediaDirectionSendReceive
|| videoChannel.negotiatedDirection == CSMediaDirectionSendOnly) {
[self.videoCapturer setVideoSink: [videoInterface
getLocalVideoSink: videoChannel.channelId]];
NSLog(@"%s Started video transmission", __PRETTY_FUNCTION__);
}
} else if(videoChannel.enabled == NO) {
if (self.viewController.remoteVideoView.hidden == NO) {
self.viewController.remoteVideoView.hidden = YES;
}
[[videoInterface getRemoteVideoSource:videoChannel.channelId]
setVideoSink:nil];
[self.videoCapturer setVideoSink: nil];
}
Detecting the remote video offer
The accepting endpoint receives a notification of the remote video offer from the callDidReceiveVideoAddRequest delegate of the CSCall. The arrival of the event should trigger the application to apply a response policy. For example, the application could accept the video escalation as reception-only, or the application could explicitly deny the accepting the additional video stream. If the application does not respond to the event, the Client SDK will automatically decline on behalf of the application.
- (void)callDidReceiveVideoAddRequest:(CSCall *)call {
NSLog(@"%s call(%@)", __PRETTY_FUNCTION__, call);
[mediaManager acceptVideoForCall:call withVideoMode:CSVideoModeSendReceive];
[[NSNotificationCenter defaultCenter] postNotificationName:
kRefreshActiveCallWindowNotification object:nil];
}
Accepting the remote video offer
The application accepts the addition of video by setting video mode, initializing it with the desired video direction.
// Note that there's no guarantee that it will
// start with the expected video direction (or any) video. A completion handler
// parameter is available, but not too useful as the App should react only to
// the didUpdateVideoChannels delegate.
[call acceptVideo:CSVideoModeSendReceive completionHandler:^(NSError *error) {
if (error) {
[[ClientManager getInstance] logState: @"Unable to accept video, error:
%@", error];
_viewController.remoteVideoView.hidden = YES;
} else {
[[ClientManager getInstance] logState: @"Video has been accepted", nil];
}
}];
Decline the remote video offer
When remote video addition is detected, the local application need not accept it; the application can use the denyVideo method on the call to decline the request. The decline will be signaled back to the originator via the completionHandler of the remote's invocation of setVideoMode.
Removing video from a call
Once a video call is established between two endpoints, either endpoint can downgrade the call to audio-only; this stops both video reception and transmission. After downgrading to an audio-only call, the endpoint may be able to add video back again if desired.
Note: Some endpoints will not allow video to be restored after downgrade.
To end video on the local endpoint, the application sets CSVideoModeDisable video mode. This will deallocate the video channel id in use for the call, release the camera, and send signaling to end video on the remote endpoint.
In response to the local removal of video, the Client SDK will send up didUpdateVideoChannels with an empty list of video channels. Such an event is the key to the application to remove its render surfaces. The remote client will also see didUpdateVideoChannels delegate relating the the call's loss of video indicates that the video channel has been deleted and the camera (if running) has been stopped. The application should clean up the corresponding render surfaces.
Turning a camera on or off
Once a video call is established between two endpoints, either endpoint can turn their video camera off or on. Turning the local camera on or off means new invocations of SetVideoMode with handling of the subsequent didUpdateVideoChannels callbacks. Such an action results in a signalling update to allow the remote side to react accordingly. The result of such an action may, in fact, result in video cessation, but not video removal.
Handling remote video cessation or resumption
Once a video call is established between two endpoints, it's possible for network services to cease video temporarily. Examples of such services are hold, standard conferencing, or transfer. In such cases, the application will receive didUpdateVideoChannels callback that video has ceased. The cessation may be temporary (one or two seconds) or may be substantive (several minutes). The application will need define policies on what to display on the UI in such cases.
