Engine Integration

Integration

The Genvid MILE SDK API consists of a single header file written in C: genvid.h. Integrating the SDK only requires including the header file and adding the proper shared library.

Add genvid.h to the include path of your compilation environment.

Link with the proper version of the shared library.
The Genvid MILE SDK includes both a 32- and 64-bit version for Windows.

Deploy the corresponding DLL alongside your game.
This typically consists of copying Genvid.dll next to your game executable.

Verify the operation was successful.
All Genvid MILE SDK methods return a GenvidStatus which you can use to determine if the operation was done properly. You can also use Genvid_StatusToString() to convert the enum value into a const char* value.

Program flow

The Genvid MILE SDK program flow consists of 4 steps:

Initialization
Configuration
Streaming
Termination

Initialization

Before calling any other routine, you initialize the Genvid library by calling Genvid_Initialize():

GenvidStatus gs = Genvid_Initialize();
if(gs != GenvidStatus_Success)
{
   // Something bad happened; you can't use Genvid.
   return false;
}

Initialization starts the asynchronous handler, which connects to Consul and reads the configuration. It automatically retries if the connection is not successful.

Calling any Genvid MILE SDK routine without properly initializing the Genvid library will result in a GenvidStatus::GenvidStatus_UninitializedSDK error.

Once it reads the configuration it starts processing requests.

Configuration

After the library is initialized, you create a stream using a unique name for the streamID.

Important

Your stream name must not contain genvid in it.

A “stream” is the context Genvid uses to associate audio, video, and game data. The streamID is used to reference a specific stream across all relevant calling locations.

Create a stream by calling GenvidStatus Genvid_CreateStream(const char *streamID):

GenvidStatus gs;
gs = Genvid_CreateStream("audio");
if(gs != GenvidStatus_Success)
{
   return false;
}
gs = Genvid_CreateStream("video");
if(gs != GenvidStatus_Success)
{
   return false;
}
gs = Genvid_CreateStream("game.players");
if(gs != GenvidStatus_Success)
{
   return false;
}
gs = Genvid_CreateStream("game.camera");
if(gs != GenvidStatus_Success)
{
   return false;
}

There are a few parameters you need to set before sending data to the stream.

You set all parameter assignments with one of these functions:

The Genvid_SetParameter() functions are variations of the same routine for integer, floating-point, and pointer values. Each function requires the following inputs:

streamID
The stream name you set during initialization.

parameter key
The name of the parameter to change. The key is case insensitive.

parameter value
The specific type for the variant used.

Calling a Genvid_SetParameter() function without a proper parameter key results in a GenvidStatus::GenvidStatus_InvalidParameter error.

Common parameters

The framerate parameter tells Genvid what the expected video-data frequency is. It can be an integer or a floating-point number:

GenvidStatus gs;
// Sending video 30 times a second.
gs = Genvid_SetParameterInt("video", "framerate", 30);
if(gs != GenvidStatus_Success)
{
   return false;
}
// Sending slow data once every 10 seconds.
gs = Genvid_SetParameterFloat("slowdata", "framerate", 0.1f);
if(gs != GenvidStatus_Success)
{
   return false;
}

Once you define the framerate, timecodes round to the nearest value. The default framerate for new streams is 30 Hz.

The video source parameter specifies where to grab the video data from when streaming. Since the Genvid MILE SDK currently only supports D3D11 sources, it must be either a DXGI swap chain or a 2D texture.

To specify the IDXGISwapChain, use Video.Source.IDXGISwapChain parameter key with a pointer:

IDXGISwapChain* mySwapChain = nullptr;
// Some code assigning mySwapChain to a valid value...
GenvidStatus gs = Genvid_SetParameterPointer("video", "Video.Source.IDXGISwapChain", mySwapChain);
if(gs != GenvidStatus_Success)
{
   return false;
}

For a 2D texture resource handle, use Video.Source.ID3D11Texture2D instead:

ID3D11Texture2D* myTexture = nullptr;
// Some code assigning myTexture to a valid value...
GenvidStatus gs = Genvid_SetParameterPointer("video", "Video.Source.ID3D11Texture2D", myTexture);
if(gs != GenvidStatus_Success)
{
   return false;
}

Specifying any of the video source parameters activates automatic frame grabbing. You still need to tell Genvid when the frame is ready (see Video Streaming) from the rendering thread.

Genvid supports passing a raw buffer in various pixel-formats if you’re using a custom capture method. When using those formats, you specify the width, height, and pixel format of the buffer instead of specifying a video source.

assert(description.Format == DXGI_FORMAT_R8G8B8A8_UNORM);
gs = Genvid_SetParameterInt(sStream_Video.c_str(), "video.pixel_format", GenvidPixelFormat_R8G8B8A8);
assert(gs == GenvidStatus_Success);
gs = Genvid_SetParameterInt(sStream_Video.c_str(), "video.width", description.Width);
assert(gs == GenvidStatus_Success);
gs = Genvid_SetParameterInt(sStream_Video.c_str(), "video.height", description.Height);
assert(gs == GenvidStatus_Success);

Important

Setting a video source when using this option activates auto-capture, which results in a conflict.

The format you use must be packed, which means that the frame-buffer pitch must be the same as the width of the image.

The Genvid MILE SDK can also auto-capture the audio stream. To enable auto-capture, set the Audio.Source.WASAPI parameter to 1:

GenvidStatus gs = Genvid_SetParameterInt("audio", "Audio.Source.WASAPI", 1);
if(gs != GenvidStatus_Success)
{
   return false;
}

Note: Enabling WASAPI auto-capture sets the audio stream granularity to the sampling rate.

The GENVID_AUDIO_DEVICE environment variable holds a string defining what audio device should be used. It is not defined by default in local mode, but is defined to “CABLE Input” in cloud configuration as this is what will most likely be used. If the specified device cannot be found then the default audio endpoint will be used.

We recommend you set the audio framerate to the same value you set your video source.

When audio auto-capture is active, the default mixing format and sample rate are used. By default, Windows uses stereo floating-point samples at 48 kHz.

It is also possible to perform the capture without the auto-capture (supported for the Unity engine currently). You need to set 3 parameters to the audio stream before being able to send data:

audio.rate
audio.format
audio.channels

The granularity defines the precision with which the timecodes get snapped. It can be set for any stream. We recommend setting it to the sample rate to more accurately represent the time code at which the data was sampled.

GenvidStatus gs;
float sampleRate = 48000;
gs = Genvid_SetParameterFloat("audio", "granularity", sampleRate);
if (gs != GenvidStatus_Success)
{
   return false;
}

If not specified, the granularity will default to the sample rate. If the sample rate is not available, the granularity will default to the framerate.

The table below summarizes the most important stream parameters.

Note

Parameter keys are case-insensitive.

Table 4 List of the main stream parameters.
Key	Type	Methods supported	Description	Default
`framerate`	float	set and get	Specifies the framerate.	30
`granularity`	float	set and get	The granularity defines the precision with which the timecodes get snapped. Defaults to the sample rate if available. Otherwise defaults to the framerate.	1.0
`video.source.idxgiswapchain`	pointer	set and get	Specifies the D3D11 swap chain to use for video source. This activates video auto-capture. If this capture method is used, the id3d11texture2d capture can’t be used.	N/A
`video.source.id3d11texture2d`	pointer	set and get	Specifies the D3D11 texture to use for video source. This activates video auto-capture. If this capture method is used, the idxgiswapchain capture can’t be used.	N/A
`video.pixel_format`	integer	set	Sets the pixel format of the video frame. The format is specified by the `GenvidPixelFormat`. This option is incompatible with video auto-capture.	N/A
`video.width`	integer	set	Sets the width of the video frame. This option is incompatible with video auto-capture.	N/A
`video.height`	integer	set	Sets the height of the video frame. This option is incompatible with video auto-capture.	N/A
`video.useopenglconvention`	integer	set	Setting this parameter to 1 flips the video stream along the horizontal axis.	0
`audio.source.wasapi`	integer	set and get	Turns WASAPI audio capture on or off (1 or 0). This activates audio auto-capture.	0
`audio.rate`	integer	set and get	Changes the sample rate of the audio.	0
`audio.format`	integer	set and get	Changes the format of the audio to `GenvidAudioFormat::GenvidAudioFormat_S16LE` or `GenvidAudioFormat::GenvidAudioFormat_F32LE`	N/A
`audio.channels`	integer	set and get	Changes the number of channels of the audio.	0

Parameter Retrieval

You can retrieve the current state of the Genvid MILE SDK using the following functions:

GenvidStatus Genvid_GetParameterInt(const char *streamID, const char *paramKey, int *paramValue)

GenvidStatus Genvid_GetParameterFloat(const char *streamID, const char *paramKey, float *paramValue)

GenvidStatus Genvid_GetParameterPointer(const char *streamID, const char *paramKey, void *paramValue)

The ‘’Genvid_GetParameter’’ functions are variations of the same routine for integer, floating-point, and pointer values.

To get parameter-key values from Consul, use:

For Genvid_GetParameterInt() and Genvid_GetParameterFloat(), use:

genvid.kv as the stream id,

paramKey as the key name, and

paramValue as the destination for the value.

int score = 0;
GenvidStatus gs = Genvid_GetParameterInt("genvid.kv", "genvid/game/score", &score);

float time = 0.0f;
GenvidStatus gs = Genvid_GetParameterFloat("genvid.kv", "genvid/game/time", &time);

For Genvid_GetParameterUTF8(), you can use:

any value as the stream id,

paramKey as the key name,

dstBuffer as the container receiving the value, and

dstBufferSize as the buffer size.

If the key received is smaller than the available allocation size, a GenvidStatus::GenvidStatus_Incomplete warning is returned.

char playerName[10];
GenvidStatus gs = Genvid_GetParameterUTF8("genvid.kv", "genvid/game/playername", playerName, 10);

Subscriptions

Until now, we showed how to push different kinds of information to the spectators. Now is the time to have a look at what the game can receive from the Genvid stack.

There are two types of incoming messages a game can subscribe to: events and commands.

Events

Spectators pass events back to the game through our web API. Events consist of key-value objects that give the game synthetic information processed from potentially massive spectator feedback.

Note

You can also send batches of events from an authenticated machine using the Webgateway service. This can be useful if you don’t want to send your events directly from the spectator browser.

In the game-side SDK, you can access these processed events (i.e., the events after they have been processed through the Genvid services) by giving a callback to a subscription function.

More specifically, use GenvidStatus Genvid_Subscribe(const char *id, GenvidEventSummaryCallback callback, void *userData) to subscribe to events. You have to specify the specific event you are interested in, a callback, and some optional data pointers to send to the callback. (You can use nullptr if you don’t need any.)

GenvidStatus gs = Genvid_Subscribe("some_event_id", &SomeEventCallback, nullptr);

This event callback would have the following signature:

void SomeEventCallback(const GenvidEventSummary* summary, void* userData*)
{
    // summary->id would be equal to "some_event_id".
    // summary->results[] would contain the results of the MapReduce.
    // userData would contain nullptr (but could be anything the user specifies).
    // ...
}

See typedef void (*GenvidEventSummaryCallback) (const GenvidEventSummary *summary, void *userData) for more information on the callback.

See struct GenvidEventSummary for more information on the structure returned.

See Event overview for more information about defining MapReduce operations.

You can register the same callback using different event ids or using data pointers.

Commands

Commands are special immediate messages used to send succinct directives to the game process. They are non-scalable and should be used with caution.

An example use case could be to control directly from a match administration web page which map to load next.

Commands are sent using the POST /commands/game URI on the Webgateway service.

Warning

Commands should not be sent from the spectator website. Keep in mind that a broadcast stream can host thousands of spectators. For gathering spectator feedback and altering the game loop accordingly, consider using the provided event system. Sending many commands to a game for each spectator input can have a negative impact on the broadcast experience or even crash the game.

Command subscriptions use GenvidStatus Genvid_SubscribeCommand(const char *id, GenvidCommandCallback callback, void *userData):

GenvidStatus gs = Genvid_SubscribeCommand("some_command_id", &SomeCommandCallback, nullptr);

This command callback would have the following signature:

void SomeCommandCallback(const GenvidCommandResult* result, void* userData)
{
    // result->id would be equal to "some_event_id".
    // result->value would contain whatever string was sent.
    // userData would contain nullptr (but could be anything the user specifies).
    // ...
}

See typedef void (*GenvidCommandCallback) (const GenvidCommandResult *result, void *userData) for more information on the callback.

See struct GenvidCommandResult for more information on the structure returned.

See Webgateway service API for more information about the /command endpoints served by the webgateway service.

Streaming

You must tell Genvid to proceed every time you have data you want to stream.

You also must associate a timecode with every data chunk, so Genvid can properly place it inside the stream. Providing an out-of-order timecode (for example, a timecode prior to the last one used) causes an error with undefined results.

Timecode

The Genvid MILE SDK provides a routine to retrieve the current timecode: GenvidTimecode Genvid_GetCurrentTimecode(). It returns a special typedef int64_t GenvidTimecode value.

Using a timecode of -1 is equivalent to calling Genvid_GetCurrentTimecode() and using that result.

If you want multiple data chunks to share the same timecode, call Genvid_GetCurrentTimecode() once and reuse the result in each subsequent Genvid_Submit*Data() call.

You can retrieve the previous value returned by Genvid_GetCurrentTimecode() by calling GenvidTimecode Genvid_GetPreviousTimecode(). You can use this when you want multiple code locations to use the same timecode, but carrying the GenvidTimecode is a problem.

Audio Streaming

Genvid currently supports two audio mode: auto-capture and manual-capture.

auto-capture automatically captures the sound from the audio output device and send it by calling the function Genvid_SubmitAudioData() internally.

manual-capture need an explicit call to the function Genvid_SubmitAudioData() on the game side.

The auto-capture is the easiest way. All you have to do is activate it.

See Common parameters for more information to activate or setup the audio data stream.

Table 5 auto-capture.
Pros	Cons
Audio capture system already implemented in the SDK	Capture all audio data (speaker, input, microphone, etc.)
Automatic audio setup (format, rate and channels)
Game Engine agnostic

Table 6 manual-capture.
Pros	Cons
Game Engine agnostic	Need game side audio capture implementation
Control on the audio data stream
Avoid sound problems in local mode

Video Streaming

When your rendered frame is ready, you inform Genvid that it can capture and encode the frame by calling GenvidStatus Genvid_SubmitVideoData(const GenvidTimecode timecode, const char *streamID, const void *videoData, const int videoDataSize.

You only send the timecode and streamID because you configured the video source previously.

See Video Source Parameter for more information.

In this example, we send a null pointer and 0 as the last two parameters for that reason.

GenvidTimecode tc = Genvid_GetCurrentTimecode();
GenvidStatus gs = Genvid_SubmitVideoData(tc, "video", nullptr, 0);
if(gs != GenvidStatus_Success)
{
   return false;
}

When this routine is called, the Genvid MILE SDK captures the video source, encodes it, then streams it. It’s done asynchronously to be as efficient as possible.

Game-Data Streaming

Every game data chunk must specify a timecode and a streamID in addition to a byte array (data and size) containing the actual game data.

Note

Genvid currently makes no assumption on that data, and carries raw bytes through the service. We’re considering placing stricter requirements on game-data format for bandwidth savings.

This sample shows how to stream both camera and player data via separate streams using the same timecode using GenvidStatus Genvid_SubmitGameData (const GenvidTimecode timecode, const char *streamID, const void *gameData, const int gameDataSize):

// Fake runtime data.
std::string playerData = R"({ "P1": {"position": [0, 0, 0]}, "P2": {"position":�[1, 0, 0]} })";
std::string cameraData = R"({ "matrix": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]�})";

GenvidStatus gs;
GenvidTimecode tc = Genvid_GetCurrentTimecode();
gs = Genvid_SubmitGameData(tc, "game.player", playerData.data(), (int)playerData.size());
if(gs != GenvidStatus_Success)
{
    return false;
}
gs = Genvid_SubmitGameData(tc, "game.camera", cameraData.data(), (int)cameraData.size());
if(gs != GenvidStatus_Success)
{
    return false;
}

Game-Annotation streaming

The annotations are similar to data streaming. The main difference is how the system and JavaScript handle the information. Annotations are stream-specific and are limited to 256 annotations per stream, per frame.

This sample shows how to send annotations about kill confirmations using GenvidStatus Genvid_SubmitAnnotation (const GenvidTimecode timecode, const char *streamID, const void *annotationData, const int annotationDataSize):

// Fake runtime data.
std::string confirmKills = R"({ "kills": ["Player1", "Player2"] })";

GenvidStatus gs;
GenvidTimecode tc = Genvid_GetCurrentTimecode();
gs = Genvid_SubmitAnnotation(tc, "game.confirmKills", playerData.data(), (int)playerData.size());
if(gs != GenvidStatus_Success)
{
    return false;
}

Game Notification

A notification sends information as fast as possible to the web site, such as realtime updates. Notifications are not persistant and are not associated with a timecode.

This sample shows how to send notifications about a player’s popularity using GenvidStatus Genvid_SubmitNotification (const char* notificationID, const void* notificationData, const int notificationDataSize):

// Fake runtime data.
std::string popularityData = R"({ "players": [{"name":"player1", "popularity": 2},{"name":"player2", "popularity": 10}] })";

GenvidStatus gs;
gs = Genvid_SubmitNotification("game.popularity", popularityData.data(), (int)popularityData.size());
if(gs != GenvidStatus_Success)
{
    return false;
}

Before a game creates a new stream, it must terminate the Genvid Cluster. Then it creates and configures the new stream and restarst the cluster. This limitation exists so the cluster can reserve resources for properly handling all potential streams.

Callback verification

If you have callbacks for any events or commands while streaming, use GenvidStatus Genvid_CheckForEvents() to use the callback when new results are available.

We recommend you use this check regularly to get results as quickly as possible. In the Tutorial sample, we use Genvid_CheckForEvents() whenever we send data.

Note

When events are available, GenvidStatus returns GenvidStatus::GenvidStatus_Success. However, if there aren’t any events, GenvidStatus::GenvidStatus_ConnectionTimeout is returned instead.

Termination

When you’re done with a stream, you can deallocate it using GenvidStatus Genvid_DestroyStream(const char *streamID) with the proper streamID:

Genvid_DestroyStream("game.player");
Genvid_DestroyStream("came.camera");
Genvid_DestroyStream("video");

This releases any internal storage related to the specified stream, such as any temporary buffer required for streaming or encoding.

Using the streamID of a destroyed stream results in a GenvidStatus::GenvidStatus_InvalidState error.

Both subscription routines have an equivalent one to unsubscribe:

Genvid_Unsubscribe("some_event_id", &SomeEventCallback, nullptr);
Genvid_UnsubscribeCommand("some_command_id", &SomeCommandCallback, nullptr);

Sending a nullptr for any argument matches any existing callback for that callback.

For example, to unsubscribe from all event callbacks registered to call the function SomeEventCallback(), use:

Genvid_Unsubscribe(nullptr, &SomeEventCallback, nullptr);

To unsubscribe from all existing command callbacks, use:

Genvid_UnsubscribeCommand(nullptr, nullptr, nullptr);

Once you are done using the Genvid MILE SDK, terminate the API by calling GenvidStatus Genvid_Terminate():

GenvidStatus gs = Genvid_Terminate();
if(gs != GenvidStatus_Success)
{
    return false;
}

Once the API is terminated, you have to reinitialize it to make any new calls.