Configuration and command-line arguments
Configuration and command-line
The ew-vod2cbm global service parameters can be specified in the following ways:
- On the command-line
- As environment variables
- In the configuration file (when specified by the command-line)
- Default values
This also shows the order of precedence from top to bottom, where values on the
command-line will override all other values.
The global service parameters cannot change during execution,
except for logging.
An example configuration file is available in
Example Configuration.
Note that the typical way of creating and configuring channels and assets is over the
REST API. It is also possible to configure
assets and channels in the configuration file. This can be handy mostly for testing.
A channel parameter, when present, will override the corresponding global
parameter. For example the global parameter defaultMaxliveWin would be overridden
by the channel parameter maxLiveWindowS.
The command-line
All command line parameters and environment variables can be seen by running:
ew-vod2cbm -h
Usage of ew-vod2cbm:
ew-vod2cbm produces linear video channels from VoD assets
with the same interface as the ESB3003 catchup buffer mgr (CBM).
Run ew-vod2cbm with options:
-cachedcfg string
Location of cached channel/assetpath configuration file
-cfg string
Config file
-defaultMaxBitratePercentAbove int
Global fallback percentage value for max higher allowed bitrate compared to a matching master track
-defaultMaxBitratePercentBelow int
Global fallback percentage value for max lower allowed bitrate compared to a matching master track
-defaultMaxliveWin int
Max live window [seconds] (default 14400)
-licenseFile string
Location of ew-vod2cbm license file (default "/etc/edgeware/ew-vod2cbm/license.json")
-logformat string
Format and type of log: [consolejson consolepretty journald discard] (default "consolejson")
-loglevel string
Initial log level (default "info")
-port int
Server TCP port (default 8090)
-routes
Generate a markdown file with HTTP routes
-s3file string
S3 credentials file
-timeout int
Maximum time in seconds before HTTP requests fail. Set to 0 to disable (default 60)
-timesubs
Generate time subtitles
-version
Get version, date, and possible expiration date
The corresponding environment variables are:
EW_ESB3019_PORT EW_ESB3019_DEFAULT_MAX_LIVE_WIN EW_ESB3019_TIMEOUT EW_ESB3019_LICENSE_PATH
EW_ESB3019_CFG EW_ESB3019_DEFAULT_MAX_BITRATE_PERCENT_ABOVE EW_ESB3019_LOG_FORMAT EW_ESB3019_CACHED_CFG_PATH
EW_ESB3019_S3_FILE EW_ESB3019_DEFAULT_MAX_BITRATE_PERCENT_BELOW EW_ESB3019_LOG_LEVEL
Command-line parameters will always override envirionment variables and configured values.
1 - Startup Configuration
Startup configuration for configuring ew-vod2cbm
When starting the ew-vod2cbm service, a configuration file may be provided. This file enables
starting ew-vod2cbm with predefined assets, channels and schedules. These items can also
be added and updated via the REST API when the service is running.
If a path to a cached configuration is set,
any change of the configuration via the REST API results in that the
complete configuration of asset paths, channels and their schedules are written to
a file at that path. On restart, an existing file at this path is used to restore the last
dynamic configuration, while any static configuration of these settings is skipped.
However, if a path to a cached configuration is set, either via a command line parameter -cachedcfg
or the corresponding environment variable, such a file will be used to restore the last
dynamic configuration of asset paths and channels.
The static configuration of these entities will then be ignored, unless the parsing of the cache
configuration fails.
JSON Schema for configuration file
The JSON snippet below provides a description of the schema and the parameters
that applies to configuration file for ew-vod2cbm.
The masterAssetID will be deprecated in a future release.
{
"defaultMaxBitratePercentAbove": {
"description": "When matching a variant, this sets the maximum upper boundary in percent for the bitrate",
"type": "integer",
"minimum": 0,
},
"defaultMaxBitratePercentBelow": {
"description": "When matching a variant, this sets the maximum lower boundary in percent for the bitrate",
"type": "integer",
"minimum": 0,
"maximum": 100,
},
"defaultMaxLiveWindowS": {
"description": "The maximum length of the live window in seconds",
"type": "integer",
"minimum": 10,
"maximum": 36000,
},
"assets": {
"description": "list of assets",
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"path"
],
"properties": {
"id": {
"type": "string"
},
"path": {
"type": "string"
}
}
}
},
"channels": {
"description": "Channels for ew-vod2cbm at startup",
"type": "array",
"items": {
"type": "object",
"required": [
"gopDurMS",
"name",
"nrGopsPerSegment",
"schedule"
],
"properties": {
"doLoop": {
"description": "Loop schedule or not",
"type": "boolean"
},
"gopDurMS": {
"description": "Exact millisecond duration of all video GoPs in all assets",
"type": "integer",
"minimum": 320,
"example": 2000
}
"contentTemplatePath": {
"description": "Path to the content template file. If this is used, masterAssetID should not be specified.",
"type": "string"
},
"masterAssetID": {
"description": "ID of the asset to use as a content template. If this is used, contentTemplatePath should not be specified.",
"type": "string"
},
"maxBitratePercentAbove": {
"description": "Percent diff above for when matching a track from the content template",
"type": "integer"
},
"maxBitratePercentBelow": {
"description": "Percent diff below for when matching a track from the content template",
"type": "integer"
},
"name": {
"description": "Unique name",
"type": "string",
"minLength": 2,
"example": "Channel 1"
},
"nrGopsPerSegment": {
"description": "How many GoPs to include in an average output segment",
"type": "integer",
"minimum": 1,
"example": 3
},
"schedule": {
"description": "The current scedule of the channel",
"type": "object",
"required": [
"entries"
],
"properties": {
"entries": {
"description": "list of programs or other entries",
"type": "array",
"items": {
"type": "object",
"required": [
"assetID",
"length",
"name"
],
"properties": {
"assetID": {
"description": "Asset identifier",
"type": "string",
"minLength": 2,
"example": "asset1234568-22"
},
"length": {
"description": "How many GoPs to play in asset. 0 is until end of asset. Beyond end results in wrap to start",
"type": "integer",
"example": 2400
},
"name": {
"description": "Name to include in EPG",
"type": "string",
"minLength": 2,
"example": "The Shark"
},
"offset": {
"description": "Zero-based GoP nr to start in asset. Negative value means from end",
"type": "integer"
},
"scteEventID": {
"description": "SCTE-35 Event ID in SCTE message. A non-zero value signals an ad. If scteEventID is set to -1, a unique event ID corresponding to the GoP number where the ad break starts will be inserted in the output.",
"type": "integer",
"minimum": -1
}
}
}
},
"gopNrAfterLastAd": {
"type": "integer"
},
"gopNrAtScheduleStart": {
"type": "integer"
}
}
},
"startTimeS": {
"description": "Start time relative epoch (1970-01-01) in seconds",
"type": "integer",
"minimum": 0
}
}
}
}
}
Example configuration
An example configuration file for two channels where all assets have relative
paths may look like:
{
"defaultMaxBitratePercentAbove": 5,
"defaultMaxBitratePercentBelow": 20,
"assets": [
{"id": "bbb", "path": "app/testdata/assets/bbb"},
{"id": "slates/slate_pre_ad", "path": "app/testdata/assets/slates/slate_pre_ad"},
{"id": "slates/slate_post_ad", "path": "app/testdata/assets/slates/slate_post_ad"},
{"id": "slates/slate_black", "path": "app/testdata/assets/slates/slate_black"},
{"id": "whistler", "path": "app/testdata/assets/whistler"},
{"id": "ed", "path": "app/testdata/assets/ed"}
],
"channels": [
{
"name": "quick",
"gopDurMS": 1920,
"nrGopsPerSegment": 2,
"contentTemplatePath": "location/of/content_template.json"
"startTimeS": 0,
"doLoop": true,
"maxBitratePercentAbove": 10,
"maxBitratePercentBelow": 30,
"schedule": {
"GopNrAtScheduleStart": 0,
"GopNrAfterLastAd": 0,
"Entries": [
{
"name": "Big Buck Bunny",
"assetID": "bbb",
"offset": -3,
"length": 3
},
{
"name": "Ad upcoming",
"assetID": "slates/slate_pre_ad",
"offset": 0,
"length": 1,
"scteEventID": 123
},
{
"name": "Whistler ad",
"assetID": "whistler",
"offset": 0,
"length": 4,
"scteEventID": 123
},
{
"name": "Ad finished",
"assetID": "slates/slate_post_ad",
"offset": 0,
"length": 1,
"scteEventID": 123
},
{
"name": "Elephants dream",
"assetID": "ed",
"offset": 3,
"length": 4
}
]
}
},
{
"name": "test",
"gopDurMS": 1920,
"nrGopsPerSegment": 3,
"contentTemplatePath": "location/of/content_template.json"
"startTimeS": 0,
"doLoop": true,
"schedule": {
"GopNrAtScheduleStart": 0,
"GopNrAfterLastAd": 0,
"Entries": [
{
"name": "Big Buck Bunny",
"assetID": "bbb",
"length": 0
},
{
"name": "Black 10s",
"assetID": "slates/slate_black",
"offset": 0,
"length": 5
},
{
"name": "Whistler ad",
"assetID": "whistler",
"length": 0,
"scteEventID": 123
},
{
"name": "Elephants dream",
"assetID": "ed",
"offset": 0,
"length": 0
}
]
}
}
]
}
2 - Content Templates
The role of content templates in ew-vod2cbm
Contents templates are used in ew-vod2cbm to specify the output segmentation and tracks
of a channel, but also serve as measures to check if asset entries in a schedule
are compatible with the channel.
They are further used to generate the manifest and playlist media data
properties as well as the media init segments. Therefore, bitrate,
resolution, frame rate, video parameter sets, number audio channels are included
in the template. The assets do not need to have exactly the same parameter values,
but are allowed to vary as described later.
The generally preferred way of creating content templates, is to use
ew-content-template-tool, described below.
JSON Schema for Content Template
The JSON schema of the content template contains core parameters of the
content_info.json schema, and some additional parameters for bitrate matching.
It can contain variants of video, audio, and subtitles as signaled in the media_type field.
The variant array can contain multiple variants
of each type. The table below the schema shows the required parameters for each
media type.
{
"type": "object",
"properties": {
"version": {
"type": "string"
},
"constant_gop_duration_ms": {
"type": "integer"
},
"variants": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"media_type": {
"type": "string"
},
"subtype": {
"type": "string"
},
"name": {
"type": "string"
},
"bitrate": {
"type": "integer"
},
"width": {
"type": "integer"
},
"height": {
"type": "integer"
},
"sample_aspect_ratio": {
"type": "string"
},
"picture_aspect_ratio": {
"type": "string"
},
"scan_type": {
"type": "string"
},
"sps": {
"type": "string"
},
"pps": {
"type": "string"
},
"codec": {
"type": "string"
},
"frame_rate_fraction": {
"type": "array",
"items": [
{
"type": "integer"
},
{
"type": "integer"
}
]
},
"num_channels": {
"type": "integer"
},
"samplerate": {
"type": "integer"
},
"decoder_config": {
"type": "string"
},
"lang": {
"type": "string"
},
"max_bitrate": {
"type": "integer"
},
"min_bitrate": {
"type": "integer"
}
},
"required": [
"media_type",
"name",
"bitrate",
"codec"
]
}
]
}
}
}
Depending on the media type, there are required fields for a
variant. The table below lists all the required fields depending on
media type.
| Media type | Required fields |
|---|
| Video | media_type
subtype
name
bitrate
width
height
sample_aspect_ratio
picture_aspect_ratio
scan_type
sps
pps
codec
frame_rate_fraction
|
| Audio | media_type
subtype
name
bitrate
codec
num_channels
samplerate
decoder_config
lang |
| Subtitles | media_type
name
bitrate
lang
codec |
The fields max_bitrate and min_bitrate are optional for all media types.
Example content template
{
"version": "1.0",
"constant_gop_duration_ms": 1920,
"variants": [
{
"media_type": "video",
"subtype": "h264",
"name": "V1000",
"bitrate": 1000000,
"width": 640,
"height": 360,
"sample_aspect_ratio": "1:1",
"picture_aspect_ratio": "16:9",
"scan_type": "progressive",
"sps": "6764001eacb201405ff2a0",
"pps": "68ebccb22c",
"codec": "avc1.64001E",
"frame_rate_fraction": [
25,
1
],
"max_bitrate": 1500000,
"min_bitrate": 900000
},
{
"media_type": "video",
"subtype": "h264",
"name": "V1500",
"bitrate": 1500000,
"width": 768,
"height": 432,
"sample_aspect_ratio": "1:1",
"picture_aspect_ratio": "16:9",
"scan_type": "progressive",
"sps": "6764001eacb201806f20",
"pps": "68ebccb22c",
"codec": "avc1.64001E",
"frame_rate_fraction": [
25,
1
],
"max_bitrate": 1600000,
"min_bitrate": 1400000
},
{
"media_type": "audio",
"subtype": "aac",
"name": "A128",
"bitrate": 128000,
"codec": "mp4a.40.2",
"num_channels": 2,
"samplerate": 48000,
"decoder_config": "1190",
"lang": "eng"
},
{
"media_type": "subtitles",
"name": "eng_1",
"bitrate": 1000,
"lang": "eng",
"codec": "wvtt"
}
]
}
Creating a Content Template: ew-content-template-tool
A content template does not need to created by hand. It can be created from an ingested asset with:
$ ew-content-template-tool
Usage: bin/ew-content-template-tool [OPTIONS] <content info path> <content_template_path> ...
Options:
-gop-dur-ms int
GOP Duration for template
-version
Get version and build date
For example:
$ ew-content-template-tool -gop-dur-ms 1920 content_info.json content_template.json
Note: The tool makes no verification that the media files actually fulfill
the specified GOP duration.
Verifying Assets Before Scheduling: ew-verify-asset
Before attempting to schedule an asset, it can be verified that it matches a
specific content template, with:
$ ew-verify-asset
Usage: ew-verify-asset [options]
-i, --input string Asset path
-t, --template string Content template file for the asset(s)
For example:
$ ew-verify-content -i media/matrix-revolutions/ -t content-templates/template_4sGOP_hevc.json
This tool will verify that the asset with regards to video-tracks, codecs etc. Missing subtitle tracks, and (secondary) audio tracks are allowed, and yield warnings.
Non-constant GOP lengths are detected, and rejected, for assets ingested with ew-vodingest. Such irregularities are not detected for assets ingested with the older ew-recorder.
3 - Assets
Assets (content_info.json) in ew-vod2cbm
The video content of a ew-vod2cbm consists of VoD video assets. Assets
are added to the server and then added to the schedule of a channel (this is
done either using the Schedule API or the Config) to the schedule of a channel. The way an asset is
specified for vod2cbm, is using a file path or URL to either a directory that contains a
content_info.json file or directly to a DASH OnDemand manifest (must end with
file extension .mpd). The content_info or manifest file of the asset contains the
necessary metadata to start processing it.
Asset requirements
The media tracks must be in one of the
supported codecs and formats.
All assets of a channel must have a GoP duration that is “compatible” with the value specified in
the channel configuration. Compatible here means that the channel GoP duration must be a multiple
of the asset GoP duration. For example, an asset with 1s GoP duration is compatible with a channel with 2s
GoP duration.
That value is specified in the channel configuration as gopDurMS.
As a schedule is updated, all included assets’ metadata as well as the first part of each
video track is loaded in order to derive a GoP duration for that asset.
The metadata is cached for future use, while the actual video and audio media data
is not.
Another requirement is that all media samples in an mp4 files must have the
same duration.
This is normally no problem if the content has fixed frame rate
and a timescale that is compatible with the frame rate and audio sample rate.
Padding the last asset GoP
It is not a requirement that the last GoP of an asset has the same duration as the others. It is natural
that the last GoP is shorter. Using the boolean padLastGop parameter
in the channel/schedule configuration it is possible to determine whether
that last GoP will either be discarded or padded with silent audio and black video frames
to become as long as the others. The GoP duration in this context is the channel’s GoP
duration.
The padding influences the length of the asset.
As an example, consider an asset that has 32.5s duration, and has an IDR-frame every 1s.
If used in a channel with GoP duration 2s, the asset length is considered to be 32s
if padLastGop is false, and 34s if padLastGopis true. In the latter case, the last
1.5s will be padded with silent audio and black video frames to achieve the full duration
of an 2s GoP. The GoP duration of 1s of the actual asset, does not matter here, since
all segments are built from GoPs of the duration specified in the channel’s configuration.
If the channel entry for the asset specifies a length that wraps the asset to start from
the beginning, the padding is included in the video so that a padded entry wraps later than
a non-padded entry.
The padding content
The silent audio is built by repeating silent audio frames from the corresponding audio
codec and setting the timing parameters appropriately.
The black video is built by taking frames from a 500frames long I-P-P-P- sequence
of AVC or HEVC video. Since the content is just a black frame, the resolution does not
influence the image quality, so a fixed resolution of 640x360 pixels is currently used.
Since this content generally doesn’t match with the content of the variant being played,
all video is sent using inband signaling parameter sets, that is avc3 for H.264/AVC
hev1 for H.265/HEVC.
3.1 - Asset Track Matching
Matching of tracks in assets for ew-vod2cbm
Matching rules
For a schedule to be valid, all assets must be compatible with the content
template of the channel.
The compatibility check is done by a set of matching rules when a channel is created
or when its schedule is updated.
If one of the assets is not matching the content template, the schedule is discarded
and an error is reported.
The matching is done by comparing the asset tracks with the tracks of
the content template.
The first condition is equal media type.
A video track will, for example, never be matched to an audio track.
The second condition depends on the media type of the track.
Some media type-specific properties of a track, must always be identical to
a track found in the content template for them to match. The table below lists
the properties that must be identical for the respective media types between a
track in an asset and a track in the content template.
| Media type | Media properties that must match |
|---|
| Video | Media type
subtype |
| Audio | Media type
subtype
Codec
Sample rate |
| Subtitles | Media type
Language
Role |
The third condition regards the bitrate range.
By default, the bitrate of an asset track must be identical to the bitrate of
the corresponding content template track. However, the max_bitrate and
min_bitrate parameters in the content template allow for a range
of matching bitrates on track level.
More generally, there are also percentage values that can be specified for
bitrates in either a channel configuration, or as global parameters
at the top level of the config file or as command line parameters.
The channel parameters are called maxBitratePercentAbove and maxBitratePercentBelow,
while the global parameters are called defaultMaxBitratePercentAbove
and defaultMaxBitratePercentBelow These are all optional with default value 0.
The rules for bitrate matching are applied in the following priority order:
- Use the track specific bitrate range if it exists
- Use the channel specific bitrate percentage range if it exists
- Use the command-line bitrate percentage range if it exists
- Use the top-level JSON bitrate percentage range if it exists
Only one of the rules will be applied (the one with the highest priority that is
valid for the scope of the track we are trying to match to).
Cases of non-direct match
There does not need to be a one-to-one match between tracks.
An asset which deviate from the content template in the following way should be accepted:
- Extra tracks are allowed and will be silently discarded
- Subtitle tracks may be missing. This results in corresponding empty subtitle segments
- Audio languages may be missing. Such a track will be replaced by tracks of another language.
An example would be a channel where some programs are in English and some in Spanish.
Matching algorithm
The matching occurs when a schedule is being updated for a channel. If no
match is found for any variant in any asset in the new schedule, the
schedule is considered invalid, and an error is returned. The matching
algorithm begins by sorting the variants in the content template by their
bitrate in descending order. The same is done for every asset. Finally, each
variant in the sorted list of template variants, will be matched with a
compatible asset variant considering media type and
the compatibility values listed in the table above.
The search for a compatible variant in the asset is started from the top of the sorted variant list in
the content template. This ensures that the most appropriate variant in each
asset will be coupled with a corresponding variant in the content template.
4 - Repackager configuration
How to configure the repackager
The ESB3002 repackager needs to be configured with a new
“ingestServer” corresponding to the ew-vod2cbm node.
First define a new ingestServer under ingestServers
services.repackaging.locations.ingestServers
"ingestServers": [
{
"name": "vod2cbm",
"url": "http://127.0.0.1:8090"
}
]
Then define a new group channelGroup for vod2live content:
services.repackaging.content.channelGroups
{
"channelGroups": [{
"locations": [
"vod2cbm"
],
"name": "vod2live"
}
]
}
5 - Channels
Channel and schedule configuration
As described in the Startup Configuration
section, one can start up the service with both assets and channels defined in
the config file.
However, one can change this configuration by using the REST API.
In this way one can
- Add asset paths
- Add channels
- Replace schedules of non-looping channels
- Delete channels
- Delete asset paths for assets which are not in any schedule
Schedule updates are further described in the
Dynamic Channels section.
If a path to a cached configuration is set,
any change of the configuration via the REST API results in that the
complete configuration of asset paths, channels and their schedules are written to
a file at that path. On restart, an existing file at this path is used to restore the last
dynamic configuration, while any static configuration of these settings is skipped.
6 - Licensing
Licensing for ew-vod2cbm
ew-vod2cbm requires a valid license file to operate. The license file must be
obtained from Agile Content, and copied to /etc/edgeware/ew-vod2cbm/license.json.
See further details here.