Introduction

Introduction to ew-vodingest

The ew-vodingest command may be used to perform either VoD ingest or to perform live recordings from an ew-live-ingest (esb3003) circular buffer.

VoD ingest

ew-vodingest (ESB3021) reads and ingests an MP4/ts audio and video + subtitles asset defined by a SMIL file into an ESF (Edgeware Storage Format) asset in a way that is compatible with the DASH OnDemand profile. A corresponding DASH MPD file is generated. The SMIL-file syntax and usage is explained in the SMIL section.

It is also possible to specify a .ts-file, an .mp4-file or a directory-path as input.

For example:

ew-vodingest --input hitchcock/vertigo/index.smil --output esf/vertigo

Note: A main focus of this tool is to ingest VoD content that can be used to generate live content by combining VoD assets using ESB3019 ew-vod2cbm. There is therefore a restriction on the ingested content that it must be possible to discover a common constant GoP duration (sync-frame distance) for all video tracks, and that all sample durations are exactly the same in their respective timescales. The timescales may differ, but only by integer factors, for example 50 for 50Hz video and 25 for 25Hz video.

Codec and input format

Please refer to Supported codecs to get more information.

Input locations

This tables shows the possible combinations of input types and locations:

filesystemhttp/httpsS3FTP
directoryyesnoyesyes
SMILyesyesyesyes
MP4yesnonono
tsyesnonono
DASH On-Demandyesyesyesyes
HLSyesyesyesyes
Unencrypted ISM/MSSyesyesyesyes

For ingesting large assets (or for slow locations) the --maxtime command-line parameter can be used to increase the timeout threshold.

Output locations

Output locations can always be local filesystem, or network locations, as above. By default the content_id in content_info.json of the output asset will equal the output directory. This can be overridden by specifying any other ID with the --content-id command-line parameter.

Temporary storage

The temporary storage, specified by --local-storage, is used to store the input files from NAS (HTTP, FTP, S3, etc) VOD to reduce overhead, and output files of recording, VOD before uploading to reduce the stress on remote file system.

For those reasons, it is recommended to set it to local filesystem on SSD. The default value is /tmp, which is a RAM disk, best for speed, but has limited capacity.

Live recordings

The input for live recordings is an ew-live-ingest (esb3003) with a circular buffer available on the instance specified by --host and --port (default localhost:8090). The output locations are the same as for VoD ingest above.

The --cbm flag triggers a live recording (and cannot be combined with --i). The start/stop times are specified in epoch seconds, and must be in the past. The recording will consist of integral segments, matching the specified time range as closely as possible. The time-stamps in the resulting VoD will be shifted to start from zero. Any “holes” in the specified intervals will be removed, thus resulting in a recording shorter than the specified interval.

SCTE-35 markers are transferred from the CBM interval and stored in a separate file.

For example:

ew-vodingest --cbm --start-epoch 1678190071 --stop-epoch 1678190099 --channel tv1 -o tv1-mar-7-2023

Note: Recording from end-of-life VCP Origin is not supported by ew-vodingest.

Usage

ew-vodingest is a command-line tool that is run as:

ew-vodingest [options]

Run as ew-vodingest [options]

  -i, --input string                        SMIL file, dir or file pattern defining the asset (mandatory). Can't be used with --cbm.
  -o, --output string                       output directory (mandatory).
  -w, --minseg int                          minimum output segment duration (milliseconds) (default 4000).
  -y, --maxseg int                          maximum output segment duration (milliseconds) (default 12000).
      --cbm                                 set to use esf export from cbm. Can't be used with --input.
      --start-epoch uint                    recording start time given in epoch seconds. Only works with --cbm.
      --stop-epoch uint                     recording stop time given in epoch seconds. Only works with --cbm.
      --channel string                      channel name to record from. Only works with --cbm.
      --drop-non-aligned-track              drop track with abnormal segment duration. Only works with --cbm.
  -c, --content-id string                   explicit content-id (instead of ingest path).
  -m, --maxtime int                         maximal execution time (seconds) to ingest asset. Setting to 0 means no limit.
      --hls                                 output HLS playlists (experimental).
      --slim                                only output content_info and dat files.
      --leavepartial                        leave partial ingested content or collect erroneous input files (if any) on recording failure. The input of failure recording will be treated as some kind of log and stored next to the log file.
      --s3file string                       path to S3 credentials file.
  -t, --template string                     path to content template file, for asset comptibility verification.
      --licensefile string                  license file path (default "/etc/edgeware/ew-vodingest/license.json").
  -l, --logfile string                      log file [default stdout]. In case of recording failure, the relevant files are collected to same folder as log file.
      --log-file string                     see above.
      --logformat string                    format and type of log: [consolejson consolepretty discard] (default "consolepretty").
      --loglevel string                     initial log level (default "info").
  -v, --version                             print version and date.
      --host string                         cbm host to connect to. Only works with --cbm (default "localhost").
      --port uint16                         cbm port to connect to. Only works with --cbm (default 8090).
      --source-rate-limit uint32            maximum read rate in Mbps from source. Only works with --cbm. Setting 0 will disable source rate limiting.
      --local-storage string                Destination for storing input temporarily when ingesting from HTTP(s), S3 or FTP. Also store output temporarily when uploading to S3 or FTP. If empty, use '/tmp'. When being used with --cbm, it is used for storing the recorded content from CBM before uploading.
      --thumbnails                          enable thumbnail generating, requires command 'ffmpeg' to work.
      --thumbnails-sizes string             comma separated list of thumbnail sizes width[xheight],... If only widths are provided, the height will be calculated to keep aspect ratio, nothing provided, auto generate a thumbnails with 240 as width.
      --thumbnails-sprite-max-size uint32   max pixel dimension (width or height) for thumbnail sprite sheet, range [1024, 16300]. Adjust based on your device software or hardware limitations (default 16300).
      --preserve-ttml-styling               preserve TTML styling when converting to wvtt. Only works with --input and is ignored for --cbm.
      --help                                extended tool help.

The following options -i, -o, -w, -y, --cbm, --start-epoch, --stop-epoch, --channel, -l, --source-rate-limit are the same as for the ESB3005 ew-recorder.

To enable image-based thumbnails generation (currently only JPEG is supported), add flag --thumbnails. About --thumbnails-sizes, it acts like --sizes in ew-thumbnails-generator tool, please refer to Tools for more information. These options are available from version 1.12.0.