Advanced features

• 1: Content popularity
• 2: Consistent Hashing
• 3: Security token verification
• 4: Subnets API
• 5: Lua Features
• 5.1: Built-in Lua Functions
• 5.2: Global Lua Tables
• 5.3: Request Translation Function
• 5.4: Session Translation Function
• 5.5: Host Request Translation Function
• 5.6: Response Translation Function

1 - Content popularity

ESB3024 Router allows routing decisions based on content popularity. All incoming content requests are tracked to continuously update a content popularity ranking list. The popularity ranking algorithm is designed to let popular content quickly rise to the top while unpopular content decays and sinks towards the bottom.

Routing

A content popularity based routing rule can be created by running

$ confcli services.routing.rules -w
Running wizard for resource 'rules'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

rules : [
  rule can be one of
    1: allow
    2: consistentHashing
    3: contentPopularity
    4: deny
    5: firstMatch
    6: random
    7: rawGroup
    8: rawHost
    9: split
    10: weighted
  Choose element index or name: contentPopularity
  Adding a 'contentPopularity' element
    rule : {
      name (default: ): content_popularity_rule
      type (default: contentPopularity):
      contentPopularityCutoff (default: 10): 5
      onPopular (default: ): edge-streamer
      onUnpopular (default: ): offload
    }
  Add another 'rule' element to array 'rules'? [y/N]: n
]
Generated config:
{
  "rules": [
    {
      "name": "content_popularity_rule",
      "type": "contentPopularity",
      "contentPopularityCutoff": 5.0,
      "onPopular": "edge-streamer",
      "onUnpopular": "offload"
    }
  ]
}
Merge and apply the config? [y/n]: y

This rule will route requests for top 5 most popular content to edge-streamer and all other requests to offload.

Some configuration settings attributed to content popularity are available:

$ confcli services.routing.settings.contentPopularity
{
    "contentPopularity": {
        "enabled": true,
        "algorithm": "score_based",
        "sessionGroupNames": []
    }
}

• enabled: Whether or not to track content popularity. When enabled is set to false, content popularity will not be tracked. Note that routing on content popularity is possible even if enabled is false and content popularity has been tracked previously.
• algorithm: Choice of content popularity tracking algorithm. There are two possible choices: score_based or time_based (detailed below).
• sessionGroupNames: Names of the session groups for which content popularity should be tracked. Note that content popularity is tracked globally, not per session group.

Algorithm tuning

The behaviour of each content popularity tracking algorithm can be tuned using the raw JSON API.

All configuration parameters for content popularity reside in the settings object of the configuration, an example of which can be seen below:

{
  "settings": {
    "content_popularity": {
      "algorithm": "scored_based",
      "session_group_names": ["vod_only"],
      "score_based:": {
        "requests_between_popularity_decay": 1000,
        "popularity_list_max_size": 100000,
        "popularity_prediction_factor": 2.5,
        "popularity_decay_fraction": 0.2
      },
      "time_based": {
        "intervals_per_hour": 10
      }
    }
  }
}

The field algorithm dictates which content popularity tracking algorithm to use, can either be score_based or time_based.

The field session_group_names defines the sessions for which content popularity should be tracked. In the example above, session belonging to the vod_only session group will be tracked for content popularity. If left empty, content popularity will be tracked for all sessions.

The remaining configuration parameters are algorithm specific.

Score based algorithm

The field popularity_list_max_size defines the maximum amount of unique contents to track for popularity. This can be used to limit memory growth. A single entry in the popularity ranking list will at most consume 180 bytes of memory. E.g. using "popularity_list_max_size": 1000 would consume at most 180⋅1,000 = 180,000 B = 0.18 MB. If the content popularity list is full, a request to unique content would replace the least popular content.

Setting a very high max size will not impact performance, it will only consume more memory.

The field requests_between_popularity_decay defines the number of requests between each popularity decay update, an integral component of this feature.

The fields popularity_prediction_factor and popularity_decay_fraction tune the behaviour of the content popularity ranking algorithm, explained further below.

Decay update

To allow for popular content to quickly rise in popularity and unpopular content to sink, a dynamic popularity ranking algorithm is used. The goal of the algorithm is to track content popularity in real time, allowing routing decisions based on the requested content’s popularity. The algorithm is applied every decay update.

The algorithm uses current trending content to predict content popularity. The field popularity_prediction_factor regulates how much the algorithm should rely on predicted popularity. A high prediction factor allows rising content to quickly rise to high popularity but can also cause unpopular content with a sudden burst of requests to wrongfully rise to the top. A low prediction factor can cause stagnation in the popularity ranking, not allowing new popular content to rise to the top.

Unpopular content decays in popularity, the magnitude of which is regulated by popularity_decay_fraction. A high value will aggressively decay content popularity every decay update while a low value will bloat the ranking, causing stagnation. Once content decays to a trivially low popularity score, it is pruned from the content popularity list.

When configuring these tuning parameters, the most crucial data to consider is the size of your asset catalog, i.e. the number of unique contents you offer. The recommended values, obtained through testing, are presented in the table below. Note that the field popularity_prediction_factor is the principal factor in controlling the algorithm’s behaviour.

Time based algorithm

The time based algorithm only requires the configuration parameter intervals_per_hour. E.g., the value "intervals_per_hour": 10 would give 10 six minute intervals per hour. During each interval, all unique content requests has an associated counter, increasing by one for each incoming request. After an hour, all intervals have been cycled through. The counters in the first interval will be reset and all incoming content requests will increase the counters in the first interval again. This cycle continues forever.

When determining a single content’s popularity, the sum of each content’s counter in all intervals is used to determine a popularity ranking.

2 - Consistent Hashing

Consistent hashing based routing is a feature that can be used to distribute requests to a set of hosts in a cache friendly manner. By using Agile Content’s consistent distributed hash algorithm, the amount of cache redistribution is minimized within a set of hosts. Requests for a content will always be routed to the same set of hosts, the amount of which is configured by the spread factor, allowing high cache usage. When adding or removing hosts, the algorithm minimizes cache redistribution.

Say you have the host group [s1, s2, s3, s4, s5] and have configured spreadFactor = 3. A request for a content asset1 would then be routed to the same three hosts with one of them being selected randomly for each request. Requests for a different content asset2 would also be routed to one of three different hosts, most likely a different combination of hosts than requests for content asset1.

Example routing results with spreadFactor = 3:

• Request for asset1 → route to one of [s1, s3, s4].
• Request for asset2 → route to one of [s2, s4, s5].
• Request for asset3 → route to one of [s1, s2, s5].

Since consistent hashing based routing ensures that requests for a specific content always get routed to the same set of hosts, the risk of cache misses are lowered on the hosts since they will be served the same content requests over and over again.

Note that the maximum value of spreadFactor is 64. Consequently, the highest amount of hosts you can use in a consistentHashing rule block is 64.

Three different hashing algorithms are available: MD5, SDBM and Murmur. The algorithm is chosen during configuration.

Configuration

Configuring consistent hashing based routing is easily done using confcli. Let’s configure the example described above:

confcli services.routing.rules -w
Running wizard for resource 'rules'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

rules : [
  rule can be one of
    1: allow
    2: consistentHashing
    3: contentPopularity
    4: deny
    5: firstMatch
    6: random
    7: rawGroup
    8: rawHost
    9: split
    10: weighted
  Choose element index or name: consistentHashing
  Adding a 'consistentHashing' element
    rule : {
      name (default: ): consistentHashingRule 
      type (default: consistentHashing): 
      spreadFactor (default: 1): 3
      hashAlgorithm (default: MD5):
      targets : [
        target : {
          target (default: ): s1
          enabled (default: True): 
        }
        Add another 'target' element to array 'targets'? [y/N]: y
        target : {
          target (default: ): s2
          enabled (default: True): 
        }
        Add another 'target' element to array 'targets'? [y/N]: y
        target : {
          target (default: ): s3
          enabled (default: True): 
        }
        Add another 'target' element to array 'targets'? [y/N]: y
        target : {
          target (default: ): s4
          enabled (default: True): 
        }
        Add another 'target' element to array 'targets'? [y/N]: y
        target : {
          target (default: ): s5
          enabled (default: True): 
        }
        Add another 'target' element to array 'targets'? [y/N]: n
      ]
    }
  Add another 'rule' element to array 'rules'? [y/N]: n
]
Generated config:
{
  "rules": [
    {
      "name": "consistentHashingRule",
      "type": "consistentHashing",
      "spreadFactor": 3,
      "hashAlgorithm": "MD5",
      "targets": [
        {
          "target": "s1",
          "enabled": true
        },
        {
          "target": "s2",
          "enabled": true
        },
        {
          "target": "s3",
          "enabled": true
        },
        {
          "target": "s4",
          "enabled": true
        },
        {
          "target": "s5",
          "enabled": true
        }
      ]
    }
  ]
}

Adding hosts

Adding a host to the list will give an additional target for the consistent hashing algorithm to route requests to. This will shift content distribution onto the new host.

confcli services.routing.rules.consistentHashingRule.targets -w
Running wizard for resource 'targets'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

targets : [
  target : {
    target (default: ): s6
    enabled (default: True): 
  }
  Add another 'target' element to array 'targets'? [y/N]: n
]
Generated config:
{
  "targets": [
    {
      "target": "s6",
      "enabled": true
    }
  ]
}
Merge and apply the config? [y/n]: y

Removing hosts

There is one very important caveat of using a consistent hashing rule block. As long as you don’t modify the list of hosts, the consistent hashing algorithm will keep routing requests to the same hosts. However, if you remove a host from the block in any position except the last, the consistent hashing algorithm’s behaviour will change and the algorithm cannot maintain a minimum amount of cache redistribution.

If you’re in a situation where you have to remove a host from the routing targets but want to keep the same consistent hashing behaviour, e.g. during very high load, you’ll have to toggle that target’s enabled field to false. E.g., disabling requests to s2 can be accomplished by:

$ confcli services.routing.rules.consistentHashingRule.targets.1.enabled false
services.routing.rules.consistentHashingRule.targets.1.enabled = False
$ confcli services.routing.rules.consistentHashingRule.targets.1
{
    "1": {
        "target": "s2",
        "enabled": false
    }
}

If you modify the list order or remove hosts, it is highly recommended to do so during moments where a higher rate of cache misses are acceptable.

3 - Security token verification

The security token verification feature allows for ESB3024 Router to only process requests that contain a correct security token. The token is generated by the client, for example in the portal, using an algorithm that it shares with the router. The router verifies the token and rejects the request if the token is incorrect.

It is beyond the scope of this document to describe how the token is generated, that is described in the Security Tokens application note that is installed with the ESB3024 Router’s extra documentation.

Setting up a routing rule

The token verification is performed by calling the verify_security_token() function from a routing rule. The function returns 1 if the token is correct, otherwise it returns 0. It should typically be called from the first routing rule, to make requests with bad tokens fail as early as possible.

The confcli example assumes that the router already has rules configured, with an entry point named select_cdn. Token verification is enabled by inserting an “allow” rule first in the rule list.

• confcli
• JSON

confcli services.routing.rules -w
Running wizard for resource 'rules'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

rules : [
  rule can be one of
    1: allow
    2: consistentHashing
    3: contentPopularity
    4: deny
    5: firstMatch
    6: random
    7: rawGroup
    8: rawHost
    9: split
    10: weighted
  Choose element index or name: allow
  Adding a 'allow' element
    rule : {
      name (default: ): token_verification
      type (default: allow):
      condition (default: always()): verify_security_token()
      onMatch (default: ): select_cdn
    }
  Add another 'rule' element to array 'rules'? [y/N]: n
]
Generated config:
{
  "rules": [
    {
      "name": "token_verification",
      "type": "allow",
      "condition": "verify_security_token()",
      "onMatch": "select_cdn"
    }
  ]
}
Merge and apply the config? [y/n]: y

$ confcli services.routing.entrypoint token_verification
services.routing.entrypoint = 'token_verification'

"routing": {
  "id": "token_verification",
  "member_order": "sequential",
  "members": [
    {
      "id": "token_verification.0.select_cdn",
      "member_order": "weighted",
      "members": [
        ...
      ],
      "weight_function": "return verify_security_token() ~= 0"
    },
    {
      "id": "token_verification.1.rejected",
      "member_order": "sequential",
      "members": [],
      "weight_function": "return 1"
    }
  ],
  "weight_function": "return 100"
},

Configuring security token options

The secret parameter is not part of the router request, but needs to be configured separately in the router. That can be done with the host-config tool that is installed with the router.

Besides configuring the secret, host-config can also configure floating sessions and a URL prefix. Floating sessions are sessions that are not tied to a specific IP address. When that is enabled, the token verification will not take the IP address into account when verifying the token.

The security token verification is configured per host, where a host is the name of the host that the request was sent to. This makes it possible for a router to support multiple customer accounts, each with their own secret. If no configuration is found for a host, a configuration with the name default is used.

host-config supports three commands: print, set and delete.

Print

The print command prints the current configuration for a host. The following parameters are supported:

host-config print [-n <host-name>]

By default it prints the configuration for all hosts, but if the optional -n flag is given it will print the configuration for a single host.

Set

The set command sets the configuration for a host. The configuration is given as command line parameters. The following parameters are supported:

host-config set
    -n <host-name>
    [-f floating]
    [-p url-prefix]
    [-r <secret-to-remove>]
    [-s <secret-to-add>]

• -n <host-name> - The name of the host to configure.
• -f floating - A boolean option that specifies if floating sessions are accepted. The parameter accepts the values true and false.
• -p url-prefix - A URL prefix that is used for identifying requests that come from a certain account. This is not used when verifying tokens.
• -r <secret-to-remove> - A secret that should be removed from the list of secrets.
• -s <secret-to-add> - A secret that should be added to the list of secrets.

For example, to set the secret “secret-1” and enable floating sessions for the default host, the following command can be used:

host-config set -n default -s secret-1 -f true

The set command only touches the configuration options that are mentioned on the command line, so the following command line will add a second secret to the default host without changing the floating session setting:

host-config set -n default -s secret-2

It is possible to set multiple secrets per host. This is useful when updating a secret, then both the old and the new secret can be valid during the transition period. After the transition period the old secret can be removed by typing:

host-config set -n default -r secret-1

Delete

The delete command deletes the configuration for a host. It supports the following parameters:

host-config delete -n <host-name>

For example, to delete the configuration for example.com, the following command can be used:

host-config delete -n example.com

Global options

host-config also has a few global options. They are:

• -k <security-key> - The security key that is used when communicating with the router. This is normally retrieved automatically.
• -h - Print a help message and exit.
• -r <router> - The router to connect to. This default to localhost, but can be changed to connect to a remote router.
• -v - Verbose output, can be given multiple times.

Debugging security token verification

The security token verification only logs messages when the log level is set to 4 or higher. Then it will only log some errors. It is possible to enable more verbose logging using the security-token-config that is installed together with the router.

When verbose logging is enabled, the router will log information about the token verification, including the configured token secrets, so it needs to be used with care.

The logged lines are prefixed with verify_security_token.

The security-token-config tool supports the commands print and set.

Print

The print command prints the current configuration. If nothing is configured it will not print anything.

Set

The set command sets the configuration. The following parameters are supported:

security-token-config set
    [-d <enabled>]

• -d <enabled> - A boolean option that specifies if debug logging should be enabled or not. The parameter accepts the values true and false.

4 - Subnets API

ESB3024 Router provides utilities to quickly match clients into subnets. Any combination of IPv4 and IPv6 addresses can be used. To begin, a JSON file is needed, defining all subnets, e.g:

{
  "255.255.255.255/24": "area1",
  "255.255.255.255/16": "area2",
  "255.255.255.255/8": "area3",
  "90.90.1.3/16": "area4",
  "5.5.0.4/8": "area5",
  "2a02:2e02:9bc0::/48": "area6",
  "2a02:2e02:9bc0::/32": "area7",
  "2a02:2e02:9bc0::/16": "area8",
  "2a02:2e02:9de0::/44": "combined_area",
  "2a02:2e02:ada0::/44": "combined_area"
}

and PUT it to the endpoint :5001/v1/subnets or :5001/v2/subnets, the API version doesn’t matter for subnets:

curl -k -T subnets.json -H "Content-Type: application/json" https://router-host:5001/v1/subnets

Note that it is possible for several subnet CIDR strings to share the same label, effectively grouping them together.

The router provides the built-in function in_subnet(subnet_name) that can to make routing decisions based on a client’s subnet. For more details, see Built-in Lua functions. To configure a rule that only allows clients in the area1 subnet, run the command

$ confcli services.routing.rules -w
Running wizard for resource 'rules'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

rules : [
  rule can be one of
    1: allow
    2: consistentHashing
    3: contentPopularity
    4: deny
    5: firstMatch
    6: random
    7: rawGroup
    8: rawHost
    9: split
    10: weighted
  Choose element index or name: allow
  Adding a 'allow' element
    rule : {
      name (default: ): only_allow_area1
      type (default: allow):
      condition (default: always()): in_subnet('area1')
      onMatch (default: ): example-host
    }
  Add another 'rule' element to array 'rules'? [y/N]: n
]
Generated config:
{
  "rules": [
    {
      "name": "only_allow_area1",
      "type": "allow",
      "condition": "in_subnet('area1')",
      "onMatch": "example-host"
    }
  ]
}
Merge and apply the config? [y/n]: y

Invalid IP-addresses will be omitted during subnet list construction accompanied by a message in the log displaying the invalid IP address.

5 - Lua Features

Detailed descriptions and examples of Lua features offered by ESB3024 Router.

5.1 - Built-in Lua Functions

This section details all built-in Lua functions provided by the router.

Logging functions

The router provides Lua logging functionality that is convenient when creating custom Lua functions. A prefix can be added to the log message which is useful to differentiate log messages from different lua files. At the top of the Lua source file, add the line

local log = log.add_prefix("my_lua_file")

to prepend all log messages with "my_lua_file".

The logging functions support formatting and common log levels:

log.critical('A log message with number %d', 1.5)
log.error('A log message with string %s', 'a string')
log.warning('A log message with integer %i', 1)
log.info('A log message with a local number variable %d', some_local_number)
log.debug('A log message with a local string variable %s', some_local_string)
log.trace('A log message with a local integer variable %i', some_local_integer)
log.message('A log message')

Many of the router’s built-in Lua functions use the logging functions.

Predictive load balancing functions

Predictive load balancing is a tool that can be used to avoid overloading hosts with traffic. Consider the case where a popular event starts at a certain time, let’s say 12 PM. A spike in traffic will be routed to the hosts that are streaming the content at 12 PM, most of them starting at low bitrates. A host might have sufficient bandwidth left to take on more clients but when the recently connected clients start ramping up in video quality and increase their bitrate, the host can quickly become overloaded, possibly dropping incoming requests or going offline. Predictive load balancing solves this issue by considering how many times a host recently been redirected to.

Four functions for predictive load balancing are provided by the router that can be used when constructing conditions/weight functions: host_bitrate() , host_bitrate_custom(), host_has_bw() and host_has_bw_custom(). All require data to be supplied to the selection input API and apply only to leaf nodes in the routing tree. In order for predictive load balancing to work properly the data must be updated at regular intervals. The data needs to be supplied by the target system.

These functions are suitable to used as host health checks. To configure host health checks, see configuring CDNs and hosts.

Note that host_bitrate() and host_has_bw() rely on data supplied by metrics agents, detailed in Cache hardware metrics: monitoring and routing.

host_bitrate_custom() and host_has_bw_custom() rely on manually supplied selection input data, detailed in selection input API. The bitrate unit depends on the data submitted to the selection input API.

Example metrics

The data supplied to the selection input API by the metrics agents uses the following structure:

{
  "streamer-1": {
    "hardware_metrics": {
      "/": {
        "free": 1741596278784,
        "total": 1758357934080,
        "used": 16761655296,
        "used_percent": 0.9532561585516977
      },
      "cpu_load1": 0.02,
      "cpu_load15": 0.12,
      "cpu_load5": 0.02,
      "mem_available": 4895789056,
      "mem_available_percent": 59.551760354263074,
      "mem_total": 8221065216,
      "mem_used": 2474393600,
      "n_cpus": 4
    },
    "per_interface_metrics": {
      "eths1": {
        "link": 1,
        "interface_up": true,
        "megabits_sent": 22322295739.378456,
        "megabits_sent_rate": 8085.2523952,
        "speed": 100000
      }
    }
  }
}

Note that all built-in functions interacting with selection input values support indexing into nested selection input data. Consider the selection input data in above. The nested values can be accessed by using dots between the keys:

si('streamer-1.per_interface_metrics.eths1.megabits_sent_rate')

Note that the whole selection input variable name must be within single quotes. The function si() is documented under general purpose functions.

host_bitrate({})

host_bitrate() returns the predicted bitrate (in megabits per second) of the host after the recently connected clients start ramping up in streaming quality. The function accepts an argument table with the following keys:

• interface: The name of the interface to use for bitrate prediction.
• Optional avg_bitrate: the average bitrate per client, defaults to 6 megabits per second.
• Optional num_routers: the number of routers that can route to this host, defaults to 1. This is important to accurately predict the incoming load if multiple routers are used.
• Optional host: The name of the host to use for bitrate prediction. Defaults to the current host if not provided.

Required selection input data

This function relies on the field megabits_sent_rate, supplied by the Telegraf metrics agent, as seen in example metrics. If these fields are missing from your selection input data, this function will not work.

Examples of usage:

host_bitrate({interface='eths0'})
host_bitrate({avg_bitrate=1, interface='eths0'})
host_bitrate({num_routers=2, interface='eths0'})
host_bitrate({avg_bitrate=1, num_routers=4, interface='eths0'})
host_bitrate({avg_bitrate=1, num_routers=4, host='custom_host', interface='eths0'})

host_bitrate({}) calculates the predicted bitrate as:

predicted_host_bitrate = current_host_bitrate + (recent_connections * avg_bitrate * num_routers)

host_bitrate_custom({})

Same functionality as host_bitrate() but uses a custom selection input variable as bitrate input instead of accessing hardware metrics. The function accepts an argument table with the following keys:

• custom_bitrate_var: The name of the selection input variable to be used for accessing current host bitrate.
• Optional avg_bitrate: see host_bitrate() documentation above.
• Optional num_routers: see host_bitrate() documentation above.

host_bitrate_custom({custom_bitrate_var='host1_current_bitrate'})
host_bitrate_custom({avg_bitrate=1, custom_bitrate_var='host1_current_bitrate'})
host_bitrate_custom({num_routers=4, custom_bitrate_var='host1_current_bitrate'})

host_has_bw({})

Instead of accessing the predicted bitrate of a host through host_bitrate(), host_has_bw() returns 1 if the host is predicted to have enough bandwidth left to take on more clients after recent connections ramp up in bitrate, otherwise it returns 0. The function accepts an argument table with the following keys:

• interface: see host_bitrate() documentation above.
• Optional avg_bitrate: see host_bitrate() documentation above.
• Optional num_routers: see host_bitrate() documentation above.
• Optional host: see host_bitrate() documentation above.
• Optional margin: the bitrate (megabits per second) headroom that should be taken into account during calculation, defaults to 0.

host_has_bw({}) returns whether or not the following statement is true:

predicted_host_bitrate + margin < host_bitrate_capacity

Required selection input data

host_has_bw({}) relies on the fields megabits_sent_rate and speed, supplied by the Telegraf metrics agent, as seen in example metrics. If these fields are missing from your selection input data, this function will not work.

Examples of usage:

host_has_bw({interface='eths0'})
host_has_bw({margin=10, interface='eth0'})
host_has_bw({avg_bitrate=1, interface='eth0'})
host_has_bw({num_routers=4, interface='eth0'})
host_has_bw({host='custom_host', interface='eth0'})

host_has_bw_custom({})

Same functionality as host_has_bw() but uses a custom selection input variable as bitrate. It also uses a number or a custom selection input variable for the capacity. The function accepts an argument table with the following keys:

• custom_capacity_var: a number representing the capacity of the network interface OR the name of the selection input variable to be used for accessing host capacity.
• custom_bitrate_var: see host_bitrate_custom() documentation
• Optional margin: see host_has_bw() documentation above. above.
• Optional avg_bitrate: see host_bitrate() documentation above.
• Optional num_routers: see host_bitrate() documentation above.

Examples of usage:

host_has_bw_custom({custom_capacity_var=10000, custom_bitrate_var='streamer-1.per_interface_metrics.eths1.megabits_sent_rate'})
host_has_bw_custom({custom_capacity_var='host1_capacity', custom_bitrate_var='streamer-1.per_interface_metrics.eths1.megabits_sent_rate'})
host_has_bw_custom({margin=10, custom_capacity_var=10000, custom_bitrate_var='streamer-1.per_interface_metrics.eths1.megabits_sent_rate'})
host_has_bw_custom({avg_bitrate=1, custom_capacity_var=10000, custom_bitrate_var='streamer-1.per_interface_metrics.eths1.megabits_sent_rate'})
host_has_bw_custom({num_routers=4, custom_capacity_var=10000, custom_bitrate_var='streamer-1.per_interface_metrics.eths1.megabits_sent_rate'})

Health check functions

This section details built-in Lua functions that are meant to be used for host health checks. Note that these functions rely on data supplied by metric agents detailed in Cache hardware metrics: monitoring and routing. Make sure cache hardware metrics are supplied to the router before using any of these functions.

cpu_load_ok({})

The function accepts an optional argument table with the following keys:

• Optional host: The name of the host. Defaults to the name of the selected host if not provided.
• Optional cpu_load5_limit: The acceptable limit for the 5-minute CPU load. Defaults to 0.9 if not provided.

The function returns 1 if the five minute CPU load average is below their respective limits, and 0 otherwise.

Examples of usage:

cpu_load_ok()
cpu_load_ok({host = 'custom_host'})
cpu_load_ok({cpu_load5_limit = 0.8})
cpu_load_ok({host = 'custom_host', cpu_load5_limit = 0.8})

memory_usage_ok({})

The function accepts an optional argument table with the following keys:

• Optional host: The name of the host. Defaults to the host of the selected host if not provided.
• Optional memory_usage_limit: The acceptable limit for the memory usage. Defaults to 0.9 if not provided.

The function returns 1 if the memory usage is below the limit, and 0 otherwise.

Examples of usage:

memory_usage_ok()
memory_usage_ok({host = 'custom_host'})
memory_usage_ok({memory_usage_limit = 0.7})
memory_usage_ok({host = 'custom_host', memory_usage_limit = 0.7})

interfaces_online({})

The function accepts an argument table with the following keys:

• Required interfaces: A string or a table of strings representing the network interfaces to check.
• Optional host: The name of the host. Defaults to the host of the selected host if not provided.

The function returns 1 if all the specified interfaces are online, and 0 otherwise.

Required selection input data

This function relies on the fields link and interface_up, supplied by the Telegraf metrics agent, as seen in example metrics. If these fields are missing from your selection input data, this function will not work.

Examples of usage:

interfaces_online({interfaces = 'eth0'})
interfaces_online({interfaces = {'eth0', 'eth1'}})
interfaces_online({host = 'custom_host', interfaces = 'eth0'})
interfaces_online({host = 'custom_host', interfaces = {'eth0', 'eth1'}})

health_check({})

The function accepts an optional argument table with the following keys:

• Required interfaces: A string or a table of strings representing the network interfaces to check.
• Optional host: The name of the host. Defaults to the host of the selected host if not provided.
• Optional cpu_load5_limit: The acceptable limit for the 5-minute CPU load. Defaults to 0.9 if not provided.
• Optional memory_usage_limit: The acceptable limit for the memory usage. Defaults to 0.9 if not provided.

The function calls the health check functions cpu_load_ok({}), memory_usage_ok({}) and interfaces_online({}). The functions returns 1 if all these functions returned 1, otherwise it returns 0.

Examples of usage:

health_check({interfaces = 'eths0'})
health_check({host = 'custom_host', interfaces = 'eths0'})
health_check({cpu_load5_limit = 0.7, memory_usage_limit = 0.8, interfaces = 'eth0'})
health_check({host = 'custom_host', cpu_load5_limit = 0.7, memory_usage_limit = 0.8, interfaces = {'eth0', 'eth1'}})

General purpose functions

The router supplies a number of general purpose Lua functions.

always()

Always returns 1.

never()

Always returns 0. Useful for temporarily disabling caches by using it as a health check.

Examples of usage:

always()
never()

si(si_name)

The function reads the value of the selection input variable si_name and returns it if it exists, otherwise it returns 0. The function accepts a string argument for the selection input variable name.

Examples of usage:

si('some_selection_input_variable_name')
si('streamer-1.per_interface_metrics.eths1.megabits_sent_rate')

Comparison functions

All comparison functions use the form function(si_name, value) and compares the selection input value with the name si_name with value.

ge(si_name, value) - greater than or equal

gt(si_name, value) - greater than

le(si_name, value) - less than or equal

lt(si_name, value) - less than

eq(si_name, value) - equal to

neq(si_name, value) - not equal to

Examples of usage:

ge('streamer-1.hardware_metrics.mem_available_percent', 30)
gt('streamer-1.hardware_metrics./.free', 1000000000)
le('streamer-1.hardware_metrics.cpu_load5', 0.8)
lt('streamer-1.per_interface_metrics.eths1.megabits_sent_rate', 9000)
eq('streamer-1.per_interface_metrics.eths1.link.', 1)
neq('streamer-1.hardware_metrics.n_cpus', 4)

Session checking functions

in_subnet(subnet)

Returns 1 if the current session belongs to subnet, otherwise it returns 0. See Subnets API for more details on how to use subnets in routing. The function accepts a string argument for the subnet name.

Examples of usage:

in_subnet('stockholm')
in_subnet('unserviced_region')
in_subnet('some_other_subnet')

These functions checks the current session’s session groups.

in_session_group(session_group)

Returns 1 if the current session has been classified into session_group, otherwise it returns 0. The function accepts a string argument for the session group name.

in_any_session_group({})

Returns 1 if the current session has been classified into any of session_groups, otherwise it returns 0. The function accepts a table array of strings as argument for the session group names.

in_all_session_groups({})

Returns 1 if the current session has been classified into all of session_groups, otherwise it returns 0. The function accepts a table array of strings as argument for the session group names.

Examples of usage:

in_session_group('safari_browser')
in_any_session_group({ 'in_europe', 'in_asia'})
in_all_session_group({ 'vod_content', 'in_america'})

Other built-in functions

base64_encode(data)

base64_encode(data) returns the base64 encoded string of data.

Arguments:

• data: The data to encode.

Example:

print(base64_encode('Hello world!'))
SGVsbG8gd29ybGQh

base64_decode(data)

base64_decode(data) returns the decoded data of the base64 encoded string, as a raw binary string.

Arguments:

• data: The data to decode.

Example:

print(base64_decode('SGVsbG8gd29ybGQh'))
Hello world!

base64_url_encode(data)

base64_url_encode(data) returns the base64 URL encoded string of data.

Arguments:

• data: The data to encode.

Example:

print(base64_url_encode('ab~~'))
YWJ-fg

base64_url_decode(data)

base64_url_decode(data) returns the decoded data of the base64 URL encoded string, as a raw binary string.

Arguments:

• data: The data to decode.

Example:

print(base64_url_decode('YWJ-fg'))
ab~~

to_hex_string(data)

to_hex_string(data) returns a string containing the hexadecimal representation of the string data.

Arguments:

• data: The data to convert.

Example:

print(to_hex_string('Hello world!\n'))
48656c6c6f20776f726c64210a

from_hex_string(data)

from_hex_string(data) returns a string containing the byte representation of the hexadecimal string data.

Arguments:

• data: The data to convert.

Example:

print(from_hex_string('48656c6c6f20776f726c6421'))
Hello world!

empty(table)

empty(table) returns true if table is empty, otherwise it returns false.

Arguments:

• table: The table to check.

Examples:

print(tostring(empty({})))
true

print(tostring(empty({1, 2, 3})))
false

md5(data)

md5(data) returns the MD5 hash of data, as a hexstring.

Arguments:

• data: The data to hash.

Example:

print(md5('Hello world!'))
86fb269d190d2c85f6e0468ceca42a20

sha256(date)

sha256(data) returns the SHA-256 hash of data, as a hexstring.

Arguments:

• data: The data to hash.

Example:

print(sha256('Hello world!'))
c0535e4be2b79ffd93291305436bf889314e4a3faec05ecffcbb7df31ad9e51a

hmac_sha256(key, data)

hmac_sha256(key, data) returns the HMAC-SHA-256 hash of data using key, as a base64 encoded string.

Note: This function is to be modified to return raw binary data instead of a base64 encoded string.

Arguments:

• key: The key to use.
• data: The data to hash.

Example:

print(hmac_sha256('secret', 'Hello world!'))
pl9M/PX0If8r4FLgZCvMvP6xJu5z68T+OzgZZDAutjI=

hmac_sha384(key, data)

hmac_sha384(key, data) returns the HMAC-SHA-384 hash of data using key, as a string containing raw binary data.

Arguments:

• key: The key to use.
• data: The data to hash.

Example:

print(to_hex_string(hmac_sha384('secret', 'Hello world!')))
917516d93d3509a371a129ca50933195dd659712652f07ba5792cbd5cade5e6285a841808842cfa0c3c69c8fb234468a

hmac_sha512(key, data)

hmac_sha512(key, data) returns the HMAC-SHA-512 hash of data using key, as a string containing raw binary data.

Arguments:

• key: The key to use.
• data: The data to hash.

Example:

print(to_hex_string(hmac_sha512('secret', 'Hello world!')))
dff6c00943387f9039566bfee0994de698aa2005eecdbf12d109e17aff5bbb1b022347fbf4bd94ede7c7d51571022525556b64f9d5e4386de99d0025886eaaff

hmac_md5(key, data)

hmac_md5(key, data) returns the HMAC-MD5 hash of data using key, as a string containing raw binary data.

Arguments:

• key: The key to use.
• data: The data to hash.

Example:

print(to_hex_string(hmac_md5('secret', 'Hello world!')))
444fad0d374d14369d6b595062da5d91

regex_replace

regex_replace(data, pattern, replacement) returns the string data with all occurrences of the regular expression pattern replaced with replacement.

Arguments:

• data: The data to replace.
• pattern: The regular expression pattern to match.
• replacement: The replacement string.

Examples:

print(regex_replace('Hello world!', 'world', 'Lua'))
Hello Lua!

print(regex_replace('Hello world!', 'l+', 'lua'))
Heluao worluad!

If the regular expression pattern is invalid, regex_replace() returns an error message.

Examples:

print(regex_replace('Hello world!', '*', 'lua'))
regex_error caught: regex_error

unixtime()

unixtime() returns the current Unix timestamp, as seconds since midnight, Janury 1 1970 UTC, as an integer.

Arguments:

• None

Example:

print(unixtime())
1733517373

now()

now() returns the current Unix timestamp, the number of seconds since midnight, Janury 1 1970 UTC, as an number with decimals.

Arguments:

• None

Example:

print(now())
1733517373.5007

timeToEpoch(time, fmt)

timeToEpoch(time, fmt) returns the Unix timestamp, the number of seconds since midnight, Janury 1 1970 UTC, of the time string time, which is formatted according to the format string fmt.

Note: This function is scheduled to be renamed to time_to_epoch().

Arguments:

• time: The time string to convert.
• fmt (Optional): The format string of the time string, as specified by the POSIX function strptime(). If not specified, it defaults to “%Y-%m-%dT%TZ”.

Examples:

print(timeToEpoch('1972-04-17T06:10:20Z'))
72339020

print(timeToEpoch('17/04-72 06:20:30', '%d/%m-%y %H:%M:%S'))
72339630

epochToTime(time, format)

epochToTime(time, format) returns the time string of the Unix timestamp time, formatted according to format.

Note: This function is scheduled to be renamed to epoch_to_time().

Arguments:

• time: The Unix timestamp to convert, as a number.
• format (Optional): The format string of the time string, as specified by the POSIX function strftime(). If not specified, it defaults to “%Y-%m-%dT%TZ”.

Examples:

print(epochToTime(123456789))
1973-11-29T21:33:09Z

print(epochToTime(1234567890, '%d/%m-%y %H:%M:%S'))
13/02-09 23:31:30

get_consistent_hashing_weight(contentName, nodeIdsString, spreadFactor, hashAlgoritm, nodeId)

get_consistent_hashing_weight(contentName, nodeIdsString, spreadFactor, hashAlgoritm, nodeId) returns the priority that node nodeId has in the list of preferred nodes, determined using consistent hashing. The first spreadfactor:th nodes should have equal weights to randomize requests between them. Remaining nodes should have decrementally decreasing weights to honor node priority during failover.

Arguments:

• contentName: The name of the content to hash.
• nodeIdsString: A string containing the node IDs to hash, on the format ‘0,1,2,3’.
• spreadFactor: The number of nodes to spread the requests between.
• hashAlgorithm: Which hash algorithm to use. Supported algorithms are “MD5”, “SDBM” and “Murmur”. Default is “MD5”.
• nodeId: The ID of the node to calculate the weight for.

Examples:

print(get_consistent_hashing_weight('/vod/film1', '0,1,2,3,4,5', 3, 'MD5', 3))
6

print(get_consistent_hashing_weight('/vod/film2', '0,1,2,3,4,5', 3, 'MD5', 3))
4

print(get_consistent_hashing_weight('/vod/film2', '0,1,2', 2, 'Murmur', 1))
2

See Consistent Hashing for more information about consistent hashing.

expand_ipv6_address(address)

expand_ipv6_address(address) returns the fully expanded form of the IPv6 address address.

Arguments:

• address: The IPv6 address to expand. If the address is not a valid IPv6 address, the function returns the contents of address unmodified. This allows for the function to pass through IPv4 addresses.

Examples:

print(expand_ipv6_address('2001:db8::1'))
2001:0db8:0000:0000:0000:0000:0000:0001

print(expand_ipv6_address('198.51.100.5'))
198.51.100.5

Configuration examples

Many of the functions documented are suitable to use in host health checks. To configure host health checks, see configuring CDNs and hosts. Here are some configuration examples of using the built-in Lua functions, utilizing the example metrics:

"healthChecks": [
    "gt('streamer-1.hardware_metrics.mem_available_percent', 20)", // More than 20% memory is left
    "lt('streamer-1.per_interface_metrics.eths1.megabits_sent_rate', 9000)" // Current bitrate is lower than 9000 Mbps
    "host_has_bw({host='streamer-1', interface='eths1', margin=1000})", // host_has_bw() uses 'streamer-1.per_interface_metrics.eths1.speed' to determine if there is enough bandwidth left with a 1000 Mbps margin
    "interfaces_online({host='streamer-1', interfaces='eths1'})",
    "memory_usage_ok({host='streamer-1'})",
    "cpu_load_ok({host='streamer-1'})",
    "health_check({host='streamer-1', interfaces='eths1'})" // Combines interfaces_online(), memory_usage_ok(), cpu_load_ok()
]

5.2 - Global Lua Tables

There are multiple global tables containing important data available while writing Lua code for the router.

selection_input

Contains arbitrary, custom fields fed into the router by clients, see API overview for details on how to inject data into this table.

Note that the selection_input table is iterable.

Usage examples:

print(selection_input['some_value'])

-- Iterate over table
if selection_input then
    for k, v in pairs(selection_input) do
        print('here is '..'selection_input!')
        print(k..'='..v)
    end
else
    print('selection_input is nil')
end

session_groups

Defines a mapping from session group name to boolean, indicating whether the session belongs to the session group or not.

Usage examples:

if session_groups.vod then print('vod') else print('not vod') end
if session_groups['vod'] then print('vod') else print('not vod') end

session_count

Provides counters of number of session types per session group. The table uses the structure qoe_score.<session_type>.<session_group>.

Usage examples:

print(session_count.instream.vod)
print(session_count.initial.vod)

qoe_score

Provides the quality of experience score per host per session group. The table uses the structure qoe_score.<host>.<session_group>.

Usage examples:

print(qoe_score.host1.vod)
print(qoe_score.host1.live)

request

Contains data related to the HTTP request between the client and the router.

• request.method
  • Description: HTTP request method.
  • Type: string
  • Example: 'GET', 'POST'
• request.body
  • Description: HTTP request body string.
  • Type: string or nil
  • Example: '{"foo": "bar"}'
• request.major_version
  • Description: Major HTTP version such as x in HTTP/x.1.
  • Type: integer
  • Example: 1
• request.minor_version
  • Description: Minor HTTP version such as x in HTTP/1.x.
  • Type: integer
  • Example: 1
• request.protocol
  • Description: Transfer protocol variant.
  • Type: string
  • Example: 'HTTP', 'HTTPS'
• request.client_ip
  • Description: IP address of the client issuing the request.
  • Type: string
  • Example: '172.16.238.128'
• request.path_with_query_params
  • Description: Full request path including query parameters.
  • Type: string
  • Example: '/mycontent/superman.m3u8?b=y&c=z&a=x'
• request.path
  • Description: Request path without query parameters.
  • Type: string
  • Example: '/mycontent/superman.m3u8'
• request.query_params
  • Description: The query parameter string.
  • Type: string
  • Example: 'b=y&c=z&a=x'
• request.filename
  • Description: The part of the path following the final slash, if any.
  • Type: string
  • Example: 'superman.m3u8'
• request.subnet
  • Description: Subnet of client_ip.
  • Type: string or nil
  • Example: 'all'

session

Contains data related to the current session.

• session.client_ip
  • Description: Alias for request.client_ip. See documentation for table request above.
• session.path_with_query_params
  • Description: Alias for request.path_with_query_params. See documentation for table request above.
• session.path
  • Description: Alias for request.path. See documentation for table request above.
• session.query_params
  • Description: Alias for request.query_params. See documentation for table request above.
• session.filename
  • Description: Alias for request.filename. See documentation for table request above.
• session.subnet
  • Description: Alias for request.subnet. See documentation for table request above.
• session.host
  • Description: ID of the currently selected host for the session.
  • Type: string or nil
  • Example: 'host1'
• session.id
  • Description: ID of the session.
  • Type: string
  • Example: '8eb2c1bdc106-17d2ff-00000000'
• session.session_type
  • Description: Type of the session.
  • Type: string
  • Example: 'initial' or 'instream'. Identical to the value of the Type argument of the session translation function.
• session.is_managed
  • Description: Identifies managed sessions.
  • Type: boolean
  • Example: true if Type/session.session_type is 'instream'

request_headers

Contains the headers from the request between the client and the router, keyed by name.

Usage example:

print(request_headers['User-Agent'])

request_query_params

Contains the query parameters from the request between the client and the router, keyed by name.

Usage example:

print(request_query_params.a)

session_query_params

Alias for metatable request_query_params.

response

Contains data related to the outgoing response apart from the headers.

• response.body
  • Description: HTTP response body string.
  • Type: string or nil
  • Example: '{"foo": "bar"}'
• response.code
  • Description: HTTP response status code.
  • Type: integer
  • Example: 200, 404
• response.text
  • Description: HTTP response status text.
  • Type: string
  • Example: 'OK', 'Not found'
• response.major_version
  • Description: Major HTTP version such as x in HTTP/x.1.
  • Type: integer
  • Example: 1
• response.minor_version
  • Description: Minor HTTP version such as x in HTTP/1.x.
  • Type: integer
  • Example: 1
• response.protocol
  • Description: Transfer protocol variant.
  • Type: string
  • Example: 'HTTP', 'HTTPS'

response_headers

Contains the response headers keyed by name.

Usage example:

print(response_headers['User-Agent'])

5.3 - Request Translation Function

Specifies the body of a Lua function that inspects every incoming HTTP request and overwrites individual fields before further processing by the router.

Returns nil when nothing is to be changed, or HTTPRequest(t) where t is a table with any of the following optional fields:

• Method
  • Description: Replaces the HTTP request method in the request being processed.
  • Type: string
  • Example: 'GET', 'POST'
• Path
  • Description: Replaces the request path in the request being processed.
  • Type: string
  • Example: '/mycontent/superman.m3u8'
• ClientIp
  • Description: Replaces client IP address in the request being processed.
  • Type: string
  • Example: '172.16.238.128'
• Body
  • Description: Replaces body in the request being processed.
  • Type: string or nil
  • Example: '{"foo": "bar"}'
• QueryParameters
  • Description: Adds, removes or replaces individual query parameters in the request being processed.
  • Type: nested table (indexed by number) representing an array of query parameters as {[1]='Name',[2]='Value'} pairs that are added to the request being processed, or overwriting existing query parameters with colliding names. To remove a query parameter from the request, specify nil as value, i.e. QueryParameters={..., {[1]='foo',[2]=nil} ...}. Returning a query parameter with a name but no value, such as a in the request '/index.m3u8?a&b=22' is currently not supported.
• Headers
  • Description: Adds, removes or replaces individual headers in the request being processed.
  • Type: nested table (indexed by number) representing an array of request headers as {[1]='Name',[2]='Value'} pairs that are added to the request being processed, or overwriting existing request headers with colliding names. To remove a header from the request, specify nil as value, i.e. Headers={..., {[1]='foo',[2]=nil} ...}. Duplicate names are supported. A multi-value header such as Foo: bar1,bar2 is defined by specifying Headers={..., {[1]='foo',[2]='bar1'}, {[1]='foo',[2]='bar2'}, ...}.

Example of a request_translation_function body that sets the request path to a hardcoded value and adds the hardcoded query parameter a=b:

-- Statements go here
print('Setting hardcoded Path and QueryParameters')
return HTTPRequest({
  Path = '/content.mpd',
  QueryParameters = {
    {'a','b'}
  }
})

Arguments

The following (iterable) arguments will be known by the function:

QueryParameters

• Type: nested table (indexed by number).

• Description: Array of query parameters as {[1]='Name',[2]='Value'} pairs that were present in the query string of the request. Format identical to the HTTPRequest.QueryParameters-field specified for the return value above.

• Example usage:

  for _, queryParam in pairs(QueryParameters) do
    print(queryParam[1]..'='..queryParam[2])
  end
  

Headers

• Type: nested table (indexed by number).

• Description: Array of request headers as {[1]='Name',[2]='Value'} pairs that were present in the request. Format identical to the HTTPRequest.Headers-field specified for the return value above. A multi-value header such as Foo: bar1,bar2 is seen in request_translation_function as Headers={..., {[1]='foo',[2]='bar1'}, {[1]='foo',[2]='bar1'}, ...}.

• Example usage:

  for _, header in pairs(Headers) do
    print(header[1]..'='..header[2])
  end
  

Additional data

In addition to the arguments above, the following Lua tables, documented in Global Lua Tables, provide additional data that is available when executing the request translation function:

• selection_input
• request
• request_query_params
• request_headers

If the request translation function modifies the request, the request, request_query_params and request_headers tables will be updated with the modified request and made available to the routing rules.

5.4 - Session Translation Function

Specifies the body of a Lua function that inspects a newly created session and may override its suggested type from “initial” to “instream” or vice versa. A number of helper functions are provided to simplify changing the session type.

Returns nil when the session type is to remain unchanged, or Session(t) where t is a table with a single field:

• Type
  • Description: New type of the session.
  • Type: string
  • Example: 'instream', 'initial'

Basic Configuration

It is possible to configure the maximum number of simultaneous managed sessions on the router. If the maximum number is reached, no more managed sessions can be created. Using confcli, it can be configured by running

$ confcli services.routing.tuning.general.maxActiveManagedSessions
{
    "maxActiveManagedSessions": 1000
}
$ confcli services.routing.tuning.general.maxActiveManagedSessions 900
services.routing.tuning.general.maxActiveManagedSessions = 900

Common Arguments

While executing the session translation function, the following arguments are available:

• Type: The current type of the session ('instream' or 'initial').

Usage examples:

-- Flip session type
local newType = 'initial'
if Type == 'initial' then
    newType = 'instream'
end
print('Changing session type from ' .. Type .. ' to ' .. newType)
return Session({['Type'] = newType})

Session Translation Helper Functions

The standard Lua library prodives four helper functions to simplify the configuration of the session translation function:

set_session_type(session_type)

This function will set the session type to the supplied session_type and the maximum number of sessions of that type has not been reached.

Parameters

• session_type: The type of session to create, possible values are ‘initial’ or ‘instream’.

Usage Examples

return set_session_type('instream')
return set_session_type('initial')

set_session_type_if_in_group(session_type, session_group)

This function will set the session type to the supplied session_type if the session is part of session_group and the maximum number of sessions of that type has not been reached.

Parameters

• session_type: The type of session to create, possible values are ‘initial’ or ‘instream’.
• session_group: The name of the session group.

Usage Examples

return set_session_type_if_in_group('instream', 'sg1')

set_session_type_if_in_all_groups(session_type, session_groups)

This function will set the session type to the supplied session_type if the session is part of all session groups given by session_groups and the maximum number of sessions of that type has not been reached.

Parameters

• session_type: The type of session to create, possible values are ‘initial’ or ‘instream’.
• session_groups: A list of session group names.

Usage Examples

return set_session_type_if_in_all_groups('instream', {'sg1', 'sg2'})

set_session_type_if_in_any_group(session_type)

This function will set the session type to the supplied session_type if the session is part of one or more of the session groups given by session_groups and the maximum number of sessions of that type has not been reached.

Parameters

• session_type: The type of session to create, possible values are ‘initial’ or ‘instream’.
• session_groups: A list of session group names.

Usage Examples

return set_session_type_if_in_any_group('instream', {'sg1', 'sg2'})

Configuration

Using confcli, example of how the functions above can be used in the session translation function can be configured by running any of

$ confcli services.routing.translationFunctions.session "return set_session_type('instream')"
services.routing.translationFunctions.session = "return set_session_type('instream')"

$ confcli services.routing.translationFunctions.session "return set_session_type_if_in_group('instream', 'sg1')"
services.routing.translationFunctions.session = "return set_session_type_if_in_group('instream', 'sg1')"

$ confcli services.routing.translationFunctions.session "return set_session_type_if_in_all_groups('instream', {'sg1', 'sg2'})"
services.routing.translationFunctions.session = "return set_session_type_if_in_all_groups('instream', {'sg1', 'sg2'})"

$ confcli services.routing.translationFunctions.session "return set_session_type_if_in_any_group('instream', {'sg1', 'sg2'})"
services.routing.translationFunctions.session = "return set_session_type_if_in_any_group('instream', {'sg1', 'sg2'})"

Additional data

In addition to the arguments above, the following Lua tables, documented in Global Lua Tables, provide additional data that is available when executing the response translation function:

• selection_input
• session_groups
• request
• request_query_params
• request_headers
• session
• session_query_params

The selection_input table will not change while a routing request is handled. A request_translation_function and the corresponding response_translation_function will see the same selection_input table, even if the selection data is updated while the request is being handled.

5.5 - Host Request Translation Function

The host request translation function defines a Lua function that modifies HTTP requests sent to a host. These hosts are configured in services.routing.hostGroups.

Hosts can receive requests for a manifest. A regular host will respond with the manifest itself, while a redirecting host and a DNS host will respond with a redirection to a streamer. This function can modify all these types of requests.

The function returns nil when nothing is to be changed, or HTTPRequest(t) where t is a table with any of the following optional fields:

• Method
  • Description: Replaces the HTTP request method in the request being processed.
  • Type: string
  • Example: 'GET', 'POST'
• Path
  • Description: Replaces the request path in the request being processed.
  • Type: string
  • Example: '/mycontent/superman.m3u8'
• Body
  • Description: Replaces body in the request being processed.
  • Type: string or nil
  • Example: '{"foo": "bar"}'
• QueryParameters
  • Description: Adds, removes or replaces individual query parameters in the request being processed.
  • Type: nested table (indexed by number) representing an array of query parameters as {[1]='Name',[2]='Value'} pairs that are added to the request being processed, or overwriting existing query parameters with colliding names. To remove a query parameter from the request, specify nil as value, i.e. QueryParameters={..., {[1]='foo',[2]=nil} ...}. Returning a query parameter with a name but no value, such as a in the request '/index.m3u8?a&b=22' is currently not supported.
• Headers
  • Description: Adds, removes or replaces individual headers in the request being processed.
  • Type: nested table (indexed by number) representing an array of request headers as {[1]='Name',[2]='Value'} pairs that are added to the request being processed, or overwriting existing request headers with colliding names. To remove a header from the request, specify nil as value, i.e. Headers={..., {[1]='foo',[2]=nil} ...}. Duplicate names are supported. A multi-value header such as Foo: bar1,bar2 is defined by specifying Headers={..., {[1]='foo',[2]='bar1'}, {[1]='foo',[2]='bar2'}, ...}.
• Host
  • Description: Replaces the host that the request is sent to.
  • Type: string
  • Example: 'new-host.example.com', '192.0.2.7'
• Port
  • Description: Replaces the TCP port that the request is sent to.
  • Type: number
  • Example: 8081
• Protocol
  • Description: Decides which protocol that will be used for sending the request. Valid protocols are 'HTTP' and 'HTTPS'.
  • Type: string
  • Example: 'HTTP', 'HTTPS'

Example of a host_request_translation_function body that sets the request path to a hardcoded value and adds the hardcoded query parameter a=b:

-- Statements go here
print('Setting hardcoded Path and QueryParameters')
return HTTPRequest({
  Path = '/content.mpd',
  QueryParameters = {
    {'a','b'}
  }
})

Arguments

The following (iterable) arguments will be known by the function:

QueryParameters

• Type: nested table (indexed by number).

• Description: Array of query parameters as {[1]='Name',[2]='Value'} pairs that are present in the query string of the request from the client to the router. Format identical to the HTTPRequest.QueryParameters-field specified for the return value above.

• Example usage:

  for _, queryParam in pairs(QueryParameters) do
    print(queryParam[1]..'='..queryParam[2])
  end
  

Headers

• Type: nested table (indexed by number).

• Description: Array of request headers as {[1]='Name',[2]='Value'} pairs that are present in the request from the client to the router. Format identical to the HTTPRequest.Headers-field specified for the return value above. A multi-value header such as Foo: bar1,bar2 is seen in host_request_translation_function as Headers={..., {[1]='foo',[2]='bar1'}, {[1]='foo',[2]='bar1'}, ...}.

• Example usage:

  for _, header in pairs(Headers) do
    print(header[1]..'='..header[2])
  end
  

Global tables

The following non-iterable global tables are available for use by the host_request_translation_function.

Table outgoing_request

The outgoing_request table contains the request that is to be sent to the host.

• outgoing_request.method
  • Description: HTTP request method.
  • Type: string
  • Example: 'GET', 'POST'
• outgoing_request.body
  • Description: HTTP request body string.
  • Type: string or nil
  • Example: '{"foo": "bar"}'
• outgoing_request.major_version
  • Description: Major HTTP version such as x in HTTP/x.1.
  • Type: integer
  • Example: 1
• outgoing_request.minor_version
  • Description: Minor HTTP version such as x in HTTP/1.x.
  • Type: integer
  • Example: 1
• outgoing_request.protocol
  • Description: Transfer protocol variant.
  • Type: string
  • Example: 'HTTP', 'HTTPS'

Table outgoing_request_headers

Contains the request headers from the request that is to be sent to the host, keyed by name.

Example:

print(outgoing_request_headers['X-Forwarded-For'])

Multiple values are separated with a comma.

Additional data

In addition to the arguments above, the following Lua tables, documented in Global Lua Tables, provide additional data that is available when executing the request translation function:

• selection_input
• request
• request_query_params
• request_headers

5.6 - Response Translation Function

Specifies the body of a Lua function that inspects every outgoing HTTP response and overwrites individual fields before being sent to the client.

Returns nil when nothing is to be changed, or HTTPResponse(t) where t is a table with any of the following optional fields:

• Code
  • Description: Replaces status code in the response being sent.
  • Type: integer
  • Example: 200, 404
• Text
  • Description: Replaces status text in the response being sent.
  • Type: string
  • Example: 'OK', 'Not found'
• MajorVersion
  • Description: Replaces major HTTP version such as x in HTTP/x.1 in the response being sent.
  • Type: integer
  • Example: 1
• MinorVersion
  • Description: Replaces minor HTTP version such as x in HTTP/1.x in the response being sent.
  • Type: integer
  • Example: 1
• Protocol
  • Description: Replaces protocol in the response being sent.
  • Type: string
  • Example: 'HTTP', 'HTTPS'
• Body
  • Description: Replaces body in the response being sent.
  • Type: string or nil
  • Example: '{"foo": "bar"}'
• Headers
  • Description: Adds, removes or replaces individual headers in the response being sent.
  • Type: nested table (indexed by number) representing an array of response headers as {[1]='Name',[2]='Value'} pairs that are added to the response being sent, or overwriting existing request headers with colliding names. To remove a header from the response, specify nil as value, i.e. Headers={..., {[1]='foo',[2]=nil} ...}. Duplicate names are supported. A multi-value header such as Foo: bar1,bar2 is defined by specifying Headers={..., {[1]='foo',[2]='bar1'}, {[1]='foo',[2]='bar2'}, ...}.

Example of a response_translation_function body that sets the Location header to a hardcoded value:

-- Statements go here
print('Setting hardcoded Location')
return HTTPResponse({
  Headers = {
    {'Location', 'cdn1.com/content.mpd?a=b'}
  }
})

Arguments

The following (iterable) arguments will be known by the function:

Headers

• Type: nested table (indexed by number).

• Description: Array of response headers as {[1]='Name',[2]='Value'} pairs that are present in the response being sent. Format identical to the HTTPResponse.Headers-field specified for the return value above. A multi-value header such as Foo: bar1,bar2 is seen in response_translation_function as Headers={..., {[1]='foo',[2]='bar1'}, {[1]='foo',[2]='bar1'}, ...}.

• Example usage:

  for _, header in pairs(Headers) do
    print(header[1]..'='..header[2])
  end
  

Additional data

In addition to the arguments above, the following Lua tables, documented in Global Lua Tables, provide additional data that is available when executing the response translation function:

• selection_input
• session_groups
• request
• request_query_params
• request_headers
• response
• response_headers
• session
• session_query_params