Route on Content Type

How to write ESB3024 Router configurations for content-based routing

This page describes how to write configuration for content-based routing. For configuration in general, see Configuration.

Two ways to route on content type will be demonstrated: content path based and hostname based using session groups and classifiers. For more details on session groups and classifiers, see Session Groups and Classification.

Content Path

Routing on content path is done through a combination of session group classifiers and Lua script weight functions. Two suitable classifiers for content path matching are string_match_rule and regex_rule.

The pattern is a string matching the path segment of a request URL, including the file name. When selecting the string_match_rule type, use asterisks for wildcard matching. When selecting the regex_rule follow the C++11 std::regex ECMAScript syntax to make valid patterns. Make sure to write patterns that match the entire path segment, not just part of it.

Note that each content classifier normally has to be in its own list. This is because classifiers within the same inner list all have to match for the entire list to be true, and that is impossible for classifiers that match against different content paths. When the classifiers are in their own lists, it’s enough that one of them matches for the outer classifier list to also match.

Simple Example

$ confcli services.routing.classifiers -w
Running wizard for resource 'classifiers'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

classifiers : [
  classifier can be one of
    1: anonymousIp
    2: asnIds
    3: contentUrlPath
    4: contentUrlQueryParameters
    5: geoip
    6: hostName
    7: ipranges
    8: random
    9: regexMatcher
    10: stringMatcher
    11: subnet
    12: userAgent
  Choose element index or name: contentUrlPath
  Adding a 'contentUrlPath' element
    classifier : {
      name (default: ): live_content_classifier
      type (default: contentUrlPath): ⏎
      inverted (default: False): ⏎
      patternType (default: stringMatch): ⏎
      pattern (default: ): *live_tv*
    }
  Add another 'classifier' element to array 'classifiers'? [y/N]: y
  classifier can be one of
    1: anonymousIp
    2: asnIds
    3: contentUrlPath
    4: contentUrlQueryParameters
    5: geoip
    6: hostName
    7: ipranges
    8: random
    9: regexMatcher
    10: stringMatcher
    11: subnet
    12: userAgent
  Choose element index or name: contentUrlPath
  Adding a 'contentUrlPath' element
    classifier : {
      name (default: ): news_content_classifier
      type (default: contentUrlPath): ⏎
      inverted (default: False): ⏎
      patternType (default: stringMatch): regex
      pattern (default: ): /([^/]+/)?news_reports_\\d+/.*
    }
  Add another 'classifier' element to array 'classifiers'? [y/N]: y
  classifier can be one of
    1: anonymousIp
    2: asnIds
    3: contentUrlPath
    4: contentUrlQueryParameters
    5: geoip
    6: hostName
    7: ipranges
    8: random
    9: regexMatcher
    10: stringMatcher
    11: subnet
    12: userAgent
  Choose element index or name: contentUrlPath
  Adding a 'contentUrlPath' element
    classifier : {
      name (default: ): hls_content_classifier
      type (default: contentUrlPath): ⏎
      inverted (default: False): ⏎
      patternType (default: stringMatch): ⏎
      pattern (default: ): *.m3u8
    }
  Add another 'classifier' element to array 'classifiers'? [y/N]: n
]
Generated config:
{
  "classifiers": [
    {
      "name": "live_content_classifier",
      "type": "contentUrlPath",
      "inverted": false,
      "patternType": "stringMatch",
      "pattern": "*live_tv*"
    },
    {
      "name": "news_content_classifier",
      "type": "contentUrlPath",
      "inverted": false,
      "patternType": "regex",
      "pattern": "/([^/]+/)?news_reports_\\\\d+/.*"
    },
    {
      "name": "hls_content_classifier",
      "type": "contentUrlPath",
      "inverted": false,
      "patternType": "stringMatch",
      "pattern": "*.m3u8"
    }
  ]
}
Merge and apply the config? [y/n]: y

confcli services.routing.sessionGroups -w
Running wizard for resource 'sessionGroups'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

sessionGroups : [
  sessionGroup : {
    name (default: ): live_content
    classifiers : [
      classifier (default: ): live_content_classifier
      Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
    ]
  }
  Add another 'sessionGroup' element to array 'sessionGroups'? [y/N]: y
  sessionGroup : {
    name (default: ): news_content
    classifiers : [
      classifier (default: ): news_content_classifier
      Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
    ]
  }
  Add another 'sessionGroup' element to array 'sessionGroups'? [y/N]: y
  sessionGroup : {
    name (default: ): hls_content
    classifiers : [
      classifier (default: ): hls_content_classifier
      Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
    ]
  }
  Add another 'sessionGroup' element to array 'sessionGroups'? [y/N]: ⏎
]
Generated config:
{
  "sessionGroups": [
    {
      "name": "live_content",
      "classifiers": [
        "live_content_classifier"
      ]
    },
    {
      "name": "news_content",
      "classifiers": [
        "news_content_classifier"
      ]
    },
    {
      "name": "hls_content",
      "classifiers": [
        "hls_content_classifier"
      ]
    }
  ]
}
Merge and apply the config? [y/n]: y

{
  "content_server": {
    "http_enable": true,
    "http_port": 80,
    "https_enable": true,
    "https_port": 443
  },
  "session_groups": [
    {
      "id": 1,
      "name": "live_content",
      "classifiers": [
        [
          {
            "id": 1,
            "inverted": false,
            "name": "live_content_classifier",
            "rule": {
              "rule_type": "string_match_rule",
              "source": "session/content_url_path",
              "pattern": "*live_tv*"
            }
          }
        ]
      ],
      [
        [
          {
            "id": 2,
            "inverted": false,
            "name": "news_content_classifier",
            "rule": {
              "rule_type": "regex_rule",
              "source": "session/content_url_path",
              "pattern": "/([^/]+/)?news_reports_\\d+/.*"
            }
          }
        ]
      ]
    },
    {
      "id": 2,
      "name": "hls_content",
      "classifiers": [
        [
          {
            "id": 1,
            "inverted": false,
            "name": "hls_content_classifier",
            "rule": {
              "rule_type": "string_match_rule",
              "source": "session/content_url_path",
              "pattern": "*.m3u8"
            }
          }
        ]
      ]
    }
  ],
  "cdns": [
    {
      "id": "live-cdn",
      "http_port": 80,
      "https_port": 443,
      "redirecting": false,
      "manifest_availability_check": {
        "enabled": false,
        "session_group_ids": []
      }
    },
    {
      "id": "vod-hls-cdn",
      "http_port": 80,
      "https_port": 443,
      "redirecting": false,
      "manifest_availability_check": {
        "enabled": false,
        "session_group_ids": []
      }
    },
    {
      "id": "vod-cdn",
      "http_port": 80,
      "https_port": 443,
      "redirecting": false,
      "manifest_availability_check": {
        "enabled": false,
        "session_group_ids": []
      }
    }
  ],
  "hosts": [
    {
      "id": "live-host",
      "cdn_id": "live-cdn",
      "address_family": "ipv4",
      "host": "live-host.example"
    },
    {
      "id": "vod-hls-host",
      "cdn_id": "vod-hls-cdn",
      "address_family": "ipv4",
      "host": "vod-hls-host.example"
    },
    {
      "id": "vod-host",
      "cdn_id": "vod-cdn",
      "address_family": "ipv4",
      "host": "vod-host.example"
    }
  ],
  "routing": {
    "id": "routing_table",
    "member_order": "sequential",
    "members": [
      {
        "id": "live-node",
        "host_id": "live-host",
        "weight_function": "return session_groups.live_content and 1 or 0"
      },
      {
        "id": "vod-hls-node",
        "host_id": "vod-hls-host",
        "weight_function": "return session_groups.hls_content and 1 or 0"
      },
      {
        "id": "vod-node",
        "host_id": "vod-host",
        "weight_function": "return 1"
      }
    ]
  }
}

Hostname

In cases where there are separate domain or subdomain names for different content types, it’s simple to make classifiers that filter on those names.

The pattern is a string matching the hostname segment of a request URL, including the file name. When selecting the string_match_rule type, use asterisks for wildcard matching. When selecting the regex_rule follow the C++11 std::regex ECMAScript syntax to make valid patterns. Make sure to write patterns that match the entire hostname segment, not just part of it.

Note that each hostname classifier has to be in its own list. This is because classifiers within the same inner list all have to match for the entire list to be true, and that is impossible for classifiers that match against different hostnames. When the classifiers are in their own lists, it’s enough that one of them matches for the outer classifier list to also match.

Simple example

$ confcli services.routing.classifiers -w
Running wizard for resource 'classifiers'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

classifiers : [
  classifier can be one of
    1: anonymousIp
    2: asnIds
    3: contentUrlPath
    4: contentUrlQueryParameters
    5: geoip
    6: hostName
    7: ipranges
    8: random
    9: regexMatcher
    10: stringMatcher
    11: subnet
    12: userAgent
  Choose element index or name: hostName
  Adding a 'hostName' element
    classifier : {
      name (default: ): live_content_classifier
      type (default: hostName): ⏎
      inverted (default: False): ⏎
      patternType (default: stringMatch): ⏎
      pattern (default: ): live.example.com
    }
  Add another 'classifier' element to array 'classifiers'? [y/N]: y
  classifier can be one of
    1: anonymousIp
    2: asnIds
    3: contentUrlPath
    4: contentUrlQueryParameters
    5: geoip
    6: hostName
    7: ipranges
    8: random
    9: regexMatcher
    10: stringMatcher
    11: subnet
    12: userAgent
  Choose element index or name: hostName
  Adding a 'hostName' element
    classifier : {
      name (default: ): news_content_classifier
      type (default: hostName): ⏎
      inverted (default: False): ⏎
      patternType (default: stringMatch): regex
      pattern (default: ): /([^\\.]+/)\\.news_reports_\\d+/\\.example\\.com
    }
  Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
]
Generated config:
{
  "classifiers": [
    {
      "name": "live_content_classifier",
      "type": "hostName",
      "inverted": false,
      "patternType": "stringMatch",
      "pattern": "live.example.com"
    },
    {
      "name": "news_content_classifier",
      "type": "hostName",
      "inverted": false,
      "patternType": "regex",
      "pattern": "/([^\\\\.]+/)\\\\.news_reports_\\\\d+/\\\\.example\\\\.com"
    }
  ]
}
Merge and apply the config? [y/n]: y

$ confcli services.routing.sessionGroups -w
Running wizard for resource 'sessionGroups'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

sessionGroups : [
  sessionGroup : {
    name (default: ): live_content
    classifiers : [
      classifier (default: ): live_content_classifier
      Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
    ]
  }
  Add another 'sessionGroup' element to array 'sessionGroups'? [y/N]: y
  sessionGroup : {
    name (default: ): news_content
    classifiers : [
      classifier (default: ): news_content_classifier
      Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
    ]
  }
  Add another 'sessionGroup' element to array 'sessionGroups'? [y/N]: y
  sessionGroup : {
    name (default: ): hls_content
    classifiers : [
      classifier (default: ): hls_content_classifier
      Add another 'classifier' element to array 'classifiers'? [y/N]: ⏎
    ]
  }
  Add another 'sessionGroup' element to array 'sessionGroups'? [y/N]: ⏎
]
Generated config:
{
  "sessionGroups": [
    {
      "name": "live_content",
      "classifiers": [
        "live_content_classifier"
      ]
    },
    {
      "name": "news_content",
      "classifiers": [
        "news_content_classifier"
      ]
    },
    {
      "name": "hls_content",
      "classifiers": [
        "hls_content_classifier"
      ]
    }
  ]
}

{
  "content_server": {
    "http_enable": true,
    "http_port": 80,
    "https_enable": true,
    "https_port": 443
  },
  "session_groups": [
    {
      "id": 1,
      "name": "live_content",
      "classifiers": [
        [
          {
            "id": 1,
            "inverted": false,
            "name": "live_content_classifier",
            "rule": {
              "rule_type": "string_match_rule",
              "source": "session/hostname",
              "pattern": "live.example.com"
            }
          }
        ],
        [
          {
            "id": 2,
            "inverted": false,
            "name": "news_content_classifier",
            "rule": {
              "rule_type": "regex_rule",
              "source": "session/hostname",
              "pattern": "/([^\\.]+/)\\.news_reports_\\d+/\\.example\\.com"
            }
          }
        ]
      ]
    }
  ],
  "cdns": [
    {
      "id": "live-cdn",
      "http_port": 80,
      "https_port": 443,
      "redirecting": false,
      "manifest_availability_check": {
        "enabled": false,
        "session_group_ids": []
      }
    },
    {
      "id": "vod-cdn",
      "http_port": 80,
      "https_port": 443,
      "redirecting": false,
      "manifest_availability_check": {
        "enabled": false,
        "session_group_ids": []
      }
    }
  ],
  "hosts": [
    {
      "id": "live-host",
      "cdn_id": "live-cdn",
      "address_family": "ipv4",
      "host": "live-host.example"
    },
    {
      "id": "vod-host",
      "cdn_id": "vod-cdn",
      "address_family": "ipv4",
      "host": "vod-host.example"
    }
  ],
  "routing": {
    "id": "routing_table",
    "member_order": "sequential",
    "members": [
      {
        "id": "live-node",
        "host_id": "live-host", 
        "weight_function": "return session_groups.live_content and 1 or 0"
      },
      {
        "id": "vod-node",
        "host_id": "vod-host",  
        "weight_function": "return 1"
      }
    ]
  }
}

Using the session groups in confcli

The constructed session groups can be used in routing by creating a rule that references them:

$ confcli services.routing.rules -w
Running wizard for resource 'rules'

Hint: Hitting return will set a value to its default.
Enter '?' to receive the help string

rules : [
  rule can be one of
    1: allow
    2: consistentHashing
    3: contentPopularity
    4: deny
    5: firstMatch
    6: random
    7: rawGroup
    8: rawHost
    9: split
    10: weighted
  Choose element index or name: firstMatch
  Adding a 'firstMatch' element
    rule : {
      name (default: ): classifier_routing_rule
      type (default: firstMatch): ⏎
      targets : [
        target : {
          onMatch (default: ): live-host
          rule (default: ): in_all_session_groups('live_content', 'news_content')
        }
        Add another 'target' element to array 'targets'? [y/N]: y
        target : {
          onMatch (default: ): vod-hls-host
          rule (default: ): in_session_group('hls_content')
        }
        Add another 'target' element to array 'targets'? [y/N]: y
        target : {
          onMatch (default: ): offload-host
          rule (default: ): always()
        }
        Add another 'target' element to array 'targets'? [y/N]: ⏎
      ]
    }
  Add another 'rule' element to array 'rules'? [y/N]: ⏎
]
Generated config:
{
  "rules": [
    {
      "name": "classifier_routing_rule",
      "type": "firstMatch",
      "targets": [
        {
          "onMatch": "live-host",
          "condition": "in_all_session_groups('live_content', 'news_content')"
        },
        {
          "onMatch": "vod-hls-host",
          "condition": "in_session_group('hls_content')"
        },
        {
          "onMatch": "offload-host",
          "condition": "always()"
        }
      ],
    }
  ]
}
Merge and apply the config? [y/n]: y
$ confcli services.routing.entrypoint classifier_routing_rule
services.routing.entrypoint = classifier_routing_rule