Operations

Operators Guide

1: Services

This guide describes how to perform day-to-day operations of the ACD Router and its associated services, collectively known as the Director.

Component Overview

To effectively operate the Director software, it is important to understand the composition of the various software components and how they are deployed.

Each Director instance functions as an independent system, comprising multiple containerized services. These containers are managed by a standard container runtime and are seamlessly integrated with the host’s operating system to enhance the overall operator experience.

The containers are managed by the Podman container runtime, which operates without additional daemon services running on the host. Unlike Docker, Podman manages each container as a separate process, eliminating the reliance on a shared daemon and mitigating the risk of a single-point-of-failure scenario.

Although several distinct services make up the Director, the primary component is the router. The router is responsible for listening for incoming requests, processing the request, and redirecting the client to the appropriate host, or CDN to deliver the requested content.

Two additional containers are responsible for configuration management. Those are confd and confd-transformer. The former manages a local database of configuration metadata and provides a REST API for managing the configuration. The confd-transformer simply listens for configuration changes from confd and adapts that configuration to a format suitable for the router to ingest. For additional information about setting up and using confd see here..

The next two components, the edns-proxy and the convoy-bridge allow the router to communicate with an EDNS server for EDNS-based routing, and with synchronization with Convoy respectively. Additional information about the EDNS-Proxy is available here.. For the Convoy Bridge service see here..

The remaining containers are useful for metrics, monitoring, and alerting. These include prometheus and grafana for monitoring and analytics, and alertmanager for monitoring and alarms.

1 - Services

Starting / Stopping / Monitoring Services

Each container shipped with the Director is fully-integrated with the systemd service on the host, enabling easy management using standard systemd commands. The logs for each container are also full-integrated with journald to simplify troubleshooting.

In order to integrate the Podman containers with systemd, a common prefix of acd- has been applied to each service name. For example the router container is managed by the service acd-router, and the confd container is managed by the service acd-confd. These same prefixed names apply while fetching logs via journald. This common prefix aids in grouping the related services as well as provides simpler filtering for tab-completion.

Starting / Stopping Services

Standard systemd commands should be used to start and stop the services.

systemctl start acd-router - Starts the router container.
systemctl stop acd-router - Stops the router container.
systemctl status acd-router - Displays the status of the router container.

Due to the limitation of needing the acd- prefix, it provides the ability to work with all ACD services in a group. For example:

systemctl status 'acd-*' - Display the status of all installed ACD components.
systemctl start 'acd-*' - Start all ACD components.

Logging

Each ACD component corresponds to a journal entry with the same unit name, with the acd- prefix. Standard journald commands can be used to view and manage the logging.

journalctl -u acd-router - Display the logs for the router container

Access Log

HTTP access logging can be enabled by typing this:

confcli services.routing.tuning.general.accessLog true

When access logging is enabled, the Director will log all content requests. The logs can be obtained from journald by using the following command:

journalctl SYSLOG_IDENTIFIER=router ACCESSLOG=1

This will output the router’s access logs in a standard format.

Troubleshooting

Some additional logging may be available in the filesystem, the paths of which can be determined by executing the ew-sysinfo command. See Diagnostics. for additional details.