Operations

Operators Guide

1: Services
2: Docker

This guide describes how to perform day-to-day operations of the ACD Router and its associated services, collectively known as the Director.

Component Overview

To effectively operate the Director software, it is important to understand the composition of the various software components and how they are deployed.

Each Director instance functions as an independent system, comprising multiple containerized services. These containers are managed by a standard container runtime and are seamlessly integrated with the host’s operating system to enhance the overall operator experience.

For Red Hat Enterprise Linux 8 based operating systems, the Podman container runtime is used for container management. In the case of older Red Hat releases, Docker serves as the container runtime. Although both

runtimes

utilize the same OCI compliant container images and offer equivalent levels of abstraction, performance, and security, the key distinction lies in Podman’s ability to operate without an additional daemon service running on the host. Unlike Docker, Podman manages each container as a separate process, eliminating the reliance on a shared daemon and mitigating the risk of a single-point-of-failure scenario.

Although several distinct services make up the Director, the primary component is the router. The router is responsible for listening for incoming requests, processing the request, and redirecting the client to the appropriate host, or CDN to deliver the requested content.

Two additional containers are responsible for configuration management. Those are confd and confd-transformer. The former manages a local database of configuration metadata and provides a REST API for managing the configuration. The confd-transformer simply listens for configuration changes from confd and adapts that configuration to a format suitable for the router to ingest. For additional information about setting up and using confd see here..

The next two components, the edns-proxy and the convoy-bridge allow the router to communicate with an EDNS server for EDNS-based routing, and with synchronization with Convoy respectively. Additional information about the EDNS-Proxy is available here.. For the Convoy Bridge service see here..

The remaining containers are useful for metrics, monitoring, and alerting. These include prometheus and grafana for monitoring and analytics, fluentbit and alertmanager for monitoring and alarms, and grafana-loki for log aggregation.

1 - Services

Starting / Stopping / Monitoring Services

Each container shipped with the Director is fully-integrated with the systemd service on the host, enabling easy management using standard systemd commands. The logs for each container are also full-integrated with journald to simplify troubleshooting.

In order to integrate the Podman containers with systemd, a common prefix of acd- has been applied to each service name. For example the router container is managed by the service acd-router, and the confd container is managed by the service acd-confd. These same prefixed names apply while fetching logs via journald. This common prefix aids in grouping the related services as well as provides simpler filtering for tab-completion.

Starting / Stopping Services

Standard systemd commands should be used to start and stop the services.

systemctl start acd-router - Starts the router container.
systemctl stop acd-router - Stops the router container.
systemctl status acd-router - Displays the status of the router container.

Due to the limitation of needing the acd- prefix, it provides the ability to work with all ACD services in a group. For example:

systemctl status 'acd-*' - Display the status of all installed ACD components.
systemctl start 'acd-*' - Start all ACD components.

Logging

Each ACD component corresponds to a journal entry with the same unit name, with the acd- prefix. Standard journald commands can be used to view and manage the logging.

journalctl -u acd-router - Display the logs for the router container

Access Log

Access logs can be obtained from journald by using the following command:

journalctl SYSLOG_IDENTIFIER=router ACCESSLOG=1

This will output the router’s access logs in a standard format.

Troubleshooting

Some additional logging may be available in the filesystem, the paths of which can be determined by executing the ew-sysinfo command. See Diagnostics. for additional details.

2 - Docker

Special operating instructions for Red Hat Enterprise Linux 7

Support for Red Hat Enterprise Linux 7 and compatible Operating Systems is considered best-effort and the instructions included here are only maintained for archival purposes.

Since Red Hat Enterprise Linux 7 uses the Docker runtime, there are several key differences which are noted below.

Starting and Stopping Services

Since the containers are not integrated into systemd under Docker, standard Docker container management commands must be used to start and stop the services.

docker ps - Displays all running containers along with their relevant metadata.
docker stop router - Stops the router container.
docker start router - Starts the router container.

Obtaining container logs

The standard docker logs command should be used to view the logs for the various containers.

docker logs router - Display the complete log for the router container.
docker logs -f router - Display and tail the log for the router container.

Router Access Logs are not available under Docker.