Module runtime 

Runtime system.

This module contains the core components of the runtime system, including supervisors and processes. It’s directly inspired by Erlang/OTP.

To quote the Erlang/OTP documentation:

Workers are processes that perform computations and other actual work. Supervisors are processes that monitor workers. A supervisor can restart a worker if something goes wrong. The supervision tree is a hierarchical arrangement of code into supervisors and workers, which makes it possible to design and program fault-tolerant software.

Processes

An asynchronous system is composed of independent units of computation running concurrently, such as a set of tasks executing on a thread pool. We refer to these as processes. In other systems, these might be called actors, tasks, fibers, virtual threads, goroutines, or something else. Processes are lightweight and able to be (generally) created and destroyed cheaply.

Processes have a few key attributes and invariants:

• every process is a future that runs as an independent asynchronous task on a Tokio runtime
• every process has a unique numerical identifier and a semi-unique name

Unlike Erlang processes, Saluki processes don’t have an inherent mailbox or message passing capabilities. As well, processes can’t run by themselves. They must be supervised.

Supervisors

Supervisors are themselves processes whose only job is to supervise other processes, also called workers. In a supervisor, workers are added and configured through a common convention that allows defining how the worker is created (or recreated on failure), how many times it can be restarted, and more. Supervisors themselves can also be workers, and so nested supervision trees can be constructed.

Supervisors include a number of configurable settings that allow customizing the behavior of how workers are managed, which in turn allows building fault-tolerant systems: we can restart workers for transient failures, give up for permanent failures, and so on.

Spawning children

Children are usually declared up front with Supervisor::add_worker, but a supervisor can also take on new children while it runs. There are exactly two ways to do that, and they differ only in how the supervisor is identified:

• spawn, which targets the ambient supervisor: the one supervising the currently running process. This is the tokio::spawn of supervision, and is what code running under supervision should reach for.
• SupervisorHandle, which names a supervisor explicitly.

Either way, a child is described with ChildBuilder – reached through worker and supervisable for the ambient supervisor, or the identically named methods on SupervisorHandle for a named one. The builder is the only way to configure a child: ChildSpecification is the description it produces and carries no settings of its own, so a combination the builder declines to offer can’t be assembled around it.

A child that is itself a Supervisor can be handed to any of the above directly, which is all most callers need. nested_supervisor (and SupervisorHandle::nested_supervisor) exists for the two settings that aren’t reachable that way – the restart policy and significance. It matters most for a dynamically spawned subtree, which would otherwise be temporary and so quietly stay dead once it terminated.

Both are synchronous and infallible. As with tokio::spawn, a child being accepted doesn’t mean it will run: a supervisor that shuts down before it reaches the child never starts it at all.

Supervision trees

As supervisors can be nested, this allows building a tree of supervisors (hence supervision trees) where leaf supervisors manage workers specific to a certain area, and parent supervisors manage the leaf supervisors. For example, for a server application serving multiple API endpoints, each endpoint might be managed by a separate supervisor: a worker for accepting connections, a worker for each connection, and so on. Above those supervisors, a parent supervisor manages each leaf supervisor, and potentially other workers that provide necessary services utilized by each endpoint, such as logging, metrics, or other infrastructure services.

As every supervisor can define its own specific restart strategy, and behavior, this allows for more granular grouping and control over which set of workers must be restarted if a related worker fails, and how those failures propagate up and down the supervision tree.

Inspecting the tree

A supervision tree is also a description of what a process is doing, which makes it worth being able to read at runtime. Supervisor::tree_handle returns a read-only handle whose snapshot walks the tree from that supervisor downward, reporting each node’s name, process, restart policy, restart count, lifetime, and resource usage, with its children nested beneath it. A handle can be taken before the supervisor starts and stays valid across every restart of it, which makes it suitable for handing to a worker that serves an operator-facing diagnostics endpoint.

Examples

See the basic_supervisor example which shows how supervisors and workers are composed together, as well as how failed workers and supervisors are restarted.

Modules

state

Runtime state management utilities.

Structs

ChildBuilder

Builder for a yet-to-be-started child task.

ChildId

Identifier for a child managed by a Supervisor.

ChildSpecification

A specification for a process to be added to a Supervisor.

FnWorker

A Supervisable worker built from a plain future.

LoweredChild

A child specification lowered into the supervisor’s internal representation.

NestedSupervisorBuilder

Builder for a yet-to-be-started nested supervisor.

NodeSnapshot

One node – a supervisor or a worker – in a TreeSnapshot.

OneShot

Marks a builder whose worker can only be initialized once.

ProcessId

Process identifier.

ResourceUsage

Cumulative resource usage for one resource group.

RestartStrategy

Restart strategy for a supervisor.

Restartable

Marks a builder whose worker can be initialized more than once.

RuntimeConfiguration

Configuration for a dedicated Tokio runtime.

SupervisionSettings

How a supervisor supervises its children, and how it has fared.

SupervisionTreeAPIHandler

An API handler for reporting the state of a supervision tree.

SupervisionTreeHandle

A read-only handle for taking snapshots of a supervision tree.

SupervisionTreeState

State used for the supervision tree API handler.

SupervisionTreeWorker

A worker that exposes a supervision tree over the control plane.

Supervisor

Supervises a set of workers.

SupervisorHandle

A handle for spawning dynamic children on a running Supervisor.

SupervisorSpec

Child specification state for a supervisor.

Terminable

Marks a builder whose worker can be initialized more than once but has been narrowed to a restart policy that lets it terminate for good.

TreeSnapshot

A point-in-time view of a supervision tree.

TreeTotals

Aggregate counts across a whole TreeSnapshot.

UnixMillis

A point in time, as milliseconds since the Unix epoch.

WorkerSpec

Child specification state for a worker.

Enums

AutoShutdown

Policy for automatically shutting a supervisor down based on the termination of its significant children.

InitializationError

Initialization errors.

NodeKind

Whether a node supervises other nodes or performs work itself.

NodeState

Where a node is in its lifecycle.

RestartMode

Restart mode for child processes.

RestartType

Restart policy for an individual child process.

RuntimeMode

Controls which runtime a supervisor runs on.

ShutdownStrategy

Strategy for shutting down a process.

SupervisorError

Supervisor errors.

Constants

SUPERVISION_TREE_ROUTE

API route serving a snapshot of a supervision tree.

Traits

BuilderState

The kind of worker a ChildBuilder is describing.

CanTerminate

Marks a builder whose child can terminate without being restarted.

ChildState

Child specification state.

IntoWorkerResult

An output type that a function-based worker can produce.

Supervisable

A supervisable process.

Functions

get_sanitized_name

Sanitizes name into a single process-name segment.

nested_supervisor

Creates a builder for a nested supervisor on the ambient supervisor.

spawn

Spawns a child on the ambient supervisor.

supervisable

Creates a builder for a supervisable child task on the ambient supervisor.

worker

Creates a builder for a child task on the ambient supervisor.

Type Aliases

SupervisorFuture

A Future that represents the execution of a supervised process.