Configuration specification¶

Every integration has a specification detailing all the options that influence behavior. These YAML files are located at <INTEGRATION>/assets/configuration/spec.yaml.

Producer¶

The producer's job is to read a specification and:

1. Validate for correctness
2. Populate all unset default fields
3. Resolve any defined templates
4. Output the complete specification as JSON for arbitrary consumers

Consumers¶

Consumers may utilize specs in a number of scenarios, such as:

• rendering example configuration shipped to end users
• documenting all options in-app & on the docs site
• form for creating configuration in multiple formats on Integration tiles
• automatic configuration loading for Checks
• Agent based and/or in-app validator for user-supplied configuration

Schema¶

The root of every spec is a map with 3 keys:

• name - The display name of what the spec refers to e.g. Postgres, Datadog Agent, etc.
• version - The released version of what the spec refers to
• files - A list of all files that influence behavior

Files¶

Every file has 3 possible attributes:

• name - This is the name of the file the Agent will look for (REQUIRED)
• example_name - This is the name of the example file the Agent will ship. If none is provided, the default will be conf.yaml.example. The exceptions are as follows:
• Auto-discovery files, which are named auto_conf.yaml
• Python-based core check default files, which are named conf.yaml.default
• options - A list of options (REQUIRED)

Options¶

Every option has 10 possible attributes:

• name - This is the name of the option (REQUIRED)
• description - Information about the option. This can be a multi-line string, but each line must contain fewer than 120 characters (REQUIRED).
• required - Whether or not the option is required for basic functionality. It defaults to false.
• hidden - Whether or not the option should not be publicly exposed. It defaults to false.
• display_priority - An integer representing the relative visual rank the option should take on compared to other options when publicly exposed. It defaults to 0, meaning that every option will be displayed in the order defined in the spec.

• deprecation - If the option is deprecated, a mapping of relevant information. For example:

  deprecation:
    Agent version: 8.0.0
    Migration: |
      do this
      and that
  

• multiple - Whether or not options may be selected multiple times like instances or just once like init_config

• multiple_instances_defined - Whether or not we separate the definition into multiple instances or just one
• metadata_tags - A list of tags (like docs:foo) that can be used for unexpected use cases
• options - Nested options, indicating that this is a section like instances or logs
• value - The expected type data

There are 2 types of options: those with and without a value. Those with a value attribute are the actual user-controlled settings that influence behavior like username. Those without are expected to be sections and therefore must have an options attribute. An option cannot have both attributes.

Options with a value (non-section) also support:

• secret - Whether or not consumers should treat the option as sensitive information like password. It defaults to false.

Values¶

The type system is based on a loose subset of OpenAPI 3 data types.

The differences are:

• Only the minimum and maximum numeric modifiers are supported
• Only the pattern string modifier is supported
• The properties object modifier is not a map, but rather a list of maps with a required name attribute. This is so consumers will load objects consistently regardless of language guarantees regarding map key order.

Values also support 1 field of our own:

• example - An example value, only required if the type is boolean. The default is <OPTION_NAME>.

Templates¶

Every option may reference pre-defined templates using a key called template. The template format looks like path/to/template_file where path/to must point an existing directory relative to a template directory and template_file must have the file extension .yaml or .yml.

You can use custom templates that will take precedence over the pre-defined templates by using the template_paths parameter of the ConfigSpec class.

Override¶

For occasions when deeply nested default template values need to be overridden, there is the ability to redefine attributes via a . (dot) accessor.

options:
- template: instances/http
  overrides:
    timeout.value.example: 42

Example file consumer¶

The example consumer uses each spec to render the example configuration files that are shipped with every Agent and individual Integration release.

It respects a few extra option-level attributes:

• example - A complete example of an option in lieu of a strictly typed value attribute
• enabled - Whether or not to un-comment the option, overriding the behavior of required
• display_priority - This is an integer affecting the order in which options are displayed, with higher values indicating higher priority. The default is 0.

It also respects a few extra fields under the value attribute of each option:

• display_default - This is the default value that will be shown in the header of each option, useful if it differs from the example. You may set it to null explicitly to disable showing this part of the header.
• compact_example - Whether or not to display complex types like arrays in their most compact representation. It defaults to false.

Usage¶

Use the --sync flag of the config validation command to render the example configuration files.

Data model consumer¶

The model consumer uses each spec to render the pydantic models that checks use to validate and interface with configuration. The models are shipped with every Agent and individual Integration release.

It respects an extra field under the value attribute of each option:

• default - This is the default value that options will be set to, taking precedence over the example.
• validators - This refers to an array of pre-defined field validators to use. Every entry will refer to a relative import path to a field validator under datadog_checks.base.utils.models.validation and will be executed in the defined order.

Usage¶

Use the --sync flag of the model validation command to render the data model files.

API¶

class ConfigSpec(object):
    def __init__(self, contents: str, template_paths: List[str] = None, source: str = None, version: str = None):
        """
        Parameters:

            contents:
                the raw text contents of a spec
            template_paths:
                a sequence of directories that will take precedence when looking for templates
            source:
                a textual representation of what the spec refers to, usually an integration name
            version:
                the version of the spec to default to if the spec does not define one
        """
        self.contents = contents
        self.source = source
        self.version = version
        self.templates = ConfigTemplates(template_paths)
        self.data: Union[dict, None] = None
        self.errors = []

    def load(self) -> None:
        """
        This function de-serializes the specification and:
        1. fills in default values
        2. populates any selected templates
        3. accumulates all error/warning messages
        If the `errors` attribute is empty after this is called, the `data` attribute
        will be the fully resolved spec object.
        """
        if self.data is not None and not self.errors:
            return

        try:
            self.data = yaml.safe_load(self.contents)
        except Exception as e:
            self.errors.append(f'{self.source}: Unable to parse the configuration specification: {e}')
            return

        spec_validator(self.data, self)

contents:
    the raw text contents of a spec
template_paths:
    a sequence of directories that will take precedence when looking for templates
source:
    a textual representation of what the spec refers to, usually an integration name
version:
    the version of the spec to default to if the spec does not define one

def __init__(self, contents: str, template_paths: List[str] = None, source: str = None, version: str = None):
    """
    Parameters:

        contents:
            the raw text contents of a spec
        template_paths:
            a sequence of directories that will take precedence when looking for templates
        source:
            a textual representation of what the spec refers to, usually an integration name
        version:
            the version of the spec to default to if the spec does not define one
    """
    self.contents = contents
    self.source = source
    self.version = version
    self.templates = ConfigTemplates(template_paths)
    self.data: Union[dict, None] = None
    self.errors = []

def load(self) -> None:
    """
    This function de-serializes the specification and:
    1. fills in default values
    2. populates any selected templates
    3. accumulates all error/warning messages
    If the `errors` attribute is empty after this is called, the `data` attribute
    will be the fully resolved spec object.
    """
    if self.data is not None and not self.errors:
        return

    try:
        self.data = yaml.safe_load(self.contents)
    except Exception as e:
        self.errors.append(f'{self.source}: Unable to parse the configuration specification: {e}')
        return

    spec_validator(self.data, self)