Metadata¶

Often, you will want to collect mostly unstructured data that doesn't map well to tags, like fine-grained product version information.

The base class provides a method that handles such cases. The collected data is captured by flares, displayed on the Agent's status page, and will eventually be queryable in-app.

Interface¶

The set_metadata method of the base class updates cached metadata values, which are then sent by the Agent at regular intervals.

It requires 2 arguments:

name - The name of the metadata.
value - The value for the metadata. If name has no transformer defined then the raw value will be submitted and therefore it must be a str.

The method also accepts arbitrary keyword arguments that are forwarded to any defined transformers.

Transformers¶

Custom transformers may be defined via a class level attribute METADATA_TRANSFORMERS.

This is a mapping of metadata names to functions. When you call self.set_metadata(name, value, **options), if name is in this mapping then the corresponding function will be called with the value, and the return value(s) will be collected instead.

Transformer functions must satisfy the following signature:

def transform_<NAME>(value: Any, options: dict) -> Union[str, Dict[str, str]]:

If the return type is str, then it will be sent as the value for name. If the return type is a mapping type, then each key will be considered a name and will be sent with its (str) value.

For example, the following would collect an entity named square with a value of '25':

from datadog_checks.base import AgentCheck


class AwesomeCheck(AgentCheck):
    METADATA_TRANSFORMERS = {
        'square': lambda value, options: str(int(value) ** 2)
    }

    def check(self, instance):
        self.set_metadata('square', '5')

There are a few default transformers, which can be overridden by custom transformers.

Source code in datadog_checks_base/datadog_checks/base/utils/metadata/core.py

class MetadataManager(object):
    """
    Custom transformers may be defined via a class level attribute `METADATA_TRANSFORMERS`.

    This is a mapping of metadata names to functions. When you call
    `#!python self.set_metadata(name, value, **options)`, if `name` is in this mapping then
    the corresponding function will be called with the `value`, and the return
    value(s) will be collected instead.

    Transformer functions must satisfy the following signature:

    ```python
    def transform_<NAME>(value: Any, options: dict) -> Union[str, Dict[str, str]]:
    ```

    If the return type is `str`, then it will be sent as the value for `name`. If the return type is a mapping type,
    then each key will be considered a `name` and will be sent with its (`str`) value.

    For example, the following would collect an entity named `square` with a value of `'25'`:

    ```python
    from datadog_checks.base import AgentCheck


    class AwesomeCheck(AgentCheck):
        METADATA_TRANSFORMERS = {
            'square': lambda value, options: str(int(value) ** 2)
        }

        def check(self, instance):
            self.set_metadata('square', '5')
    ```

    There are a few default transformers, which can be overridden by custom transformers.
    """

    __slots__ = ('check_id', 'check_name', 'logger', 'metadata_transformers')

    def __init__(self, check_name, check_id, logger=None, metadata_transformers=None):
        self.check_name = check_name
        self.check_id = check_id
        self.logger = logger or LOGGER
        self.metadata_transformers = {'version': self.transform_version}

        if metadata_transformers:
            self.metadata_transformers.update(metadata_transformers)

    def submit_raw(self, name, value):
        datadog_agent.set_check_metadata(self.check_id, to_native_string(name), to_native_string(value))

    def submit(self, name, value, options):
        transformer = self.metadata_transformers.get(name)
        if transformer:
            try:
                transformed = transformer(value, options)
            except Exception as e:
                if is_primitive(value):
                    self.logger.debug('Unable to transform `%s` metadata value `%s`: %s', name, value, e)
                else:
                    self.logger.debug('Unable to transform `%s` metadata: %s', name, e)

                return

            if isinstance(transformed, str):
                self.submit_raw(name, transformed)
            else:
                for transformed_name, transformed_value in iteritems(transformed):
                    self.submit_raw(transformed_name, transformed_value)
        else:
            self.submit_raw(name, value)

    def transform_version(self, version, options):
        """
        Transforms a version like `1.2.3-rc.4+5` to its constituent parts. In all cases,
        the metadata names `version.raw` and `version.scheme` will be collected.

        If a `scheme` is defined then it will be looked up from our known schemes. If no
        scheme is defined then it will default to `semver`. The supported schemes are:

        - `regex` - A `pattern` must also be defined. The pattern must be a `str` or a pre-compiled
          `re.Pattern`. Any matching named subgroups will then be sent as `version.<GROUP_NAME>`. In this case,
          the check name will be used as the value of `version.scheme` unless `final_scheme` is also set, which
          will take precedence.
        - `parts` - A `part_map` must also be defined. Each key in this mapping will be considered
          a `name` and will be sent with its (`str`) value.
        - `semver` - This is essentially the same as `regex` with the `pattern` set to the standard regular
          expression for semantic versioning.

        Taking the example above, calling `#!python self.set_metadata('version', '1.2.3-rc.4+5')` would produce:

        | name | value |
        | --- | --- |
        | `version.raw` | `1.2.3-rc.4+5` |
        | `version.scheme` | `semver` |
        | `version.major` | `1` |
        | `version.minor` | `2` |
        | `version.patch` | `3` |
        | `version.release` | `rc.4` |
        | `version.build` | `5` |
        """
        scheme, version_parts = parse_version(version, options)
        if scheme == 'regex' or scheme == 'parts':
            scheme = options.get('final_scheme', self.check_name)

        data = {'version.{}'.format(part_name): part_value for part_name, part_value in iteritems(version_parts)}
        data['version.raw'] = version
        data['version.scheme'] = scheme

        return data

`transform_version(version, options)` ¶

Transforms a version like 1.2.3-rc.4+5 to its constituent parts. In all cases, the metadata names version.raw and version.scheme will be collected.

If a scheme is defined then it will be looked up from our known schemes. If no scheme is defined then it will default to semver. The supported schemes are:

regex - A pattern must also be defined. The pattern must be a str or a pre-compiled re.Pattern. Any matching named subgroups will then be sent as version.<GROUP_NAME>. In this case, the check name will be used as the value of version.scheme unless final_scheme is also set, which will take precedence.
parts - A part_map must also be defined. Each key in this mapping will be considered a name and will be sent with its (str) value.
semver - This is essentially the same as regex with the pattern set to the standard regular expression for semantic versioning.

Taking the example above, calling self.set_metadata('version', '1.2.3-rc.4+5') would produce:

name	value
`version.raw`	`1.2.3-rc.4+5`
`version.scheme`	`semver`
`version.major`	`1`
`version.minor`	`2`
`version.patch`	`3`
`version.release`	`rc.4`
`version.build`	`5`

Source code in datadog_checks_base/datadog_checks/base/utils/metadata/core.py

def transform_version(self, version, options):
    """
    Transforms a version like `1.2.3-rc.4+5` to its constituent parts. In all cases,
    the metadata names `version.raw` and `version.scheme` will be collected.

    If a `scheme` is defined then it will be looked up from our known schemes. If no
    scheme is defined then it will default to `semver`. The supported schemes are:

    - `regex` - A `pattern` must also be defined. The pattern must be a `str` or a pre-compiled
      `re.Pattern`. Any matching named subgroups will then be sent as `version.<GROUP_NAME>`. In this case,
      the check name will be used as the value of `version.scheme` unless `final_scheme` is also set, which
      will take precedence.
    - `parts` - A `part_map` must also be defined. Each key in this mapping will be considered
      a `name` and will be sent with its (`str`) value.
    - `semver` - This is essentially the same as `regex` with the `pattern` set to the standard regular
      expression for semantic versioning.

    Taking the example above, calling `#!python self.set_metadata('version', '1.2.3-rc.4+5')` would produce:

    | name | value |
    | --- | --- |
    | `version.raw` | `1.2.3-rc.4+5` |
    | `version.scheme` | `semver` |
    | `version.major` | `1` |
    | `version.minor` | `2` |
    | `version.patch` | `3` |
    | `version.release` | `rc.4` |
    | `version.build` | `5` |
    """
    scheme, version_parts = parse_version(version, options)
    if scheme == 'regex' or scheme == 'parts':
        scheme = options.get('final_scheme', self.check_name)

    data = {'version.{}'.format(part_name): part_value for part_name, part_value in iteritems(version_parts)}
    data['version.raw'] = version
    data['version.scheme'] = scheme

    return data

Last update: August 16, 2023

Metadata¶

Interface¶

Transformers¶

transform_version(version, options) ¶

`transform_version(version, options)` ¶