Metadata¶
Often, you will want to collect mostly unstructured data that doesn't map well to tags, like fine-grained product version information.
The base class provides a method that handles such cases. The collected data is captured by flares, displayed on the Agent's status page, and will eventually be queryable in-app.
Interface¶
The set_metadata method of the base class updates cached metadata values, which are then sent by the Agent at regular intervals.
It requires 2 arguments:
name- The name of the metadata.value- The value for the metadata. Ifnamehas no transformer defined then the rawvaluewill be submitted and therefore it must be astr.
The method also accepts arbitrary keyword arguments that are forwarded to any defined transformers.
Transformers¶
Custom transformers may be defined via a class level attribute METADATA_TRANSFORMERS.
This is a mapping of metadata names to functions. When you call self.set_metadata(name, value, **options), if name is in this mapping then the corresponding function will be called with the value, and the return value(s) will be collected instead.
Transformer functions must satisfy the following signature:
def transform_<NAME>(value: Any, options: dict) -> Union[str, Dict[str, str]]:
If the return type is str, then it will be sent as the value for name. If the return type is a mapping type, then each key will be considered a name and will be sent with its (str) value.
For example, the following would collect an entity named square with a value of '25':
from datadog_checks.base import AgentCheck
class AwesomeCheck(AgentCheck):
METADATA_TRANSFORMERS = {
'square': lambda value, options: str(int(value) ** 2)
}
def check(self, instance):
self.set_metadata('square', '5')
There are a few default transformers, which can be overridden by custom transformers.
Source code in datadog_checks_base/datadog_checks/base/utils/metadata/core.py
class MetadataManager(object):
"""
Custom transformers may be defined via a class level attribute `METADATA_TRANSFORMERS`.
This is a mapping of metadata names to functions. When you call
`#!python self.set_metadata(name, value, **options)`, if `name` is in this mapping then
the corresponding function will be called with the `value`, and the return
value(s) will be collected instead.
Transformer functions must satisfy the following signature:
```python
def transform_<NAME>(value: Any, options: dict) -> Union[str, Dict[str, str]]:
```
If the return type is `str`, then it will be sent as the value for `name`. If the return type is a mapping type,
then each key will be considered a `name` and will be sent with its (`str`) value.
For example, the following would collect an entity named `square` with a value of `'25'`:
```python
from datadog_checks.base import AgentCheck
class AwesomeCheck(AgentCheck):
METADATA_TRANSFORMERS = {
'square': lambda value, options: str(int(value) ** 2)
}
def check(self, instance):
self.set_metadata('square', '5')
```
There are a few default transformers, which can be overridden by custom transformers.
"""
__slots__ = ('check_id', 'check_name', 'logger', 'metadata_transformers')
def __init__(self, check_name, check_id, logger=None, metadata_transformers=None):
self.check_name = check_name
self.check_id = check_id
self.logger = logger or LOGGER
self.metadata_transformers = {'version': self.transform_version}
if metadata_transformers:
self.metadata_transformers.update(metadata_transformers)
def submit_raw(self, name, value):
datadog_agent.set_check_metadata(self.check_id, to_native_string(name), to_native_string(value))
def submit(self, name, value, options):
transformer = self.metadata_transformers.get(name)
if transformer:
try:
transformed = transformer(value, options)
except Exception as e:
if is_primitive(value):
self.logger.debug('Unable to transform `%s` metadata value `%s`: %s', name, value, e)
else:
self.logger.debug('Unable to transform `%s` metadata: %s', name, e)
return
if isinstance(transformed, str):
self.submit_raw(name, transformed)
else:
for transformed_name, transformed_value in transformed.items():
self.submit_raw(transformed_name, transformed_value)
else:
self.submit_raw(name, value)
def transform_version(self, version, options):
"""
Transforms a version like `1.2.3-rc.4+5` to its constituent parts. In all cases,
the metadata names `version.raw` and `version.scheme` will be collected.
If a `scheme` is defined then it will be looked up from our known schemes. If no
scheme is defined then it will default to `semver`. The supported schemes are:
- `regex` - A `pattern` must also be defined. The pattern must be a `str` or a pre-compiled
`re.Pattern`. Any matching named subgroups will then be sent as `version.<GROUP_NAME>`. In this case,
the check name will be used as the value of `version.scheme` unless `final_scheme` is also set, which
will take precedence.
- `parts` - A `part_map` must also be defined. Each key in this mapping will be considered
a `name` and will be sent with its (`str`) value.
- `semver` - This is essentially the same as `regex` with the `pattern` set to the standard regular
expression for semantic versioning.
Taking the example above, calling `#!python self.set_metadata('version', '1.2.3-rc.4+5')` would produce:
| name | value |
| --- | --- |
| `version.raw` | `1.2.3-rc.4+5` |
| `version.scheme` | `semver` |
| `version.major` | `1` |
| `version.minor` | `2` |
| `version.patch` | `3` |
| `version.release` | `rc.4` |
| `version.build` | `5` |
"""
scheme, version_parts = parse_version(version, options)
if scheme == 'regex' or scheme == 'parts':
scheme = options.get('final_scheme', self.check_name)
data = {'version.{}'.format(part_name): part_value for part_name, part_value in version_parts.items()}
data['version.raw'] = version
data['version.scheme'] = scheme
return data
transform_version(version, options) ¶
Transforms a version like 1.2.3-rc.4+5 to its constituent parts. In all cases, the metadata names version.raw and version.scheme will be collected.
If a scheme is defined then it will be looked up from our known schemes. If no scheme is defined then it will default to semver. The supported schemes are:
regex- Apatternmust also be defined. The pattern must be astror a pre-compiledre.Pattern. Any matching named subgroups will then be sent asversion.<GROUP_NAME>. In this case, the check name will be used as the value ofversion.schemeunlessfinal_schemeis also set, which will take precedence.parts- Apart_mapmust also be defined. Each key in this mapping will be considered anameand will be sent with its (str) value.semver- This is essentially the same asregexwith thepatternset to the standard regular expression for semantic versioning.
Taking the example above, calling self.set_metadata('version', '1.2.3-rc.4+5') would produce:
| name | value |
|---|---|
version.raw | 1.2.3-rc.4+5 |
version.scheme | semver |
version.major | 1 |
version.minor | 2 |
version.patch | 3 |
version.release | rc.4 |
version.build | 5 |
Source code in datadog_checks_base/datadog_checks/base/utils/metadata/core.py
def transform_version(self, version, options):
"""
Transforms a version like `1.2.3-rc.4+5` to its constituent parts. In all cases,
the metadata names `version.raw` and `version.scheme` will be collected.
If a `scheme` is defined then it will be looked up from our known schemes. If no
scheme is defined then it will default to `semver`. The supported schemes are:
- `regex` - A `pattern` must also be defined. The pattern must be a `str` or a pre-compiled
`re.Pattern`. Any matching named subgroups will then be sent as `version.<GROUP_NAME>`. In this case,
the check name will be used as the value of `version.scheme` unless `final_scheme` is also set, which
will take precedence.
- `parts` - A `part_map` must also be defined. Each key in this mapping will be considered
a `name` and will be sent with its (`str`) value.
- `semver` - This is essentially the same as `regex` with the `pattern` set to the standard regular
expression for semantic versioning.
Taking the example above, calling `#!python self.set_metadata('version', '1.2.3-rc.4+5')` would produce:
| name | value |
| --- | --- |
| `version.raw` | `1.2.3-rc.4+5` |
| `version.scheme` | `semver` |
| `version.major` | `1` |
| `version.minor` | `2` |
| `version.patch` | `3` |
| `version.release` | `rc.4` |
| `version.build` | `5` |
"""
scheme, version_parts = parse_version(version, options)
if scheme == 'regex' or scheme == 'parts':
scheme = options.get('final_scheme', self.check_name)
data = {'version.{}'.format(part_name): part_value for part_name, part_value in version_parts.items()}
data['version.raw'] = version
data['version.scheme'] = scheme
return data