Skip to content

Configuring DogStatsD on Agent Data Plane

The DogStatsD implementation on ADP has been redesigned in Rust for better resource guarantees and efficiency. Because the architecture is different from the original implementation, certain configuration values may behave differently, be planned but not yet implemented, or not apply at all. This page documents those nuances.

ADP is designed to be transparent: customers configure DogStatsD the same way they always have. The sections below call out the cases where that's either not yet true, or not quite possible.

If you find an error on this page, please open an issue.

Unsupported Settings

Being Worked On

The following settings aren't yet supported in ADP but are planned with GitHub issue links for tracking.

Config KeyDescriptionIssue
allow_arbitrary_tagsAllow arbitrary tag values#1377
cri_connection_timeoutCRI runtime connection timeout#1348
cri_query_timeoutCRI runtime query timeout#1348
dogstatsd_capture_depthTraffic capture channel depth#1381
dogstatsd_capture_pathTraffic capture file location#1381
dogstatsd_pipe_nameWindows named pipe path#1466
dogstatsd_windows_pipe_security_descriptorWindows named pipe ACL descriptor#1466
forwarder_http_protocolHTTP version (auto/http1)#1361
forwarder_outdated_file_in_daysRetry file retention (days)#1360
log_format_rfc3339Use RFC3339 timestamp format#1373
min_tls_versionMinimum TLS version for HTTPS#1370
serializer_experimental_use_v3_api.*V3 metrics API migration flags#1468
sslkeylogfileTLS key log file path#1372
statsd_forward_hostHost for packet forwarding#1476
statsd_forward_portPort for packet forwarding#1476
tls_handshake_timeoutHTTP TLS handshake timeout#178

Not Planned

The following settings exist in the core agent but aren't planned for ADP, typically because ADP's architecture is fundamentally different or the feature is platform-specific.

Config KeyDescriptionReason
dogstatsd_mem_based_rate_limiter.enabledEnable memory rate limiterGo GC specific; use memory_limit
dogstatsd_no_aggregation_pipeline_batch_sizeNo-agg pipeline batch sizeFixed in ADP topology
dogstatsd_packet_buffer_flush_timeoutPacket buffer flush timeoutADP decodes inline
dogstatsd_packet_buffer_sizeDatagrams per packet bufferADP decodes inline
dogstatsd_pipeline_autoadjustAuto-adjust pipeline workersADP uses async tasks
dogstatsd_pipeline_countParallel processing pipelinesADP uses async tasks
dogstatsd_queue_sizePacket channel buffer sizeADP uses async tasks
dogstatsd_telemetry_enabled_listener_idPer-listener telemetry taggingNot feasible to thread through
dogstatsd_workers_countNum DSD processing workersADP uses async tasks
use_dogstatsdMaster DogStatsD enable toggleCore Agent evaluates and sets data_plane.dogstatsd.enabled

Behavioral Differences

The following settings are recognized by both ADP and the core agent, but with different behavior or default values.

Config KeyDescriptionAgent BehaviorADP Behavior
dogstatsd_metrics_stats_enableEnable per-metric debug statsConfig toggleGates debug log; stats API on-demand (#1352, #1356)
dogstatsd_stats_enableEnable internal stats endpointConfig toggleOn-demand via API (#1352)
dogstatsd_stats_bufferInternal stats buffer sizeConfigurableOn-demand via API (#1352)
dogstatsd_stats_portInternal stats endpoint portConfigurable portOn-demand via API (#1352)
log_levelLog verbosity directivesControls Agent logsPlain levels control ADP/Saluki-owned targets only
logging_frequencyTransaction success log intervalThrottles success logsIntentionally unused
serializer_zstd_compressor_levelZstd compression levelDefault level 1Default level 3 (intentional)
skip_ssl_validationSkip TLS cert validationDisables validation for outbound HTTPS clientsApplies to the shared Datadog forwarder; rejected in FIPS mode
statsd_metric_namespace_blacklistPrefixes exempt from namespace_blacklist keyUse _blocklist key (#1353)
telemetry.enabledGlobal telemetry toggleAgent toggleUse data_plane.telemetry_enabled (#1338)

Datadog intake TLS validation (skip_ssl_validation)

ADP supports skip_ssl_validation for Datadog intake forwarding through the shared Datadog forwarder. The default is false, which preserves normal server certificate validation. To accept invalid server certificates for Datadog intake requests, set skip_ssl_validation: true or DD_SKIP_SSL_VALIDATION=true.

When enabled, this setting affects the Datadog intake clients used by metrics, logs, traces, events, and service checks that flow through the shared forwarder.

WARNING

Setting skip_ssl_validation: true disables TLS server certificate validation for Datadog intake forwarding. Use it only when you understand and accept that risk.

This setting doesn't affect ADP IPC, local privileged APIs, ADP control-plane clients, OTLP proxying to the core agent, or unrelated HTTP clients. In FIPS builds, ADP rejects skip_ssl_validation: true because disabling TLS certificate validation isn't FIPS-compliant.

Logging verbosity (log_level / logging_frequency)

ADP accepts log_level as the startup logging control. A plain level applies to ADP-owned and Saluki-owned targets only, including agent_data_plane, saluki_*, and runtime crates under lib/.

yaml
log_level: debug

This keeps third-party dependencies such as hyper, tokio, and tonic at their default filtering unless you opt them in.

To control dependency logs or set a global fallback, use advanced EnvFilter directives in log_level. ADP applies those directive strings as configured:

yaml
log_level: warn,agent_data_plane=debug,hyper=warn

logging_frequency is intentionally unused by ADP. The core agent uses it to throttle repetitive successful transaction logs. ADP logs successful forwarder operations below the default info level, so there is no matching info-level success-log stream to throttle.

DogStatsD statistics (dogstatsd_stats_enable / dogstatsd_metrics_stats_enable)

The core agent has two DogStatsD statistics mechanisms with different scopes. dogstatsd_stats_enable enables packet-level throughput statistics from a ring buffer, exposed as Go expvar data on dogstatsd_stats_port (default 5000). Operators must configure an OpenMetrics check to scrape that endpoint before the data is submitted. dogstatsd_metrics_stats_enable enables runtime-toggleable metric-level debug statistics that track count and last-seen time per unique metric and tag combination. That data powers the core agent's dogstatsd-stats CLI command and HTTP endpoint.

ADP doesn't mirror the packet-level statistics config path. Instead, ADP provides an on-demand metric-level view through a DogStatsD statistics destination that's always wired into the topology, but only collects data during a time-bounded request. To collect statistics, run agent-data-plane dogstatsd stats --duration-secs N or call the privileged /dogstatsd/stats?collection_duration_secs=N API. The handler waits for the requested collection window, then returns count and last-seen time per metric context inline as JSON. The CLI uses the same API and renders the result as either summary or cardinality analysis.

ADP also exposes internal DogStatsD telemetry through its OpenMetrics endpoint when data_plane.telemetry_enabled is enabled. Scrape data_plane.telemetry_listen_addr to collect aggregate DogStatsD counters such as processed message counts, packet and byte counts, packet pool usage, and channel latency. This telemetry endpoint is separate from /dogstatsd/stats: it doesn't return the per-metric count and last-seen map, and it's not controlled by the core agent's dogstatsd_stats_* keys.

ADP doesn't expose the core agent's packet-per-second expvar endpoint or a persistent per-metric DogStatsD statistics endpoint to scrape. You don't need to set up scraper configuration for this per-metric data. The config keys dogstatsd_stats_enable, dogstatsd_stats_buffer, and dogstatsd_stats_port have no effect in ADP. See #1352.

DogStatsD metric debug log

ADP supports the core agent's DogStatsD metric debug log. To write this file, set dogstatsd_metrics_stats_enable: true. dogstatsd_logging_enabled also must be true; it defaults to true, so most configurations only need to enable dogstatsd_metrics_stats_enable.

When dogstatsd_logging_enabled is true, ADP connects an extra DogStatsD destination to the decoded metric stream. The destination writes one line per metric sample with the metric name, tags, count, and last-seen time while dogstatsd_metrics_stats_enable is true. When dogstatsd_metrics_stats_enable is false, the destination drains decoded metrics and drops them. This lets runtime configuration changes start and stop the debug log without rebuilding the topology. This feature is for support and troubleshooting. It doesn't change normal metric forwarding, and it doesn't replace the on-demand /dogstatsd/stats API.

Use these settings to control the file:

Config KeyBehavior
dogstatsd_log_fileOutput path. If empty, ADP uses the platform default DogStatsD stats log path.
dogstatsd_log_file_max_rollsNumber of rotated files to keep. Defaults to 3.
dogstatsd_log_file_max_sizeMaximum active file size before rotation. Defaults to 10Mb.
dogstatsd_logging_enabledControls whether ADP wires the debug log destination into the topology. Defaults to true.

The default dogstatsd_log_file path is /var/log/datadog/dogstatsd_info/dogstatsd-stats.log on Linux and other Unix platforms, /opt/datadog-agent/logs/dogstatsd_info/dogstatsd-stats.log on macOS, and %ProgramData%\datadog\logs\dogstatsd_info\dogstatsd-stats.log on Windows.

This debug log differs from the dogstatsd_capture_* settings. The debug log records decoded metric summaries after DogStatsD parsing. The capture settings record raw DogStatsD traffic for packet-level investigation, and they remain tracked separately under #1381.

telemetry.enabled

The core agent enables internal Prometheus metrics via telemetry.enabled and ingests them with an OpenMetrics check pointed at the agent's own telemetry endpoint. ADP has a separate telemetry endpoint controlled by data_plane.telemetry_enabled. Customers enabling ADP telemetry must configure a separate OpenMetrics check pointed at ADP's endpoint. See #1338.

Compatibility Unknown

The following settings need further investigation. ADP behavior may differ from the core agent in ways that aren't yet fully characterized.

Config KeyDescriptionIssue
dogstatsd_disable_verbose_logsSuppress noisy parse error logs
forwarder_apikey_validation_intervalAPI key check interval (mins)
forwarder_flush_to_disk_mem_ratioMem-to-disk flush threshold
forwarder_high_prio_buffer_sizeHigh-priority request queue size
forwarder_low_prio_buffer_sizeLow-priority request queue size
forwarder_max_concurrent_requestsMax concurrent HTTP requests
forwarder_retry_queue_capacity_time_interval_secRetry queue time-based capacity
serializer_max_payload_sizeMax compressed payload size
serializer_max_series_payload_sizeMax series compressed size
serializer_max_series_points_per_payloadMax series points per payload
serializer_max_series_uncompressed_payload_sizeMax series uncompressed size
serializer_max_uncompressed_payload_sizeMax uncompressed payload size
statsd_metric_blocklistMetric name blocklist#1433
statsd_metric_blocklist_match_prefixBlocklist matches by prefix#1434
statsd_metric_namespacePrefix prepended to all metrics

ADP-Only Settings

The following settings are specific to ADP and have no equivalent in the core agent.

Config KeyDescriptionDefault
agent_ipc_endpointRemote agent IPC URI
aggregate_flush_intervalAggregator flush period
aggregate_flush_open_windowsFlush open windows on stop
aggregate_passthrough_idle_flush_timeoutPassthrough buffer flush delay
aggregate_window_durationAggregation window size
connect_retry_attemptsIPC client connect retries
connect_retry_backoffIPC client retry delay
counter_expiry_secondsIdle counter keep-alive duration300s
data_plane.api_listen_addressADP unprivileged API address
data_plane.remote_agent_enabledRegister as remote agent
data_plane.secure_api_listen_addressADP privileged API address
data_plane.standalone_modeADP standalone mode toggle
data_plane.use_new_config_stream_endpointUse new config stream endpoint
dogstatsd_allow_context_heap_allocsAllow heap allocs for contexts
dogstatsd_buffer_countNumber of receive buffers
dogstatsd_cached_contexts_limitMax cached metric contexts
dogstatsd_cached_tagsets_limitMax cached tagsets
dogstatsd_mapper_string_interner_sizeMapper string interner capacity
dogstatsd_minimum_sample_rateFloor for metric sample rates
dogstatsd_permissive_decodingRelaxes decoder strictnesstrue
dogstatsd_tcp_portTCP listen port for DSD
enable_global_limiterToggle global memory limiter
flush_timeout_secsEncoder flush timeout (secs)
memory_limitProcess memory limit (bytes)
memory_slop_factorMemory headroom fraction
otlp_string_interner_sizeOTLP context interner capacity
remote_agent_string_interner_size_bytesTag string interner capacity512 KB
serializer_max_metrics_per_payloadMax metrics per payload
statsd_metric_namespace_blocklistRenamed alias for blacklist key

memory_limit / memory_slop_factor

ADP uses an explicit process memory limit (memory_limit) rather than relying on Go's garbage collector. The memory_slop_factor reserves a fraction of the limit to account for allocations not tracked by ADP's internal accounting. When memory usage approaches memory_limit, ADP's global limiter begins exerting backpressure (see enable_global_limiter).

dogstatsd_minimum_sample_rate

ADP enforces a minimum sample rate on incoming metrics to prevent memory exhaustion from extremely low sample rates on histograms and sketches. Sending metrics with a very high inverse sample rate (for example, @0.0000001) can cause unbounded memory growth in a sketch; this setting prevents that. The default is conservative enough that normal clients are unaffected.

dogstatsd_permissive_decoding

By default, ADP parses DogStatsD packets with the same leniency as the core agent, accepting packets that technically violate the spec. Setting this to false enables strict mode, which rejects non-conformant packets. Strict mode isn't available in the core agent.

data_plane.remote_agent_enabled / data_plane.use_new_config_stream_endpoint

These two keys are transitional flags being phased out. Both will be implied by data_plane.standalone_mode=false in a future release. Don't rely on them for new deployments.

Transparent Settings

The following settings work in ADP with the same behavior as the core agent.

To enable syslog logging, set log_to_syslog: true. Console logging remains controlled by log_to_console; enabling syslog doesn't disable console or file logging. If syslog_uri is empty while syslog logging is enabled, ADP uses the platform default local syslog socket: unixgram:///dev/log on Linux and unixgram:///var/run/syslog on macOS. Set syslog_rfc: true when the receiving syslog daemon expects the Agent's RFC-style header.

Config KeyDescription
additional_endpointsDual-ship to extra endpoints
aggregate_context_limitMax contexts per agg window
api_keyAPI key for endpoint auth
auth_token_file_pathIPC auth token file path
bind_hostGlobal listen host fallback
cmd_portAgent IPC/CMD API port
container_cgroup_rootCgroup filesystem root path
container_proc_rootProcfs root path for containers
cri_socket_pathCRI/containerd socket path
data_plane.dogstatsd.enabledEnable DSD in data plane
data_plane.enabledEnable ADP globally
dd_urlOverride intake endpoint URL
dogstatsd_buffer_sizeReceive buffer size (bytes)
dogstatsd_context_expiry_secondsContext cache TTL (seconds)
dogstatsd_entity_id_precedenceEntity ID over auto-detection
dogstatsd_eol_requiredRequire newline-terminated messages
dogstatsd_expiry_secondsCounter zero-value TTL (secs)
dogstatsd_flush_incomplete_bucketsFlush open buckets on shutdown
dogstatsd_log_fileDSD metric debug log path
dogstatsd_log_file_max_rollsMax rotated DSD debug log files
dogstatsd_log_file_max_sizeMax DSD debug log file size
dogstatsd_logging_enabledEnable DSD metric debug logging
dogstatsd_mapper_profilesMetric mapping profile defs
dogstatsd_no_aggregation_pipelineEnable no-agg timestamped path
dogstatsd_non_local_trafficAccept non-localhost UDP/TCP
dogstatsd_origin_detectionEnable UDS origin detection
dogstatsd_origin_detection_clientHonor client origin proto fields
dogstatsd_origin_optout_enabledAllow clients to opt out origin
dogstatsd_portUDP listen port
dogstatsd_so_rcvbufSocket receive buffer size
dogstatsd_socketUDS datagram socket path
dogstatsd_stream_log_too_bigLog oversized UDS stream frames
dogstatsd_stream_socketUDS stream socket path
dogstatsd_string_interner_sizeString interner capacity
dogstatsd_tag_cardinalityDefault tag cardinality level
dogstatsd_tagsExtra tags added to all DSD data
enable_payloads.eventsAllow sending event payloads
enable_payloads.seriesAllow sending series payloads
enable_payloads.service_checksAllow sending svc check payloads
enable_payloads.sketchesAllow sending sketch payloads
expected_tags_durationHost tag enrichment duration
extra_tagsAdditional static tags
forwarder_backoff_baseRetry backoff base (secs)
forwarder_backoff_factorRetry backoff jitter factor
forwarder_backoff_maxRetry backoff ceiling (secs)
forwarder_connection_reset_intervalHTTP conn reset interval (secs)
forwarder_num_workersConcurrent forwarder workers
forwarder_recovery_intervalBackoff recovery decrease factor
forwarder_recovery_resetReset errors on success
forwarder_retry_queue_max_sizeRetry queue max size (depr.)
forwarder_retry_queue_payloads_max_sizeRetry queue max size (bytes)
forwarder_storage_max_disk_ratioMax disk usage ratio for retry
forwarder_storage_max_size_in_bytesMax on-disk retry storage size
forwarder_storage_pathOn-disk retry storage directory
forwarder_timeoutForwarder HTTP request timeout
histogram_aggregatesHistogram aggregate statistics
histogram_copy_to_distributionCopy histograms to distributions
histogram_copy_to_distribution_prefixPrefix for hist-to-dist copies
histogram_percentilesHistogram percentile quantiles
hostnameConfigured hostname override
ipc_cert_file_pathIPC TLS certificate path
log_fileLog output file path
log_file_max_rollsMax rotated log files kept
log_file_max_sizeMax log file size before rotate
log_format_jsonUse JSON log format
log_to_consoleLog to stdout/stderr
log_to_syslogLog to syslog daemon
metric_filterlistMetric name blocklist
metric_filterlist_match_prefixBlocklist uses prefix matching
metric_tag_filterlistPer-metric tag include/exclude
no_proxy_nonexact_matchDomain/CIDR no_proxy matching
observability_pipelines_worker.metrics.enabledRoute metrics to OPW instance
observability_pipelines_worker.metrics.urlOPW metrics intake URL
origin_detection_unifiedUnified origin detection mode
provider_kindProvider kind static tag
proxyHTTP/HTTPS proxy configuration
run_pathRuntime data directory path
secret_backend_commandSecret resolver executable path
secret_backend_timeoutSecret backend timeout (seconds)
serializer_compressor_kindPayload compression algorithm
siteDatadog site domain
syslog_rfcUse RFC-style syslog header
syslog_uriSyslog destination URI
tagsGlobal tags (DD_TAGS)
use_proxy_for_cloud_metadataProxy cloud metadata endpoints
vector.metrics.enabledRoute metrics to OPW (legacy alias)
vector.metrics.urlOPW metrics intake URL (legacy alias)