Struct AggregateConfiguration 

pub struct AggregateConfiguration {
    pub window_duration_seconds: NonZeroU64,
    pub primary_flush_interval: Duration,
    pub context_limit: usize,
    pub flush_open_windows: bool,
    pub counter_expiry_seconds: Option<u64>,
    pub passthrough_timestamped_metrics: bool,
    pub passthrough_idle_flush_timeout: Duration,
    pub hist_config: HistogramConfiguration,
    pub context_snapshot_receiver: AggregateContextSnapshotReceiver,
}

Aggregate transform.

Aggregates metrics into fixed-size windows, flushing them at a regular interval.

Zero-value counters

When metrics are aggregated and then flushed, they’re typically removed entirely from the aggregation state. Unless they’re updated again, they won’t be emitted again. However, for counters, a slightly different approach is taken by tracking “zero-value” counters.

Counters are aggregated and flushed normally. However, when flushed, counters are added to a list of “zero-value” counters, and if those counters aren’t updated again, the transform emits a copy of the counter with a value of zero. It does this until the counter is updated again, or the zero-value counter expires (no updates), whichever comes first.

This provides a continuity in the output of a counter, from the perspective of a downstream system, when counters are otherwise sparse. The expiration period is configurable, and allows a trade-off in how sparse/infrequent the updates to counters can be versus how long it takes for counters that don’t exist anymore to actually cease to be emitted.

Fields

window_duration_seconds: NonZeroU64

Size of the aggregation window, in seconds.

Metrics are aggregated into fixed-size windows, such that all updates to the same metric within a window are aggregated into a single metric. The window size controls how efficiently metrics are aggregated, and in turn, how many data points are emitted downstream.

primary_flush_interval: Duration

How often to flush buckets.

This represents a trade-off between the savings in network bandwidth (sending fewer requests to downstream systems, etc) and the frequency of updates (how often updates to a metric are emitted).

context_limit: usize

Maximum number of contexts to aggregate per window.

A context is the unique combination of a metric name and its set of tags. For example, metric.name.here{tag1=A,tag2=B} represents a single context, and would be different than metric.name.here{tag1=A,tag2=C}.

When the maximum number of contexts is reached in the current aggregation window, additional metrics are dropped until the next window starts.

flush_open_windows: bool

Whether to flush open buckets when stopping the transform.

Normally, open buckets (a bucket whose end hasn’t yet occurred) aren’t flushed when the transform is stopped. This is done to avoid the chance of flushing a partial window, restarting the process, and then flushing the same window again. Downstream systems sometimes can’t cope with this gracefully, as there is no way to determine that it’s an incremental update, and so they treat it as an absolute update, overwriting the previously flushed value.

In cases where flushing all outstanding data is paramount, this can be enabled.

counter_expiry_seconds: Option<u64>

How long to keep idle counters alive after they’ve been flushed, in seconds.

When metrics are flushed, they’re removed from the aggregation state. However, if a counter expiration is set, counters will be kept alive in an “idle” state. For as long as a counter is idle, but not yet expired, a zero value will be emitted for it during each flush. This allows more gracefully handling sparse counters, where updates are infrequent but leaving gaps in the time series would be undesirable from a user experience perspective.

After a counter has been idle (no updates) for longer than the expiry period, it will be completely removed and no further zero values will be emitted.

A value of 0, or None, disables idle counter keep-alive.

passthrough_timestamped_metrics: bool

Whether or not to immediately forward (passthrough) metrics with pre-defined timestamps.

When enabled, this causes the aggregator to immediately forward metrics that already have a timestamp present. Only metrics without a timestamp will be aggregated. This can be useful when metrics are already pre-aggregated client-side and both timeliness and memory efficiency are paramount, as it avoids the overhead of aggregating within the pipeline.

passthrough_idle_flush_timeout: Duration

How often to flush buffered passthrough metrics.

While passthrough metrics aren’t re-aggregated by the transform, they will still be temporarily buffered in order to optimize the efficiency of processing them in the next component. This setting controls the maximum amount of time that passthrough metrics will be buffered before being forwarded.

hist_config: HistogramConfiguration

Statistics to calculate over histograms, and how to copy them to distributions.

context_snapshot_receiver: AggregateContextSnapshotReceiver

Owner side of the channel used to serve retained-context snapshot requests.

This is runtime wiring rather than configuration: it carries no settings, and is created by the caller with aggregate_context_snapshot_channel so that the caller keeps the matching handle.

Trait Implementations

impl MemoryBounds for AggregateConfiguration

fn specify_bounds(&self, builder: &mut MemoryBoundsBuilder<'_>)

Specifies the minimum and firm memory bounds for this component and its subcomponents.

impl TransformBuilder for AggregateConfiguration

fn build<'life0, 'async_trait>( &'life0 self, context: ComponentContext, ) -> Pin<Box<dyn Future<Output = Result<Box<dyn Transform + Send>, GenericError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Builds an instance of the transform.

fn input_event_type(&self) -> EventType

Event type allowed as input events to this transform.

fn outputs(&self) -> &[OutputDefinition<EventType>]

Event outputs exposed by this transform.