Struct MemoryLimiter 

pub struct MemoryLimiter { /* private fields */ }

A process-wide memory limiter.

In many cases, it can be useful to know when the process has exceeded a certain memory usage threshold in order to be able to react that: clearing caches, temporarily blocking requests, and so on.

MemoryLimiter watches the process’s physical memory usage (Resident Set Size/Working Set) and keeps track of when the usage exceeds the configured limit. Cooperating tasks call wait_for_capacity to participate in accepting backpressure that can be used to throttle work that is responsible for allocating memory.

The backpressure is scaled based on how close the memory usage is to the configured limit, with a minimum and maximum backoff duration that is applied. This means that until we are using a certain percentage of the configured limit, no backpressure is applied. Once that threshold is crossed, backpressure is applied proportionally to how close to the limit we are. Callers are never fully blocked even if the limit is reached.

Implementations

impl MemoryLimiter

pub fn new(grant: MemoryGrant) -> Option<Self>

Creates a new MemoryLimiter based on the given MemoryGrant.

A background task is spawned that frequently checks the used memory for the entire process, and callers of wait_for_capacity will observe waiting once the memory usage exceeds the configured limit threshold. The waiting time will scale progressively the closer the memory usage is to the configured limit.

Defaults to a 95% threshold (i.e. threshold begins at 95% of the limit), a minimum backoff duration of 1ms, and a maximum backoff duration of 25ms. The effective limit of the grant is used as the memory limit.

pub fn noop() -> Self

Creates a no-op MemoryLimiter that does not perform any limiting.

All calls to wait_for_capacity will return immediately.

pub async fn wait_for_capacity(&self)

Waits a short period of time based on the available memory.

If the used memory is not within the threshold of the configured limit, no waiting will occur. Otherwise, the call will wait a variable amount of time depending on how close to the configured limit the process is.

Trait Implementations

impl Clone for MemoryLimiter

fn clone(&self) -> MemoryLimiter

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.