Struct RetryQueue 

pub struct RetryQueue<T> { /* private fields */ }

A queue for storing requests to be retried.

Implementations

impl<T> RetryQueue<T>

where T: Retryable,

pub fn new(queue_name: String, max_in_memory_bytes: u64) -> Self

Creates a new RetryQueue instance with the given name and maximum size.

The queue will only hold as many entries as can fit within the given maximum size. If the queue is full, the oldest entries will be removed (or potentially persisted to disk, see with_disk_persistence) to make room for new entries.

pub async fn with_disk_persistence( self, root_path: PathBuf, max_disk_size_bytes: u64, storage_max_disk_ratio: f64, disk_usage_retriever: Arc<dyn DiskUsageRetriever + Send + Sync>, ) -> Result<Self, GenericError>

Configures the queue to persist pending entries to disk.

Disk persistence is used as a fallback to in-memory storage when the queue is full. When attempting to add a new entry to the queue, and the queue cannot fit the entry in-memory, in-memory entries will be persisted to disk, oldest first.

When reading entries from the queue, in-memory entries are read first, followed by persisted entries. This provides priority to the most recent entries added to the queue, but allows for bursting over the configured in-memory size limit without having to immediately discard entries.

Files are stored in a subdirectory, with the same name as the given queue name, within the given root_path.

Errors

If there is an error initializing the disk persistence layer, an error is returned.

pub fn is_empty(&self) -> bool

Returns true if the queue is empty.

This includes both in-memory and persisted entries.

pub fn len(&self) -> usize

Returns the number of entries in the queue

This includes both in-memory and persisted entries.

pub async fn push(&mut self, entry: T) -> Result<PushResult, GenericError>

Enqueues an entry.

If the queue is full and the entry cannot be enqueue in-memory, and disk persistence is enabled, in-memory entries will be moved to disk (oldest first) until enough capacity is available to enqueue the new entry in-memory.

Errors

If the entry is too large to fit into the queue, or if there is an error when persisting entries to disk, an error is returned.

pub async fn pop(&mut self) -> Result<Option<T>, GenericError>

Consumes an entry.

In-memory entries are consumed first, followed by persisted entries if disk persistence is enabled.

If no entries are available, None is returned.

Errors

If there is an error when consuming an entry from disk, whether due to reading or deserializing the entry, an error is returned.

pub async fn flush(self) -> Result<PushResult, GenericError>

Flushes all entries, potentially persisting them to disk.

When disk persistence is configured, this will flush all in-memory entries to disk. Flushing to disk still obeys the the normal limiting behavior in terms of maximum on-disk size. When disk persistence is not enabled, all in-memory entries will be dropped.

Errors

If an error occurs while persisting an entry to disk, an error is returned.