Checkpointer

class beneath.Checkpointer

Checkpointers store (small) key-value records in a meta table (in the specified project). They are useful for maintaining consumer and pipeline state, such as the cursor for a subscription or the last time a scraper ran.

Checkpoint keys are strings and values are serialized with msgpack (supports most normal Python values, but not custom classes).

New checkpointed values are flushed at regular intervals (every 30 seconds by default). Checkpointers should not be used for storing large amounts of data. Checkpointers are not currently suitable for synchronizing parallel processes.

instance: beneath.instance.TableInstance

The meta-table instance that checkpoints are written to

async get(key: str, default: Optional[Any] = None) → Any

Gets a checkpointed value

async set(key: str, value: Any)

Sets a checkpoint value. Value will be encoded with msgpack.