Datastores

In order to reduce operational complexity, SpiceDB leverages existing, popular systems for persisting data.

Each datastore implementation comes with various trade-offs.

• CockroachDB - Recommended for planet-scale, multi-region deployments
• Cloud Spanner - Recommended for Google Cloud deployments
• PostgreSQL - Recommended for single-region deployments
• MySQL - Not recommended; only use if you cannot use PostgreSQL
• memdb - Recommended for local development and integration testing against applications

Migrations

Before a datastore can be used by SpiceDB or before running a new version of SpiceDB, you must execute all available migrations.

In order to migrate a datastore, run the following command with your desired values:

spicedb migrate head \
    --datastore-engine $DESIRED_ENGINE \
    --datastore-conn-uri $CONNECTION_STRING

For more information, refer to the Datastore Migration documentation.

CockroachDB

Usage Notes

• Recommended for multi-region deployments, with configurable region awareness
• Enables horizonally scalablity by adding more SpiceDB and CockroachDB instances
• Resiliency to individual CockroachDB instance failures
• Query and data balanced across the CockroachDB
• Setup and operational complexity of running CockroachDB

Developer Notes

• Code can be found here (opens in a new tab)
• Documentation can be found here (opens in a new tab)
• Implemented using pgx (opens in a new tab) for a SQL driver and connection pooling
• Has a native changefeed
• Stores migration revisions using the same strategy as Alembic (opens in a new tab)

Configuration

Required Parameters

Optional Parameters

Overlap Strategy

In distributed systems, you can trade-off consistency for performance.

CockroachDB datastore users that are willing to rely on more subtle guarantees to mitigate the New Enemy Problem can configure --datastore-tx-overlap-strategy.

The available strategies are:

For more information, refer to the CockroachDB datastore README (opens in a new tab) or our blog post "The One Crucial Difference Between Spanner and CockroachDB (opens in a new tab)".

Garbage Collection Window

SpiceDB warns if the garbage collection window as configured in CockroachDB is smaller than the SpiceDB configuration.

If you need a longer time window for the Watch API or querying at exact snapshots, you can adjust the value in CockroachDB (opens in a new tab):

ALTER ZONE default CONFIGURE ZONE USING gc.ttlseconds = 90000;

Cloud Spanner

Usage Notes

• Requires a Google Cloud Account with an active Cloud Spanner instance
• Take advantage of Google's TrueTime. The Spanner driver assumes the database is linearizable and skips the transaction overlap strategy required by CockroachDB.

Developer Notes

• Code can be found here (opens in a new tab)
• Documentation can be found here (opens in a new tab)
• Starts a background GC worker (opens in a new tab) to clean up old entries from the manually-generated changelog table

Configuration

• The Cloud Spanner docs (opens in a new tab) outline how to set up an instance
• Authentication via service accounts: The service account that runs migrations must have Cloud Spanner Database Admin; SpiceDB (non-migrations) must have Cloud Spanner Database User.

Required Parameters

Optional Parameters

PostgreSQL

Usage Notes

• Recommended for single-region deployments
• Postgres 15 or newer is required for optimal performance
• Resiliency to failures only when PostgreSQL is operating with a follower and proper failover
• Setup and operational complexity of running PostgreSQL
• Does not rely on any non-standard PostgreSQL extensions
• Compatible with managed PostgreSQL services (e.g. AWS RDS)

Developer Notes

• Code can be found here (opens in a new tab)
• Documentation can be found here (opens in a new tab)
• Implemented using pgx (opens in a new tab) for a SQL driver and connection pooling
• Stores migration revisions using the same strategy as Alembic (opens in a new tab)
• Implements its own MVCC (opens in a new tab) model by storing its data with transaction IDs

Configuration

Required Parameters

Optional Parameters

MySQL

Usage Notes

• Recommended for single-region deployments
• Setup and operational complexity of running MySQL
• Does not rely on any non-standard MySQL extensions
• Compatible with managed MySQL services

Developer Notes

• Code can be found here (opens in a new tab)
• Documentation can be found here (opens in a new tab)
• Implemented using Go-MySQL-Driver (opens in a new tab) for a SQL driver
• Query optimizations are documented here (opens in a new tab)
• Implements its own MVCC (opens in a new tab) model by storing its data with transaction IDs

Configuration

Required Parameters

Optional Parameters

memdb

Usage Notes

• Fully ephemeral; all data is lost when the process is terminated
• Intended for usage with SpiceDB itself and testing application integrations
• Cannot be ran highly-available as multiple instances will not share the same in-memory data

Developer Notes

• Code can be found here (opens in a new tab)
• Documentation can be found here (opens in a new tab)
• Implements its own MVCC (opens in a new tab) model by storing its data with transaction IDs

Configuration

Required Parameters

Optional Parameters