Skip to main content

Lakebase architecture

Beta

As of June 15, Lakebase is available in Beta on GCP. See Region availability for supported regions.

Lakebase separates storage from compute. The Postgres engine that runs your queries is stateless, and your data lives in a durable storage layer that persists independently. This separation is what makes autoscaling, scale-to-zero, instant branches, read replicas, and fast failover possible.

To show what Lakebase changes, this page starts with the traditional, single-machine database design for contrast, then explains how Lakebase separates that same design into independent layers and what each piece does.

How a traditional database is built

Before looking at Lakebase, consider the model it replaces. A conventional Postgres database is a monolith. A single machine runs the query engine and writes both the write-ahead log (WAL) and the data files to a disk attached at a local mount point. Traditionally these disks were truly local, part of the same machine, but as infrastructure evolved they are often network-attached storage devices instead.

The WAL and the data files play two complementary roles:

  • The WAL speeds up writes. Postgres appends each change to the log sequentially before acknowledging a commit, which is fast and durable on a single disk.
  • The data files speed up reads. Postgres materializes the current version of every page into data files, so a query can read a row without replaying the log.

A traditional database monolith on a single machine, where the query engine writes to a write-ahead log and reads from data files on local disk.

Accessing all your data through one machine has drawbacks:

  • Durability is tied directly to that machine's physical infrastructure. You also have to pre-provision storage and predict how much your workload will grow, which complicates both cost management and resiliency planning.
  • High availability and many kinds of horizontal scaling require physical clones of the whole database.
  • If that machine fails, you can lose data. Techniques like RAID storage reduce this risk, but the added redundancy can significantly increase the cost of running the system.

The Lakebase architecture

Lakebase keeps the same responsibilities but separates them into two independent layers:

  • A compute layer that runs standard, stateless Postgres.
  • A storage layer made up of safekeepers, pageservers, and cloud object storage.

The two roles from the monolith map directly onto the new components. The WAL, which sped up writes, becomes the safekeepers, which scale writes. The data files, which sped up reads, become the pageservers, which scale reads.

The Lakebase architecture with a stateless Postgres compute layer above a storage layer of safekeepers, pageservers, and object storage.

Because data lives in cloud object storage rather than on a single machine, Lakebase delivers elastic, scalable compute and durable writes replicated across availability zones. There's no storage to provision: you pay only for the storage you consume, and you don't have to plan around failure modes like running out of disk.

This model also improves performance. Lakebase writes each change directly to multiple locations, so it avoids the overhead of traditional torn-write protection and block alignment. Because every write already goes to multiple locations, performance stays consistent whether or not high availability is enabled.

The following table maps each part of the monolith to its Lakebase counterpart.

Traditional monolith

Lakebase

Role

Single machine

Stateless compute

Runs the Postgres query engine

Local WAL disk

Safekeepers

Durably records every committed change

Local data files

Pageservers and object storage

Materializes and stores page versions

Traditional monolith

Lakebase

Role

Single machine

Stateless compute

Runs the Postgres query engine

Local WAL disk

Safekeepers

Durably records every committed change

Local data files

Pageservers and object storage

Materializes and stores page versions

Compute layer

The compute layer runs Postgres. It holds only transient state: the Postgres shared buffers in memory and a local compute cache backed by fast local disk. It owns no durable data.

Because compute owns no durable state:

  • It can be replaced, restarted, autoscaled, or scaled to zero without moving or losing data.
  • Instead of writing to a local filesystem, it streams the WAL to the storage layer.
  • Multiple compute instances can attach to the same storage layer, which is how Lakebase read replicas and fast failover work.

Storage layer

The storage layer is durable and operates independently of compute. It has three components.

Safekeepers

Safekeepers are the WAL, pulled out of the single machine and made highly available. As Postgres produces WAL records, it streams them to a group of safekeepers that replicate the log across a quorum using a Paxos-based consensus protocol.

A transaction commits when a quorum of safekeepers acknowledges the WAL record, not when a single machine finishes a local fsync. Durability comes from replication across nodes rather than from one disk.

Pageservers

Pageservers are the data files, pulled out and rebuilt from the WAL. A pageserver consumes the WAL stream from the safekeepers and materializes page versions on demand. When compute requests a page at a specific log sequence number (LSN), the pageserver reconstructs and returns it.

Pageservers act as a write-through cache above object storage. They asynchronously persist materialized pages to cloud object storage, and page reconstruction doesn't block a transaction commit.

Cloud object storage

Cloud object storage is the durability foundation for the entire storage layer. It holds the page data that pageservers persist.

On GCP, Lakebase persists data to Google Cloud Storage.

Object storage stays off the hot query path. Only pageservers read from it. For details on how storage redundancy works and why it is independent of the compute high availability setting, see Storage architecture.

How a write works

A write flows from compute through the storage layer:

  1. Postgres modifies the affected pages in memory and produces WAL records.
  2. Compute streams the WAL records to the safekeepers.
  3. When a quorum of safekeepers acknowledges the records, the transaction commits and the client receives success.
  4. Pageservers apply the WAL asynchronously and persist the updated pages to object storage.

The Lakebase write path, where Postgres streams WAL to safekeepers whose quorum acknowledgment commits the transaction before pageservers apply it.

A transaction is durable as soon as a quorum of safekeepers has the WAL record, because the log alone is enough to reconstruct the data. Pageservers rebuild and store the data pages afterward, off the commit path, so writes stay fast without putting any committed change at risk.

How a read works

Reads check a hierarchy of caches, from fastest to slowest, and stop at the first layer that has the page:

  1. Buffer pool (memory): The Postgres shared buffers in compute RAM.
  2. Local compute cache: A disk-backed cache on the compute node, sized relative to the compute's memory.
  3. Pageserver: On a cache miss, compute requests the page from a pageserver, which reconstructs it at the requested LSN.
  4. Object storage: The pageserver reads from object storage internally when needed. Queries don't reach object storage directly.

The Lakebase read cache hierarchy from fastest to slowest: buffer pool, local compute cache, pageserver, and object storage.

What this architecture enables

Separating stateless compute from durable storage is what makes several Lakebase features possible:

Feature

What it enables

Autoscaling

Because compute is stateless, Lakebase scales compute size up or down in response to workload without moving data.

Scale-to-zero

Compute can pause completely while storage persists, and data is immediately available when compute resumes.

Instant branches

Create an isolated, writable copy of your database in seconds. Because branching is a copy-on-write metadata operation against shared storage, it duplicates no data.

Read replicas

Multiple compute instances read from the same storage layer, so replicas need no data copies and start in seconds.

Point-in-time queries

Because the storage layer retains history, compute can attach to a past point in time and read the database as it existed then, without copying data back into place.

Fast failover

Failover promotes a secondary compute instance that attaches to the existing storage, with no data to move.

RPO = 0 (no committed data loss)

Lakebase durably records every committed transaction before acknowledging it, so you lose no committed data when compute fails, restarts, or scales to zero.

Feature

What it enables

Autoscaling

Because compute is stateless, Lakebase scales compute size up or down in response to workload without moving data.

Scale-to-zero

Compute can pause completely while storage persists, and data is immediately available when compute resumes.

Instant branches

Create an isolated, writable copy of your database in seconds. Because branching is a copy-on-write metadata operation against shared storage, it duplicates no data.

Read replicas

Multiple compute instances read from the same storage layer, so replicas need no data copies and start in seconds.

Point-in-time queries

Because the storage layer retains history, compute can attach to a past point in time and read the database as it existed then, without copying data back into place.

Fast failover

Failover promotes a secondary compute instance that attaches to the existing storage, with no data to move.

RPO = 0 (no committed data loss)

Lakebase durably records every committed transaction before acknowledging it, so you lose no committed data when compute fails, restarts, or scales to zero.

How this architecture supports LTAP

Because Lakebase durably stores every committed change in cloud object storage, the same data can serve analytical workloads alongside transactions without a separate replication pipeline. This is the foundation for Lake Transactional and Analytical Processing (LTAP), where a single copy of your data supports both transactional and analytical engines. To learn how LTAP builds on this architecture, see LTAP architecture.

Next steps

  • Storage architecture: Learn how storage redundancy works and why it is independent of the compute high availability setting. See Storage architecture.
  • Database branches: See how branches use copy-on-write storage to create instant, isolated environments. See Branches.
  • Read replicas: Add read-only compute instances that share the same storage layer. See Read replicas.
  • Core concepts: Review the full set of concepts that make Lakebase unique. See Core concepts.