--- introduction.mdx ---

Zero is a new kind of sync engine powered by queries.

Rather than syncing entire tables to the client, or using static rules to carefully specify what to sync, you just write queries directly in your client code. Queries can access the entire backend database.

Zero caches the data for queries locally on the device, and reuses that data automatically to answer future queries whenever possible.

For typical applications, the result is that almost all queries are answered locally, instantly. It feels like you have access to the entire backend database directly from the client in memory. Occasionally, when you do a more specific query, Zero falls back to the server. But this happens automatically without any extra work required.

Zero is made possible by a custom streaming query engine we built called [ZQL](reading-data), which uses [Incremental View Maintenance](https://www.vldb.org/pvldb/vol16/p1601-budiu.pdf) on both client and server to efficiently keep large, complex queries up to date.

## Status

Zero is in alpha. There are still some rough edges, and to run it, you need to [deploy it yourself](deployment) to AWS or similar.

Even so, Zero is already quite fun to work with. We are using it ourselves to build our very own [Linear-style bug tracker](https://bugs.rocicorp.dev/). We find that Zero is already much more productive than alternatives, even having to occasionally work around a missing feature.

If you are building a new web app that needs to be fast and reactive, and can do the deployment yourself, it's a great time to get started with Zero. We're working toward a [beta release and full production readiness](roadmap) this year.


--- quickstart.mdx ---

## Prerequisites

- Docker
- Node 20+

<Note type="note" slug="other-frameworks">
  This quickstart uses React, but we also have it available for SolidJS. See:
  [SolidJS](/docs/solidjs).
</Note>

## Run

In one terminal, install and start the database:

```bash
git clone https://github.com/rocicorp/hello-zero.git
cd hello-zero
npm install

npm run dev:db-up
```

In a second terminal, start `zero-cache`:

```bash
cd hello-zero
npm run dev:zero-cache
```

In a final terminal, start the UI:

```bash
cd hello-zero
npm run dev:ui
```

## Quick Overview

`hello-zero` is a demo app that allows querying over a small dataset of fake messages between early Zero users.

Here are some things to try:

- Press the **Add Messages** button to add messages to the UI. Any logged-in or anonymous users are allowed to add messages.
- Press the **Remove Messages** button to remove messages. Only logged-in users are allowed to remove messages. You can **hold shift** to bypass the UI warning and see that write access control is being enforced server-side – the UI flickers as the optimistic write happens instantly and is then reverted by the server. Press **login** to login as a random user, then the remove button will work.
- Open two different browsers and see how fast sync propagates changes.
- Add a filter using the **From** and **Contains** controls. Notice that filters are fully dynamic and synced.
- Edit a message by pressing the **pencil icon**. You can only edit messages from the user you’re logged in as. As before you can attempt to bypass by holding shift.
- Check out the SQL schema for this database in `seed.sql`.
- Login to the database with `psql postgresql://user:password@127.0.0.1:5430/postgres` (or any other pg viewer) and delete or alter a row. Observe that it deletes from UI automatically.

## Detailed Walkthrough

<Video
  src="https://d2nzlypqdo5xbb.cloudfront.net/video/detailed-walkthrough.mp4"
  alt="Detailed UI and code walkthrough"
/>

## Deployment

You can deploy Zero apps to most cloud providers that support Docker and Postgres. See [Deployment](/docs/deployment) for more information.


--- add-to-existing-project.mdx ---

Zero integrates easily into most JavaScript or TypeScript projects, whether
you're using React, Vue, Svelte, Solid, or vanilla JavaScript.

## Prerequisites

- A PostgreSQL database with Write-Ahead Logging (WAL) enabled. See [Connecting
  to Postgres](connecting-to-postgres) for setup instructions.
- If you are using [TypeScript](https://www.typescriptlang.org/) ensure that
  [`strictNullChecks`](https://www.typescriptlang.org/tsconfig/#strictNullChecks)
  is set to `true` in [tsconfig.json](https://www.typescriptlang.org/tsconfig/). If this is not set then the advanced types Zero uses do not work
  as expected.

## Installation

Install the Zero package:

```bash
npm install @rocicorp/zero
```

<details>
<summary id="not-npm">Not using npm?</summary>

Zero's server component depends on `@rocicorp/zero-sqlite3`, which contains a
binary that requires running a postinstall script. Most alternative package
managers (non-npm) disable these scripts by default for security reasons. Here's
how to enable installation for common alternatives:

### pnpm

For [pnpm](https://pnpm.io/), either:

- Run `pnpm approve-builds` to approve all build scripts, or
- Add the specific dependency to your `package.json`:

```json
"pnpm": {
  "onlyBuiltDependencies": [
    "@rocicorp/zero-sqlite3"
  ]
}
```

### Bun

For [Bun](https://bun.sh/), add the dependency to your trusted dependencies
list:

```json
"trustedDependencies": [
  "@rocicorp/zero-sqlite3"
],
```

</details>

## Environment Variables

Configure Zero by creating a `.env` file in your project root:

```bash
ZERO_UPSTREAM_DB="postgresql://user:password@127.0.0.1/postgres"
ZERO_REPLICA_FILE="/tmp/sync-replica.db"
```

Replace the placeholders with your database connection details. For more
options, see [configuration options](zero-cache-config).

## Starting the Server

Start the Zero server using the CLI:

```bash
npx zero-cache-dev
```

The server runs on port 4848 by default. To verify, open `http://localhost:4848`
in your browser. If everything is configured correctly, you'll see "OK".

<Note slug="schema-file-location" heading="Schema file location">
  By default, `zero-cache` looks for a `schema.ts` file in the root of your
  project. You can change this by setting the `ZERO_SCHEMA_PATH` environment
  variable or by providing a `-p --schema-path` flag.
</Note>

## Defining Your Schema

Define your data model schema as described in the [Zero schema
documentation](zero-schema).

Example:

```ts
// schema.ts
import {createSchema, table, string} from '@rocicorp/zero';

const message = table('message')
  .columns({
    id: string(),
    body: string(),
  })
  .primaryKey('id');

export const schema = createSchema({
  tables: [message],
});

export type Schema = typeof schema;
```

If you're using [Prisma](https://www.prisma.io/) or
[Drizzle](https://orm.drizzle.team/), you can convert their schemas to Zero
schemas using tools listed in the [community
section](./community#database-tools).

### Permissions

Update `schema.ts` to include permissions for your tables. For example, to allow
all users to read and write to the `message` table, add the following:

```ts
// schema.ts
import {ANYONE_CAN_DO_ANYTHING, definePermissions} from '@rocicorp/zero';

export const permissions = definePermissions<unknown, Schema>(schema, () => ({
  message: ANYONE_CAN_DO_ANYTHING,
}));
```

For more details, see [permissions](permissions).

## Creating a Zero Instance

To create a Zero client instance:

```js
import {Zero} from '@rocicorp/zero';

const z = new Zero({
  userID: 'anon',
  server: 'http://localhost:4848',
  schema,
});
```

In production, avoid hardcoding the server URL. Use environment variables like
`import.meta.env.VITE_PUBLIC_SERVER` or `process.env.NEXT_PUBLIC_SERVER`.

## Reading Data

To read data, use the `materialize` method on a `Query` from the `Zero`
instance. This creates a materialized view that listens for real-time updates to
the data:

```js
const view = z.query.message.materialize();
view.addListener(data => console.log('Data updated:', data));
```

When the view is no longer needed, ensure you clean up by destroying it:

```js
view.destroy();
```

For more details, see [Reading Data with ZQL](reading-data).

### React

React developers can use the `useZero` hook for seamless integration. See
[Integrations React](react) for more details.

### SolidJS

For SolidJS, use the `createZero` function instead of `new Zero`.

Refer to [Integrations SolidJS](solidjs) for additional information.

### Other Frameworks

For other frameworks, see the [UI frameworks](community#ui-frameworks)
documentation.

## Writing Data

Zero supports both simple and advanced data mutations. For basic use cases,
use the [CRUD mutator](writing-data):

```ts
z.mutate.message.insert({id: nanoid(), body: 'Hello World!'});
```

For more complex scenarios, such as custom business logic, use
[custom mutators](custom-mutators) to define tailored mutation behavior.

## Server-Side Rendering (SSR)

Zero does not yet support SSR. See [SSR](zql-on-the-server#ssr) for details on disabling SSR for your framework.

## Deployment

Ensure all `.env` variables are set in the production environment. For Zero
cache deployment, see [Deployment](deployment). For frontend deployment, consult
your framework's documentation.


--- samples.mdx ---

## zbugs

A complete Linear-style bug tracker. Not just a demo app, this is our actual live bug db. We use it every day and depend on it.

<a href="https://bugs.rocicorp.dev/">
  <img src="/images/samples/zbugs.png" alt="zbugs" style={{border: 'none', padding: 'none'}} />
</a>

**Demo:** https://bugs.rocicorp.dev/<br/>
**Stack:** Vite/Fastify/React/AWS<br/>
**Source:** https://github.com/rocicorp/mono/tree/main/apps/zbugs<br/>
**Features:** Instant reads and writes, realtime updates, Github auth, write permissions, read permissions, custom mutators, complex filters, unread indicators, basic text search, emojis, short numeric bug IDs, notifications, and more.

## ztunes

An ecommerce store built with Zero, TanStack, Drizzle, and PlanetScale for Postgres.

<a href="https://ztunes.rocicorp.dev">
  <img src="/images/samples/ztunes.png" alt="ztunes" style={{border: 'none', padding: 'none'}} />
</a>

**Demo:** https://ztunes.rocicorp.dev/<br/>
**Stack:** TanStack/Drizzle/Better Auth/Fly.io<br/>
**Source:** https://github.com/rocicorp/ztunes<br/>
**Features:** 88k artists, 200k albums, single-command dev, full drizzle integration, text search, read permissions, write permissions.

## hello-zero

Simple quickstart for Zero/React.

<ImageLightbox src="/images/samples/quickstart.png" alt="Quickstart" />

**Stack:** Vite/Hono/React<br/>
**Source:** https://github.com/rocicorp/hello-zero<br/>
**Docs:** [Quickstart](/docs/quickstart)<br/>
**Features:** Instant reads and writes, realtime updates.

## hello-zero-solid

Simple quickstart for Zero/SolidJS.

<ImageLightbox src="/images/samples/hello-zero-solid.png" alt="Quickstart" />

**Stack:** Vite/Hono/SolidJS<br/>
**Source:** https://github.com/rocicorp/hello-zero-solid
**Features:** Instant reads and writes, realtime updates, custom mutators.

## hello-zero-do

Shows how to use the Zero client from a Cloudflare Durable Objects.

<Video
  src="https://d2nzlypqdo5xbb.cloudfront.net/video/hello-zero-do.mp4"
  alt="hello-zero-do demo"
/>

This sample runs `zero-client` within a Durable Object and monitors changes to a Zero query. This can be used to do things like send notifications, update external services, etc.

**Stack:** Vite/Hono/React/Cloudflare Workers<br/>
**Source:** [https://github.com/rocicorp/hello-zero-do](https://github.com/rocicorp/hello-zero-do)


--- overview.mdx ---




--- connecting-to-postgres.mdx ---

In the future, Zero will work with many different backend databases. Today only Postgres is supported. Specifically, Zero requires Postgres v15.0 or higher, and support for [logical replication](https://www.postgresql.org/docs/current/logical-replication.html).

Here are some common Postgres options and what we know about their support level:

| Postgres                          | Support Status                                                                                                |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| AWS RDS                           | ✅ |
| AWS Aurora                        | ✅&nbsp;&nbsp;v15.6+ |
| Google Cloud SQL                  | ✅&nbsp;&nbsp;See [notes below](#google-cloud-sql) |
| [Fly.io](http://Fly.io) Postgres  | ✅&nbsp;&nbsp;See [notes below](#flyio) |
| Neon                              | ✅&nbsp;&nbsp;See [notes below](#neon) |
| Postgres.app                      | ✅ |
| postgres:16.2-alpine docker image | ✅ |
| Supabase                          | ✅&nbsp;&nbsp;See [notes below](#supabase) |
| PlanetScale for Postgres | 🤷‍♂️&nbsp;&nbsp;No [event triggers](#event-triggers), see [notes below](#planetscale-for-postgres) |
| Render                            | 🤷‍♂️&nbsp;&nbsp;No [event triggers](#event-triggers) |
| Heroku                            | 🤷‍♂️&nbsp;&nbsp;No [event triggers](#event-triggers) |

## Event Triggers

Zero uses Postgres “[Event Triggers](https://www.postgresql.org/docs/current/sql-createeventtrigger.html)” when possible to implement high-quality, efficient [schema migration](zero-schema/#migrations).

Some hosted Postgres providers don’t provide access to Event Triggers.

Zero still works out of the box with these providers, but for correctness, any schema change triggers a full reset of all server-side and client-side state. For small databases (< 10GB) this can be OK, but for bigger databases we recommend choosing a provider that grants access to Event Triggers.

## Configuration

The Postgres `wal_level` config parameter has to be set to `logical`. You can check what level your pg has with this command:

```bash
psql -c 'SHOW wal_level'
```

If it doesn’t output `logical` then you need to change the wal level. To do this, run:

```bash
psql -c "ALTER SYSTEM SET wal_level = 'logical';"
```

Then restart Postgres. On most pg systems you can do this like so:

```bash
data_dir=$(psql -t -A -c 'SHOW data_directory')
pg_ctl -D "$data_dir" restart
```

After your server restarts, show the `wal_level` again to ensure it has changed:

```bash
psql -c 'SHOW wal_level'
```

## Provider-Specific Notes

### Google Cloud SQL

To use Google Cloud SQL you must [manually create a `PUBLICATION`](/docs/postgres-support#limiting-replication)
and specify that publication name in the [App Publications](/docs/zero-cache-config#app-publications)
option when running `zero-cache`.

(Google Cloud SQL does not provide sufficient permissions for `zero-cache` to create its default publication.)

## Fly.io

Fly does not support TLS on their internal networks. If you run both `zero-cache` and Postgres on Fly, you need
to stop `zero-cache` from trying to use TLS to talk to Postgres. You can do this by adding the `sslmode=disable`
query parameter to your connection strings from `zero-cache`.

### Supabase

In order to connect to Supabase you must use the "Direct Connection" style connection string, not the pooler:

<ImageLightbox
  src="/images/connecting-to-postgres/direct.png"
  caption='Use the "Direct Connection" option to connect zero-cache to your Supabase database.'
/>

This is because Zero sets up a logical replication slot, which is only supported with a direct connection.

Additionally, you may need to assign an IPv4 address to your Supabase instance:

<ImageLightbox
  src="/images/connecting-to-postgres/ipv4.png"
  caption="Assign an IPv4 address if you have trouble connecting from residential internet."
/>

This will be required if you
cannot use IPv6 from wherever `zero-cache` is running. Most cloud providers
support IPv6, but some do not. For example, if you are running `zero-cache` in AWS, it is possible to use IPv6 but
difficult.

IPv4 addresses are only supported on the Pro plan and are an extra $4/month.

## PlanetScale for Postgres

PlanetScale doesn't support [event triggers](#event-triggers) yet, but they say this is something they are working on.

PlanetScale also doesn't support creating publications with the [`FOR ALL TABLES` clause](https://www.postgresql.org/docs/current/sql-createpublication.html). Zero typically uses this to create an initial default publication during setup. You can workaround this by [creating a publication](postgres-support#limiting-replication) explicitly listing the tables you want to replicate.

## Neon

Neon fully supports Zero, but you should be aware of how Neon's pricing model and Zero interact.

Because Zero keeps an open connection to Postgres to replicate changes, as long as zero-cache is running, Postgres will be running and you will be charged by Neon.

For production databases that have enough usage to always be running anyway, this is fine. But for smaller applications that would otherwise not always be running, this can create a surprisingly high bill. You may want to choose a provider that charge a flat monthly rate instead.

Also some users choose Neon because they hope to use branching for previews. Note that Zero doesn't support this usage model well yet, and if not done with care, Zero can end up keeping each Neon *preview* branch running too 😳.

We are actively working on better preview support.


--- postgres-support.mdx ---

Postgres has a massive feature set, of which Zero supports a growings subset.

## Object Names

- Table and column names must begin with a letter or underscore
  - This can be followed letters, numbers, underscores, and hyphens
  - Regex: `/^[A-Za-z_]+[A-Za-z0-9_-]*$/`
- The column name `_0_version` is reserved for internal use

## Object Types

- Tables are synced
- Views are not synced
- `identity` generated columns are synced
- All other generated columns are not synced
- Indexes aren’t _synced_ per-se but we do implicitly add indexes to the replica that match the upstream indexes. In the future this will be customizable.

## Column Types

<table>
  <thead>
    <tr>
      <th>Postgres Type</th>
      <th>Type&nbsp;to&nbsp;put&nbsp;in&nbsp;`schema.ts`</th>
      <th>Resulting&nbsp;JS/TS&nbsp;Type</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>All numeric types</td>
      <td><code>number</code></td>
      <td><code>number</code></td>
    </tr>
    <tr>
      <td><code>char</code>, <code>varchar</code>, <code>text</code>, <code>uuid</code></td>
      <td><code>string</code></td>
      <td><code>string</code></td>
    </tr>
    <tr>
      <td><code>bool</code></td>
      <td><code>boolean</code></td>
      <td><code>boolean</code></td>
    </tr>
    <tr>
      <td><code>date</code>, <code>timestamp</code>, <code>timestampz</code></td>
      <td><code>number</code></td>
      <td><code>number</code></td>
    </tr>
    <tr>
      <td><code>json</code>, <code>jsonb</code></td>
      <td><code>json</code></td>
      <td><code>JSONValue</code></td>
    </tr>
    <tr>
      <td><code>enum</code></td>
      <td><code>enumeration</code></td>
      <td><code>string</code></td>
    </tr>
    <tr>
      <td><code>T[]</code> <em>where <code>T</code> is a supported Postgres type (but please see ⚠️ below)</em></td>
      <td><code>json&lt;U[]&gt;</code> <em>where <code>U</code> is the schema.ts type for <code>T</code></em></td>
      <td><code>V[]</code> <em>where <code>V</code> is the JS/TS type for <code>T</code></em></td>
    </tr>
  </tbody>
</table>

<Note emoji="⚠️" type="note" heading="No ZQL operators for arrays yet">

Zero will sync arrays to the client, but there is no support for filtering or joining on array elements yet in ZQL.

</Note>



Other Postgres column types aren’t supported. They will be ignored when replicating (the synced data will be missing that column) and you will get a warning when `zero-cache` starts up.

If your schema has a pg type not listed here, you can support it in Zero by using a trigger to map it to some type that Zero can support. For example if you have a [GIS polygon type](https://www.postgresql.org/docs/current/datatype-geometric.html#DATATYPE-POLYGON) in the column `my_poly polygon`, you can use a trigger to map it to a `my_poly_json json` column. You could either use another trigger to map in the reverse direction to support changes for writes, or you could use a [custom mutator](./custom-mutators) to write to the polygon type directly on the server.

Let us know if the lack of a particular column type is hindering your use of Zero. It can likely be added.

## Column Defaults

Default values are allowed in the Postgres schema but there currently is no way to use them from a Zero app. The create mutation requires all columns to be specified, except when columns are nullable (in which case,they default to null). Since there is no way to leave non-nullable columns off the insert, there is no way for PG to apply the default. This is a known issue and will be fixed in the future.

## IDs

It is strongly recommended that primary keys be client-generated random strings like [uuid](https://www.npmjs.com/package/uuid), [ulid](https://www.npmjs.com/package/ulid), [nanoid](https://www.npmjs.com/package/nanoid), etc. This makes optimistic creation and updates much easier.

<Note type="note" slug="client-generated-ids" heading="Why are client-generated IDs better?">
  Imagine that the PK of your table is an auto-incrementing integer. If you optimistically create an entity of this type, you will have to give it some ID – the type will require it locally, but also if you want to optimistically create relationships to this row you’ll need an ID.

You could sync the highest value seen for that table, but there are race conditions and it is possible for that ID to be taken by the time the creation makes it to the server. Your database can resolve this and assign the next ID, but now the relationships you created optimistically will be against the wrong row. Blech.

GUIDs makes a lot more sense in synced applications.

If your table has a natural key you can use that and it has less problems. But there is still the chance for a conflict. Imagine you are modeling orgs and you choose domainName as the natural key. It is possible for a race to happen and when the creation gets to the server, somebody has already chosen that domain name. In that case, the best thing to do is reject the write and show the user an error.

</Note>

If you want to have a short auto-incrementing numeric ID for ux reasons (ie, a bug number), that is possible – See [Demo Video](https://discord.com/channels/830183651022471199/1288232858795769917/1298114323272568852)!

## Primary Keys

Each table synced with Zero must have either a primary key or at least one unique index.

This is needed so that Zero can identify rows during sync, to distinguish between an edit and a remove/add.

Multi-column primary and foreign keys are supported.

## Limiting Replication

You can use [Permissions](permissions) to limit tables and rows from replicating to Zero. In the near future you’ll also be able to use Permissions to limit individual columns.

Until then, a workaround is to use the Postgres [_publication_](https://www.postgresql.org/docs/current/sql-createpublication.html) feature to control the tables and columns that are replicated into `zero-cache`.

In your pg schema setup, create a Postgres `publication` with the tables and columns you want:

```sql
CREATE PUBLICATION zero_data FOR TABLE users (col1, col2, col3, ...), issues, comments;
```

Then, specify this publication in the [App Publications](/docs/zero-cache-config#app-publications) `zero-cache` option. (By default, Zero creates a publication that publishes the entire public schema.)

To limit what is synced from the `zero-cache` replica to actual clients (e.g., web browsers) you can use [read permissions](/docs/permissions#select-permissions).

## Schema changes

Most Postgres schema changes are supported as is.

Two cases require special handling:

### Adding columns

Adding a column with a non-constant `DEFAULT` value is not supported.

This includes any expression with parentheses, as well as the special functions `CURRENT_TIME`, `CURRENT_DATE`, and `CURRENT_TIMESTAMP`
(due to a [constraint of SQLite](https://www.sqlite.org/lang_altertable.html#altertabaddcol)).

However, the `DEFAULT` value of an _existing_ column can be changed to any value, including non-constant expressions. To achieve the desired column default:

- Add the column with no `DEFAULT` value
- Backfill the column with desired values
- Set the column's `DEFAULT` value

```sql
BEGIN;

ALTER TABLE foo ADD bar ...;  -- without a DEFAULT value
UPDATE foo SET bar = ...;
ALTER TABLE foo ALTER bar SET DEFAULT ...;

COMMIT;
```

### Changing publications

Postgres allows you to change published tables/columns with an `ALTER PUBLICATION` statement. Zero automatically adjusts the table schemas on the replica, but it does not receive the pre-existing data.

To stream the pre-existing data to Zero, make an innocuous `UPDATE` after adding the tables/columns to the publication:

```sql
BEGIN;

ALTER PUBLICATION zero_data ADD TABLE foo;

ALTER TABLE foo REPLICA IDENTITY FULL;
UPDATE foo SET id = id;  -- For some column "id" in "foo"
ALTER TABLE foo REPLICA IDENTITY DEFAULT;

COMMIT;
```

## Self-Referential Relationships

See [zero-schema](/docs/zero-schema#self-referential-relationships)


--- zero-schema.mdx ---

Zero applications have both a _database schema_ (the normal backend database schema that all web apps have) and a _Zero schema_. The purpose of the Zero schema is to:

1. Provide typesafety for ZQL queries
2. Define first-class relationships between tables
3. Define permissions for access control

<Note type="note" slug="generating" heading="You do not need to define the Zero schema by hand">
  [Community-contributed converters](./community#database-tools) exist for
  Prisma and Drizzle that generate the tables and relationships. It is good to
  know how the underlying Zero schemas work, however, for debugging and
  conceptual understanding.
</Note>

This page describes using the schema to define your tables, columns, and relationships.

## Defining the Zero Schema

The Zero schema is encoded in a TypeScript file that is conventionally called `schema.ts` file. For example, see [the schema file for`hello-zero`](https://github.com/rocicorp/hello-zero/blob/main/src/schema.ts).

## Table Schemas

Use the `table` function to define each table in your Zero schema:

```tsx
import {table, string, boolean} from '@rocicorp/zero';

const user = table('user')
  .columns({
    id: string(),
    name: string(),
    partner: boolean(),
  })
  .primaryKey('id');
```

Column types are defined with the `boolean()`, `number()`, `string()`, `json()`, and `enumeration()` helpers. See [Column Types](/docs/postgres-support#column-types) for how database types are mapped to these types.

<Note type="warning" slug="type-mismatches">
  Currently, if the database type doesn’t map correctly to the Zero type,
  replication will continue and succeed but the data won't match the TypeScript
  type. This is a bug – in the future, this will be an error. See
  https://bugs.rocicorp.dev/issue/3112.
</Note>

### Name Mapping

Use `from()` to map a TypeScript table or column name to a different database name:

```ts
const userPref = table('userPref')
  // Map TS "userPref" to DB name "user_pref"
  .from('user_pref')
  .columns({
    id: string(),
    // Map TS "orgID" to DB name "org_id"
    orgID: string().from('org_id'),
  });
```

### Multiple Schemas

You can also use `from()` to access other Postgres schemas:

```ts
// Sync the "event" table from the "analytics" schema.
const event = table('event').from('analytics.event');
```

### Optional Columns

Columns can be marked _optional_. This corresponds to the SQL concept `nullable`.

```tsx
const user = table('user')
  .columns({
    id: string(),
    name: string(),
    nickName: string().optional(),
  })
  .primaryKey('id');
```

An optional column can store a value of the specified type or `null` to mean _no value_.

<Note type="note" heading="Null and undefined">
    Note that `null` and `undefined` mean different things when working with Zero rows.

    - When reading, if a column is `optional`, Zero can return `null` for that field. `undefined` is not used at all when Reading from Zero.
    - When writing, you can specify `null` for an optional field to explicitly write `null` to the datastore, unsetting any previous value.
    - For `create` and `upsert` you can set optional fields to `undefined` (or leave the field off completely) to take the default value as specified by backend schema for that column. For `update` you can set any non-PK field to `undefined` to leave the previous value unmodified.

</Note>

### Enumerations

Use the `enumeration` helper to define a column that can only take on a specific set of values. This is most often used alongside an [`enum` Postgres column type](postgres-support#column-types).

```tsx
import {table, string, enumeration} from '@rocicorp/zero';

const user = table('user')
  .columns({
    id: string(),
    name: string(),
    mood: enumeration<'happy' | 'sad' | 'taco'>(),
  })
  .primaryKey('id');
```

### Custom JSON Types

Use the `json` helper to define a column that stores a JSON-compatible value:

```tsx
import {table, string, json} from '@rocicorp/zero';

const user = table('user')
  .columns({
    id: string(),
    name: string(),
    settings: json<{theme: 'light' | 'dark'}>(),
  })
  .primaryKey('id');
```

### Compound Primary Keys

Pass multiple columns to `primaryKey` to define a compound primary key:

```ts
const user = table('user')
  .columns({
    orgID: string(),
    userID: string(),
    name: string(),
  })
  .primaryKey('orgID', 'userID');
```

## Relationships

Use the `relationships` function to define relationships between tables. Use the `one` and `many` helpers to define singular and plural relationships, respectively:

```ts
const messageRelationships = relationships(message, ({one, many}) => ({
  sender: one({
    sourceField: ['senderID'],
    destField: ['id'],
    destSchema: user,
  }),
  replies: many({
    sourceField: ['id'],
    destSchema: message,
    destField: ['parentMessageID'],
  }),
}));
```

This creates "sender" and "replies" relationships that can later be queried with the [`related` ZQL clause](./reading-data#relationships):

```ts
const messagesWithSenderAndReplies = z.query.messages
  .related('sender')
  .related('replies');
```

This will return an object for each message row. Each message will have a `sender` field that is a single `User` object or `null`, and a `replies` field that is an array of `Message` objects.

### Many-to-Many Relationships

You can create many-to-many relationships by chaining the relationship definitions. Assuming `issue` and `label` tables, along with an `issueLabel` junction table, you can define a `labels` relationship like this:

```ts
const issueRelationships = relationships(issue, ({many}) => ({
  labels: many(
    {
      sourceField: ['id'],
      destSchema: issueLabel,
      destField: ['issueID'],
    },
    {
      sourceField: ['labelID'],
      destSchema: label,
      destField: ['id'],
    },
  ),
}));
```

<Note slug="chain-limit" heading="Only two levels of chaining are supported">
  See https://bugs.rocicorp.dev/issue/3454.
</Note>

### Compound Keys Relationships

Relationships can traverse compound keys. Imagine a `user` table with a compound primary key of `orgID` and `userID`, and a `message` table with a related `senderOrgID` and `senderUserID`. This can be represented in your schema with:

```ts
const messageRelationships = relationships(message, ({one}) => ({
  sender: one({
    sourceField: ['senderOrgID', 'senderUserID'],
    destSchema: user,
    destField: ['orgID', 'userID'],
  }),
}));
```

### Circular Relationships

Circular relationships are fully supported:

```tsx
const commentRelationships = relationships(comment, ({one}) => ({
  parent: one({
    sourceField: ['parentID'],
    destSchema: comment,
    destField: ['id'],
  }),
}));
```

## Database Schemas

Use `createSchema` to define the entire Zero schema:

```tsx
import {createSchema} from '@rocicorp/zero';

export const schema = createSchema(
  {
    tables: [user, medium, message],
    relationships: [
      userRelationships,
      mediumRelationships,
      messageRelationships,
    ],
  },
);
```

## Migrations

Zero uses TypeScript-style structural typing to detect schema changes and implement smooth migrations.

### How it Works

When the Zero client connects to `zero-cache` it sends a copy of the schema it was constructed with. `zero-cache` compares this schema to the one it has, and rejects the connection with a special error code if the schema is incompatible.

By default, The Zero client handles this error code by calling `location.reload()`. The intent is to to get a newer version of the app that has been updated to handle the new server schema.

<Note type="note" heading="Update Order">
It's important to update the database schema first, then the app. Otherwise a reload loop will occur.

If a reload loop does occur, Zero uses exponential backoff to avoid overloading the server.

</Note>

If you want to delay this reload, you can do so by providing the `onUpdateNeeded` constructor parameter:

```ts
const z = new Zero({
  onUpdateNeeded: updateReason => {
    if (reason.type === 'SchemaVersionNotSupported') {
      // Do something custom here, like show a banner.
      // When you're ready, call `location.reload()`.
    }
  },
});
```

If the schema changes while a client is running in a compatible way, `zero-cache` syncs the schema change to the client so that it's ready when the app reloads and gets new code that needs it. If the schema changes while a client is running in an incompatible way, `zero-cache` will close the client connection with the same error code as above.

### Schema Change Process

Like other database-backed applications, Zero schema migration generally follow an “expand/migrate/contract” pattern:

1. Implement and run an “expand” migration on the backend that is backwards compatible with existing schemas. Add new rows, tables, as well as any defaults and triggers needed for backwards compatibility.
2. Add any new permissions required for the new tables/columns by running [`zero-deploy-permissions`](/docs/permissions#permission-deployment).
3. Update and deploy the client app to use the new schema.
4. Optionally, after some grace period, implement and run a “contract” migration on the backend, deleting any obsolete rows/tables.

Steps 1-3 can generally be done as part of one deploy by your CI pipeline, but step 4 would be weeks later when most open clients have refreshed and gotten new code.

<Note type="warning" slug="postgres-special-cases">
  Certain schema changes require special handling in Postgres. See [Schema
  Changes](/docs/postgres-support#schema-changes) for details.
</Note>


--- reading-data.mdx ---

ZQL is Zero’s query language.

Inspired by SQL, ZQL is expressed in TypeScript with heavy use of the builder pattern. If you have used [Drizzle](https://orm.drizzle.team/) or [Kysely](https://kysely.dev/), ZQL will feel familiar.

ZQL queries are composed of one or more _clauses_ that are chained together into a _query_.

Unlike queries in classic databases, the result of a ZQL query is a _view_ that updates automatically and efficiently as the underlying data changes. You can call a query’s `materialize()` method to get a view, but more typically you run queries via some framework-specific bindings. For example see `useQuery` for [React](react) or [SolidJS](solidjs).

<Note type="note" emoji="🧑‍🏫" slug="immutability" heading="Data returned from ZQL should be considered immutable">
This means you should not modify the data directly. Instead, clone the data and modify the clone.

ZQL caches values and returns them multiple times. If you modify a value returned from ZQL, you will modify it everywhere it is used. This can lead to subtle bugs.

JavaScript and TypeScript lack true immutable types so we use `readonly` to help enforce it. But it's easy to cast away the `readonly` accidentally.

In the future, we'll [`freeze`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze) all returned data in `dev` mode to help prevent this.

</Note>

## Select

ZQL queries start by selecting a table. There is no way to select a subset of columns; ZQL queries always return the entire row (modulo column permissions).

```tsx
const z = new Zero(...);

// Returns a query that selects all rows and columns from the issue table.
z.query.issue;
```

This is a design tradeoff that allows Zero to better reuse the row locally for future queries. This also makes it easier to share types between different parts of the code.

## Ordering

You can sort query results by adding an `orderBy` clause:

```tsx
z.query.issue.orderBy('created', 'desc');
```

Multiple `orderBy` clauses can be present, in which case the data is sorted by those clauses in order:

```tsx
// Order by priority descending. For any rows with same priority,
// then order by created desc.
z.query.issue.orderBy('priority', 'desc').orderBy('created', 'desc');
```

All queries in ZQL have a default final order of their primary key. Assuming the `issue` table has a primary key on the `id` column, then:

```tsx
// Actually means: z.query.issue.orderBy('id', 'asc');
z.query.issue;

// Actually means: z.query.issue.orderBy('priority', 'desc').orderBy('id', 'asc');
z.query.issue.orderBy('priority', 'desc');
```

## Limit

You can limit the number of rows to return with `limit()`:

```tsx
z.query.issue.orderBy('created', 'desc').limit(100);
```

## Paging

You can start the results at or after a particular row with `start()`:

```tsx
let start: IssueRow | undefined;
while (true) {
  let q = z.query.issue.orderBy('created', 'desc').limit(100);
  if (start) {
    q = q.start(start);
  }
  const batch = await q.run();
  console.log('got batch', batch);

  if (batch.length < 100) {
    break;
  }
  start = batch[batch.length - 1];
}
```

By default `start()` is _exclusive_ - it returns rows starting **after** the supplied reference row. This is what you usually want for paging. If you want _inclusive_ results, you can do:

```tsx
z.query.issue.start(row, {inclusive: true});
```

## Getting a Single Result

If you want exactly zero or one results, use the `one()` clause. This causes ZQL to return `Row|undefined` rather than `Row[]`.

```tsx
const result = await z.query.issue.where('id', 42).one().run();
if (!result) {
  console.error('not found');
}
```

`one()` overrides any `limit()` clause that is also present.

## Relationships

You can query related rows using _relationships_ that are defined in your [Zero schema](/docs/zero-schema).

```tsx
// Get all issues and their related comments
z.query.issue.related('comments');
```

Relationships are returned as hierarchical data. In the above example, each row will have a `comments` field which is itself an array of the corresponding comments row.

You can fetch multiple relationships in a single query:

```tsx
z.query.issue.related('comments').related('reactions').related('assignees');
```

### Refining Relationships

By default all matching relationship rows are returned, but this can be refined. The `related` method accepts an optional second function which is itself a query.

```tsx
z.query.issue.related(
  'comments',
  // It is common to use the 'q' shorthand variable for this parameter,
  // but it is a _comment_ query in particular here, exactly as if you
  // had done z.query.comment.
  q => q.orderBy('modified', 'desc').limit(100).start(lastSeenComment),
);
```

This _relationship query_ can have all the same clauses that top-level queries can have.

<Note type="warning" heading="Order and limit not supported in junction relationships">
Using `orderBy` or `limit` in a relationship that goes through a junction table (i.e., a many-to-many relationship) is not currently supported and will throw a runtime error. See [bug 3527](https://bugs.rocicorp.dev/issue/3527).

You can sometimes work around this by making the junction relationship explicit, depending on your schema and usage.
</Note>

### Nested Relationships

You can nest relationships arbitrarily:

```tsx
// Get all issues, first 100 comments for each (ordered by modified,desc),
// and for each comment all of its reactions.
z.query.issue.related(
	'comments', q => q.orderBy('modified', 'desc').limit(100).related(
		'reactions')
	)
);
```

## Where

You can filter a query with `where()`:

```tsx
z.query.issue.where('priority', '=', 'high');
```

The first parameter is always a column name from the table being queried. Intellisense will offer available options (sourced from your [Zero Schema](/docs/zero-schema)).

### Comparison Operators

Where supports the following comparison operators:

| Operator                                 | Allowed Operand Types         | Description                                                              |
| ---------------------------------------- | ----------------------------- | ------------------------------------------------------------------------ |
| `=` , `!=`                               | boolean, number, string       | JS strict equal (===) semantics                                          |
| `<` , `<=`, `>`, `>=`                    | number                        | JS number compare semantics                                              |
| `LIKE`, `NOT LIKE`, `ILIKE`, `NOT ILIKE` | string                        | SQL-compatible `LIKE` / `ILIKE`                                          |
| `IN` , `NOT IN`                          | boolean, number, string       | RHS must be array. Returns true if rhs contains lhs by JS strict equals. |
| `IS` , `IS NOT`                          | boolean, number, string, null | Same as `=` but also works for `null`                                    |

TypeScript will restrict you from using operators with types that don’t make sense – you can’t use `>` with `boolean` for example.

<Note type="note" slug="adding-operators">
  If you don’t see the comparison operator you need, let us know, many are easy
  to add.
</Note>

### Equals is the Default Comparison Operator

Because comparing by `=` is so common, you can leave it out and `where` defaults to `=`.

```tsx
z.query.issue.where('priority', 'high');
```

### Comparing to `null`

As in SQL, ZQL’s `null` is not equal to itself (`null ≠ null`).

This is required to make join semantics work: if you’re joining `employee.orgID` on `org.id` you do **not** want an employee in no organization to match an org that hasn’t yet been assigned an ID.

When you purposely want to compare to `null` ZQL supports `IS` and `IS NOT` operators that work just like in SQL:

```tsx
// Find employees not in any org.
z.query.employee.where('orgID', 'IS', null);
```

TypeScript will prevent you from comparing to `null` with other operators.

### Compound Filters

The argument to `where` can also be a callback that returns a complex expression:

```tsx
// Get all issues that have priority 'critical' or else have both
// priority 'medium' and not more than 100 votes.
z.query.issue.where(({cmp, and, or, not}) =>
  or(
    cmp('priority', 'critical'),
    and(cmp('priority', 'medium'), not(cmp('numVotes', '>', 100))),
  ),
);
```

`cmp` is short for _compare_ and works the same as `where` at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below).

Note that chaining `where()` is also a one-level `and`:

```tsx
// Find issues with priority 3 or higher, owned by aa
z.query.issue.where('priority', '>=', 3).where('owner', 'aa');
```

### Relationship Filters

Your filter can also test properties of relationships. Currently the only supported test is existence:

```tsx
// Find all orgs that have at least one employee
z.query.organization.whereExists('employees');
```

The argument to `whereExists` is a relationship, so just like other relationships it can be refined with a query:

```tsx
// Find all orgs that have at least one cool employee
z.query.organization.whereExists('employees', q =>
  q.where('location', 'Hawaii'),
);
```

As with querying relationships, relationship filters can be arbitrarily nested:

```tsx
// Get all issues that have comments that have reactions
z.query.issue.whereExists('comments',
	q => q.whereExists('reactions'));
);
```

The `exists` helper is also provided which can be used with `and`, `or`, `cmp`, and `not` to build compound filters that check relationship existence:

```tsx
// Find issues that have at least one comment or are high priority
z.query.issue.where({cmp, or, exists} =>
  or(
    cmp('priority', 'high'),
    exists('comments'),
  ),
);
```

## Data Lifetime and Reuse

Zero reuses data synced from prior queries to answer new queries when possible. This is what enables instant UI transitions.

But what controls the lifetime of this client-side data? How can you know whether any partiular query will return instant results? How can you know whether those results will be up to date or stale?

The answer is that the data on the client is simply the union of rows returned from queries which are currently syncing. Once a row is no longer returned by any syncing query, it is removed from the client. Thus, there is never any stale data in Zero.

So when you are thinking about whether a query is going to return results instantly, you should think about _what other queries are syncing_, not about what data is local. Data exists locally if and only if there is a query syncing that returns that data.

<Note type="note" heading="Caches vs Replicas">
  This is why we often say that despite the name `zero-cache`, Zero is not technically a cache. It's a *replica*.

A cache has a random set of rows with a random set of versions. There is no expectation that the cache any particular rows, or that the rows' have matching versions. Rows are simply updated as they are fetched.

A replica by contrast is eagerly updated, whether or not any client has requested a row. A replica is always very close to up-to-date, and always self-consistent.

Zero is a _partial_ replica because it only replicates rows that are returned by syncing queries.

</Note>

## Query Lifecycle

<ImageLightbox
  src="/images/reading-data/query-lifecycle.png"
  caption="The lifecycle of a ZQL query."
/>

Queries can be either _active_ or _backgrounded_. An active query is one that is currently being used by the application. Backgrounded queries are not currently in use, but continue syncing in case they are needed again soon.

Active queries are created one of three ways:

1. The app calls `q.materialize()` to get a `View`.
2. The app uses a platform binding like React's `useQuery(q)`.
3. The app calls [`preload()`](#preloading) to sync larger queries without a view.

Active queries sync until they are _deactivated_. The way this happens depends on how the query was created:

1. For `materialize()` queries, the UI calls `destroy()` on the view.
2. For `useQuery()`, the UI unmounts the component (which calls `destroy()` under the covers).
3. For `preload()`, the UI calls `cleanup()` on the return value of `preload()`.

### Background Queries

By default a deactivated query stops syncing immediately.

But it's often useful to keep queries syncing beyond deactivation in case the UI needs the same or a similar query in the near future. This is accomplished with the `ttl` parameter:

```ts
const [user] = useQuery(z.query.user.where('id', userId), {ttl: '1d'});
```

The `ttl` parameter specifies how long the app developer wishes the query to run in the background. The following formats are allowed (where `%d` is a positive integer):

| Format    | Meaning                                                                              |
| --------- | ------------------------------------------------------------------------------------ |
| `none`    | No backgrounding. Query will immediately stop when deactivated. This is the default. |
| `%ds`     | Number of seconds.                                                                   |
| `%dm`     | Number of minutes.                                                                   |
| `%dh`     | Number of hours.                                                                     |
| `%dd`     | Number of days.                                                                      |
| `%dy`     | Number of years.                                                                     |
| `forever` | Query will never be stopped.                                                         |

If the UI re-requests a background query, it becomes an active query again. Since the query was syncing in the background, the very first synchronous result that the UI receives after reactivation will be up-to-date with the server (i.e., it will have `resultType` of `complete`).

Just like other types of queries, the data from background queries is available for use by new queries. A common pattern in to [preload](#preloading) a subset of most commonly needed data with `{ttl: 'forever'}` and then do more specific queries from the UI with, e.g., `{ttl: '1d'}`. Most often the preloaded data will be able to answer user queries, but if not, the new query will be answered by the server and backgrounded for a day in case the user revisits it.

### Client Capacity Management

Zero has a default soft limit of 20,000 rows on the client-side, or about 20MB of data assuming 1KB rows.

This limit can be increased with the [`--target-client-row-count`](./zero-cache-config#target-client-row-count) flag, but we do not recommend setting it higher than 100,000.

<Note slug="client-capacity" heading="Why does Zero store so little data client-side?">
Contrary to the design of other sync engines, we believe that storing tons of data client-side doesn't make sense. Here are some reasons why:

- Initial sync will be slow, slowing down initial app load.
- Because storage in browser tabs is unreliable, initial sync can occur surprisingly often.
- We want to answer queries _instantly_ as often as possible. This requires client-side data in memory on the main thread. If we have to page to disk, we may as well go to the network and reduce complexity.
- Even though Zero's queries are very efficient, they do still have some cost, expecially hydration. Massive client-side storage would result in hydrating tons of queries that are unlikely to be used every time the app starts.

Most importantly, no matter how much data you store on the client, there will be cases where you have to fallback to the server:

- Some users might have huge amounts of data.
- Some users might have tiny amounts of available client storage.
- You will likely want the app to start fast and sync in the background.

Because you have to be able to fallback to server the question becomes _what is the **right** amount of data to store on the client?_, not _how can I store the absolute max possible data on the client?_

The goal with Zero is to answer 99% of queries on the client from memory. The remaining 1% of queries can fallback gracefully to the server. 20,000 rows was chosen somewhat arbitrarily as a number of rows that was likely to be able to do this for many applications.

There is no hard limit at 20,000 or 100,000. Nothing terrible happens if you go above. The thing to keep in mind is that:

1. All those queries will revalidate everytime your app boots.
2. All data synced to the client is in memory in JS.

</Note>

Here is how this limit is managed:

1. Active queries are never destroyed, even if the limit is exceeded. Developers are expected to keep active queries well under the limit.
2. The `ttl` value counts from the moment a query deactivates. Backgrounded queries are destroyed immediately when the `ttl` is reached, even if the limit hasn't been reached.
3. If the client exceeds its limit, Zero will destroy backgrounded queries, least-recently-used first, until the store is under the limit again.

### Thinking in Queries

Although IVM is a very efficient way to keep queries up to date relative to re-running them, it isn't free. You still need to think about how many queries you are creating, how long they are kept alive, and how expensive they are.

This is why Zero defaults to _not_ backgrounding queries and doesn't try to aggressively fill its client datastore to capacity. You should put some thought into what queries you want to run in the background, and for how long.

Zero currently provides a few basic tools to understand the cost of your queries:

- The client logs a warning for slow query materializations. Look for `Slow query materialization` in your logs. The default threshold is `5s` (including network) but this is configurable with the `slowMaterializeThreshold` parameter.
- The client logs the materialization time of all queries at the `debug` level. Look for `Materialized query` in your logs.
- The server logs a warning for slow query materializations. Look for `Slow query materialization` in your logs. The default threshold is `5s` but this is configurable with the `log-slow-materialize-threshold` configuration parameter.

We will be adding more tools over time.

## Completeness

Zero returns whatever data it has on the client immediately for a query, then falls back to the server for any missing data. Sometimes it's useful to know the difference between these two types of results. To do so, use the `result` from `useQuery`:

```tsx
const [issues, issuesResult] = useQuery(z.query.issue);
if (issuesResult.type === 'complete') {
  console.log('All data is present');
} else {
  console.log('Some data is missing');
}
```

The possible values of `result.type` are currently `complete` and `unknown`.

The `complete` value is currently only returned when Zero has received the server result. But in the future, Zero will be able to return this result type when it _knows_ that all possible data for this query is already available locally. Additionally, we plan to add a `prefix` result for when the data is known to be a prefix of the complete result. See [Consistency](#consistency) for more information.

## Handling Missing Data

It is inevetable that there will be cases where the requested data cannot be found. Because Zero returns local results immediately, and server results asynchronously, displaying "not found" / 404 UI can be slightly tricky. If you just use a simple existence check, you will often see the 404 UI flicker while the server result loads:

```tsx
const [issue, issuesResult] = useQuery(
  z.query.issue.where('id', 'some-id').one(),
);

// ❌ This causes flickering of the UI
if (!issue) {
  return <div>404 Not Found</div>;
} else {
  return <div>{issue}</div>;
}
```

The way to do this correctly is to only display the "not found" UI when the result type is `complete`. This way the 404 page is slow but pages with data are still just as fast.

```tsx
const [issue, issuesResult] = useQuery(
  z.query.issue.where('id', 'some-id').one(),
);

if (!issue && issueResult.type === 'complete') {
  return <div>404 Not Found</div>;
}

if (!issue) {
  return null;
}

return <div>{issue}</div>;
```

## Listening to Changes

Currently, the way to listen for changes in query results is not ideal. You can add a listener to a materialized view which has the new data and result as parameters:

```ts
z.query.issue.materialize().addListener((issues, issuesResult) => {
  // do stuff...
});
```

However, using this method will maintain its own materialized view in memory which is wasteful. It also doesn't allow for granular listening to events like `add` and `remove` of rows.

A better way would be to create your own view without actually storing the data which will also allow you to listen to specific events. Again, the API is not good and will be improved in the future.

```ts
// Inside the View class
// Instead of storing the change, we invoke some callback
push(change: Change): void {
  switch (change.type) {
    case 'add':
      this.#onAdd?.(change)
      break
    case 'remove':
      this.#onRemove?.(change)
      break
    case 'edit':
      this.#onEdit?.(change)
      break
    case 'child':
      this.#onChild?.(change)
      break
    default:
      throw new Error(`Unknown change type: ${change['type']}`)
  }
}
```

(see View implementations in [`zero-vue`](https://github.com/danielroe/zero-vue/blob/f25808d4b7d1ef0b8e01a5670d7e3050d6a64bbf/src/view.ts#L77-L89) or [`zero-solid`](https://github.com/rocicorp/mono/blob/51995101d0657519207f1c4695a8765b9016e07c/packages/zero-solid/src/solid-view.ts#L119-L131))

## Preloading

Almost all Zero apps will want to preload some data in order to maximize the feel of instantaneous UI transitions.

In Zero, preloading is done via queries – the same queries you use in the UI and for auth.

However, because preload queries are usually much larger than a screenful of UI, Zero provides a special `preload()` helper to avoid the overhead of materializing the result into JS objects:

```tsx
// Preload the first 1k issues + their creator, assignee, labels, and
// the view state for the active user.
//
// There's no need to render this data, so we don't use `useQuery()`:
// this avoids the overhead of pulling all this data into JS objects.
z.query.issue
  .related('creator')
  .related('assignee')
  .related('labels')
  .related('viewState', q => q.where('userID', z.userID).one())
  .orderBy('created', 'desc')
  .limit(1000)
  .preload();
```

## Running Queries Once

Usually subscribing to a query is what you want in a reactive UI, but every so often you'll need to run a query just once. To do this, use the `run()` method:

```tsx
const results = await z.query.issue.where('foo', 'bar').run();
```

By default, `run()` only returns results that are currently available on the client. That is, it returns the data that would be given for [`result.type === 'unknown'`](#completeness).

If you want to wait for the server to return results, pass `{type: 'complete'}` to `run`:

```tsx
const results = await z.query.issue.where('foo', 'bar').run({type: 'complete'});
```

<Note slug="default-run">
As a convenience you can also directly await queries:

```ts
await z.query.issue.where('foo', 'bar');
```

This is the same as saying `run()` or `run({type: 'unknown'})`.

</Note>

## Consistency

Zero always syncs a consistent partial replica of the backend database to the client. This avoids many common consistency issues that come up in classic web applications. But there are still some consistency issues to be aware of when using Zero.

For example, imagine that you have a bug database w/ 10k issues. You preload the first 1k issues sorted by created.

The user then does a query of issues assigned to themselves, sorted by created. Among the 1k issues that were preloaded imagine 100 are found that match the query. Since the data we preloaded is in the same order as this query, we are guaranteed that any local results found will be a _prefix_ of the server results.

The UX that result is nice: the user will see initial results to the query instantly. If more results are found server-side, those results are guaranteed to sort below the local results. There's no shuffling of results when the server response comes in.

Now imagine that the user switches the sort to ‘sort by modified’. This new query will run locally, and will again find some local matches. But it is now unlikely that the local results found are a prefix of the server results. When the server result comes in, the user will probably see the results shuffle around.

To avoid this annoying effect, what you should do in this example is also preload the first 1k issues sorted by modified desc. In general for any query shape you intend to do, you should preload the first `n` results for that query shape with no filters, in each sort you intend to use.

<Note slug="no-duplicate-rows" heading="Zero does not sync duplicate rows">
Zero syncs the *union* of all active queries' results. You don't have to worry about syncing many sorts of the same query when it's likely the results will overlap heavily.

</Note>

In the future, we will be implementing a consistency model that fixes these issues automatically. We will prevent Zero from returning local data when that data is not known to be a prefix of the server result. Once the consistency model is implemented, preloading can be thought of as purely a performance thing, and not required to avoid unsightly flickering.


--- writing-data.mdx ---

Zero generates basic CRUD mutators for every table you sync. Mutators are available at `zero.mutate.<tablename>`:

```tsx
const z = new Zero(...);
z.mutate.user.insert({
	id: nanoid(),
	username: 'abby',
	language: 'en-us',
});
```

<Note slug="moar-power" heading="Need more power?">
  To build mutators with more complex logic or server-specific behavior, see the
  new [Custom Mutators API](./custom-mutators).
</Note>

## Insert

Create new records with `insert`:

```tsx
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: 'js',
});
```

Optional fields can be set to `null` to explicitly set the new field to `null`. They can also be set to `undefined` to take the default value (which is often `null` but can also be some generated value server-side).

```tsx
// schema.ts
import {createTableSchema} from '@rocicorp/zero';

const userSchema = createTableSchema({
  tableName: 'user',
  columns: {
    id: {type: 'string'},
    name: {type: 'string'},
    language: {type: 'string', optional: true},
  },
  primaryKey: ['id'],
  relationships: {},
});

// app.tsx

// Sets language to `null` specifically
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: null,
});

// Sets language to the default server-side value. Could be null, or some
// generated or constant default value too.
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
});

// Same as above
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: undefined,
});
```

## Upsert

Create new records or update existing ones with `upsert`:

```tsx
z.mutate.user.upsert({
  id: samID,
  username: 'sam',
  language: 'ts',
});
```

`upsert` supports the same `null` / `undefined` semantics for optional fields that `insert` does (see above).

## Update

Update an existing record. Does nothing if the specified record (by PK) does not exist.

You can pass a partial, leaving fields out that you don’t want to change. For example here we leave the username the same:

```tsx
// Leaves username field to previous value.
z.mutate.user.update({
  id: samID,
  language: 'golang',
});

// Same as above
z.mutate.user.update({
  id: samID,
  username: undefined,
  language: 'haskell',
});

// Reset language field to `null`
z.mutate.user.update({
  id: samID,
  language: null,
});
```

## Delete

Delete an existing record. Does nothing if specified record does not exist.

```tsx
z.mutate.user.delete({
  id: samID,
});
```

<Note type="note" heading="Cache Persistence" slug="cache-persistence">
If you want to remove cached data from the browser you can call the `dropAllDatabases` function provided by Zero.

You can read more about it in [Authentication](/docs/auth#data-storage).

</Note>

## Batch Mutate

You can do multiple CRUD mutates in a single _batch_. If any of the mutations fails, all will. They also all appear together atomically in a single transaction to other clients.

```tsx
z.mutateBatch(async tx => {
  const samID = nanoid();
  tx.user.insert({
    id: samID,
    username: 'sam',
  });

  const langID = nanoid();
  tx.language.insert({
    id: langID,
    userID: samID,
    name: 'js',
  });
});
```


--- custom-mutators.mdx ---

_Custom Mutators_ are a new way to write data in Zero that is much more powerful than the original ["CRUD" mutator API](./writing-data).

Instead of having only the few built-in `insert`/`update`/`delete` write operations for each table, custom mutators allow you to _create your own write operations_ using arbitrary code. This makes it possible to do things that are impossible or awkward with other sync engines.

For example, you can create custom mutators that:

- Perform arbitrary server-side validation
- Enforce fine-grained permissions
- Send email notifications
- Query LLMs
- Use Yjs for collaborative editing
- … and much, _much_ more – custom mutators are just code, and they can do anything code can do!

Despite their increased power, custom mutators still participate fully in sync. They execute instantly on the local device, immediately updating all active queries. They are then synced in the background to the server and to other clients.

<Note
  slug="deprecating-crud"
  heading="Custom mutators will eventually become Zero's only write API"
>
  We're still refining the design of custom mutators. During this phase, the old
  CRUD mutators will continue to work. But we do want to deprecate CRUD
  mutators, and eventually remove them. So please try out custom mutators and
  [let us know](https://discord.rocicorp.dev/) how they work for you, and what
  improvements you need before the cutover.
</Note>

## Understanding Custom Mutators

### Architecture

Custom mutators introduce a new _server_ component to the Zero architecture.

![diagram](/images/custom-mutators/overview.png)

This server is implemented by you, the developer. It's typically just your existing backend, where you already put auth or other server-side functionality.

The server can be a serverless function, a microservice, or a full stateful server. The only real requirement is that it expose a special _push endpoint_ that `zero-cache` can call to process mutations. This endpoint implements the [push protocol](#custom-push-implementation) and contains your custom logic for each mutation.

Zero provides utilities in `@rocicorp/zero` that make it really easy implement this endpoint in TypeScript. But you can also implement it yourself if you want. As long as your endpoint fulfills the push protocol, `zero-cache` doesn't care. You can even write it in a different programming language.

### What Even is a Mutator?

Zero's custom mutators are based on [_server reconciliation_](https://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html) – a technique for robust sync that has been used by the video game industry for decades.

<Note emoji="🌈" heading="The more you know">
  Our previous sync engine, [Replicache](https://replicache.dev/), also used
  server reconciliation. The ability to implement arbitrary mutators was one of
  Replicache's most popular features. Custom mutators bring this same power to
  Zero, but with a much better developer experience.
</Note>

A custom mutator is just a function that runs within a database transaction, and which can read and write to the database. Here's an example of a very simple custom mutator written in TypeScript:

```ts
async function updateIssue(
  tx: Transaction,
  {id, title}: {id: string; title: string},
) {
  // Validate title length.
  if (title.length > 100) {
    throw new Error(`Title is too long`);
  }

  await tx.mutate.issue.update({id, title});
}
```

Each custom mutator gets **two implementations**: one on the client and one on the server.

The client implementation must be written in TypeScript against the Zero `Transaction` interface, using [ZQL](#read-data-on-the-client) for reads and a [CRUD-style API](#write-data-on-the-client) for writes.

The server implementation runs on your server, in your push endpoint, against your database. In principle, it can be written in any language and use any data access library. For example you could have the following Go-based server implementation of the same mutator:

```go
func updateIssueOnServer(tx *sql.Tx, id string, title string) error {
  // Validate title length.
  if len(title) > 100 {
    return errors.New("Title is too long")
  }

  _, err := tx.Exec("UPDATE issue SET title = $1 WHERE id = $2", title, id)
  return err
}
```

In practice however, most Zero apps use TypeScript on the server. For these users we provide a handy `ServerTransaction` that implements ZQL against Postgres, so that you can share code between client and server mutators naturally.

So on a TypeScript server, that server mutator can just be:

```ts
async function updateIssueOnServer(
  tx: ServerTransaction,
  {id, title},
  {id: string, title: string},
) {
  // Delegate to client mutator.
  // The `ServerTransaction` here has a different implementation
  // that runs the same ZQL queries against Postgres!
  await updateIssue(tx, {id, title});
}
```

<Note slug="code-sharing" heading="Code sharing in mutators is optional">
Even in TypeScript, you can do as little or as much code sharing as you like. In your server mutator, you can [use raw SQL](#dropping-down-to-raw-sql), any data access libraries you prefer, or add as much extra server-specific logic as you need.

Reusing ZQL on the server is a handy – and we expect frequently used – option, but not a requirement.

</Note>

### Server Authority

You may be wondering what happens if the client and server mutators implementations don't match.

Zero is an example of a _server-authoritative_ sync engine. This means that the server mutator always takes precedence over the client mutator. The result from the client mutator is considered _speculative_ and is discarded as soon as the result from the server mutator is known. This is a very useful feature: it enables server-side validation, permissions, and other server-specific logic.

Imagine that you wanted to use an LLM to detect whether an issue update is spammy, rather than a simple length check. We can just add that to our server mutator:

```ts
async function updateIssueOnServer(
  tx: ServerTransaction,
  {id, title}: {id: string; title: string},
) {
  const response = await llamaSession.prompt(
    `Is this title update likely spam?\n\n${title}\n\nResponse "yes" or "no"`,
  );
  if (/yes/i.test(response)) {
    throw new Error(`Title is likely spam`);
  }

  // delegate rest of implementation to client mutator
  await updateIssue(tx, {id, title});
}
```

If the server detects that the mutation is spammy, the client will see the error message and the mutation will be rolled back. If the server mutator succeeds, the client mutator will be rolled back and the server result will be applied.

### Life of a Mutation

Now that we understand what client and server mutations are, let's walk through how they work together with Zero to sync changes from a source client to the server and then other clients:

1. When you call a custom mutator on the client, Zero runs your client-side mutator immediately on the local device, updating all active queries instantly.
2. In the background, Zero then sends a _mutation_ (a record of the mutator having run with certain arguments) to your server's push endpoint.
3. Your push endpoint runs the [push protocol](#custom-push-implementation), executing the server-side mutator in a transaction against your database and recording the fact that the mutation ran. Optionally, you use our `PushProcessor` class to handle this for you, but you can also implement it yourself.
4. The changes to the database are replicated to `zero-cache` as normal.
5. `zero-cache` calculates the updates to active queries and sends rows that have changed to each client. It also sends information about the mutations that have been applied to the database.
6. Clients receive row updates and apply them to their local cache. Any pending mutations which have been applied to the server have their local effects rolled back.
7. Client-side queries are updated and the user sees the changes.

## Using Custom Mutators

### Registering Client Mutators

By convention, the client mutators are defined with a function called `createMutators` in a file called `mutators.ts`:

```ts
// mutators.ts
import {CustomMutatorDefs} from '@rocicorp/zero';
import {schema} from './schema';

export function createMutators() {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Validate title length. Legacy issues are exempt.
        if (title.length > 100) {
          throw new Error(`Title is too long`);
        }
        await tx.mutate.issue.update({id, title});
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

The `mutators.ts` convention allows mutator implementations to be easily [reused server-side](#setting-up-the-server). The `createMutators` function convention is used so that we can pass authentication information in to [implement permissions](#permissions).

You are free to make different code layout choices – the only real requirement is that you register your map of mutators in the `Zero` constructor:

```ts
// main.tsx
import {Zero} from '@rocicorp/zero';
import {schema} from './schema';
import {createMutators} from './mutators';

const zero = new Zero({
  schema,
  mutators: createMutators(),
});
```

### Write Data on the Client

The `Transaction` interface passed to client mutators exposes the same `mutate` API as the existing [CRUD-style mutators](./writing-data):

```ts
async function myMutator(tx: Transaction) {
  // Insert a new issue
  await tx.mutate.issue.insert({
    id: 'issue-123',
    title: 'New title',
    description: 'New description',
  });

  // Upsert a new issue
  await tx.mutate.issue.upsert({
    id: 'issue-123',
    title: 'New title',
    description: 'New description',
  });

  // Update an issue
  await tx.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  });

  // Delete an issue
  await tx.mutate.issue.delete({
    id: 'issue-123',
  });
}
```

See [the CRUD docs](./writing-data) for detailed semantics on these methods.

### Read Data on the Client

You can read data within a client mutator using [ZQL](./reading-data):

```ts
export function createMutators() {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Read existing issue
        const prev = await tx.query.issue.where('id', id).one();

        // Validate title length. Legacy issues are exempt.
        if (!prev.isLegacy && title.length > 100) {
          throw new Error(`Title is too long`);
        }

        await tx.mutate.issue.update({id, title});
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

You have the full power of ZQL at your disposal, including relationships, filters, ordering, and limits.

Reads and writes within a mutator are transactional, meaning that the datastore is guaranteed to not change while your mutator is running. And if the mutator throws, the entire mutation is rolled back.

<Note type="note" heading="Run in Custom Mutators">
Outside of mutators, the `run()` method has a [`type` parameter](reading-data#running-queries-once) that can be used to wait for server results.

This parameter isn't supported within mutators, because waiting for server results makes no sense in an optimistic mutation – it defeats the purpose of running optimistically to begin with.

When a mutator runs on the client (`tx.location === "client"`), ZQL reads only return data already cached on the client. When mutators run on the server (`tx.location === "server"`), ZQL reads always return all data.

You can use `run()` within custom mutators, but the `type` argument does nothing. In the future, passing `type` in this situation will throw an error.

</Note>

### Invoking Client Mutators

Once you have registered your client mutators, you can call them from your client-side application:

```ts
zero.mutate.issue.update({
  id: 'issue-123',
  title: 'New title',
});
```

The result of a call to a mutator is a `Promise`. You do not usually need to `await` this promise as Zero mutators run very fast, usually completing in a tiny fraction of one frame.

However because mutators ocassionally need to access browser storage, they are technically `async`. Reading a row that was written by a mutator immediately after it is written may not return the new data, because the mutator may not have completed writing to storage yet.

### Waiting for Mutator Result

We typically recommend that you "fire and forget" mutators.

Optimistic mutations make sense when the common case is that a mutation succeeds. If a mutation frequently fails, then showing the user an optimistic result doesn't make sense, because it will likely be wrong.

That said there are cases where it is useful to know when a write succeeded on either the client or server.

One example is if you need to read a row directly after writing it. Zero's local writes are very fast (almost always < 1 frame), but because Zero is backed by IndexedDB, writes are still _technically_ asynchronous and reads directly after a write may not return the new data.

You can use the `.client` promise in this case to wait for a write to complete on the client side:

```ts
try {
  const write = zero.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  });

  // issue-123 not guaranteed to be present here. read1 may be undefined.
  const read1 = await zero.query.issue.where('id', 'issue-123').one();

  // Await client write – almost always less than 1 frame, and same
  // macrotask, so no browser paint will occur here.
  await write.client;

  // issue-123 definitely can be read now.
  const read2 = await zero.query.issue.where('id', 'issue-123').one();
} catch (e) {
  console.error('Mutator failed on client', e);
}
```

You can also wait for the server write to succeed:

```ts
try {
  await zero.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  }).server;

  // issue-123 is written to server
} catch (e) {
  console.error('Mutator failed on client or server', e);
}
```

If the client-side mutator fails, the `.server` promise is also rejected with the same error. You don't have to listen to both promises, the server promise covers both cases.

<Note heading="Returning data from mutators" slug="returning-data">
  There is not yet a way to return data from mutators in the success case – the type of `.clent` and `.server` is always `Promise<void>`. [Let us know](https://discord.rocicorp.dev/) if you need this.
</Note>

### Setting Up the Server

You will need a server somewhere you can run an endpoint on. This is typically a serverless function on a platform like Vercel or AWS but can really be anything.

Set the push URL with the [`ZERO_PUSH_URL` env var or `--push-url`](./zero-cache-config#push-url).

If there is per-client configuration you need to send to the push endpoint, you can do that with `push.queryParams`:

```ts
const z = new Zero({
  push: {
    queryParams: {
      workspaceID: '42',
    },
  },
});
```

The push endpoint receives a `PushRequest` as input describing one or more mutations to apply to the backend, and must return a `PushResponse` describing the results of those mutations.

If you are implementing your server in TypeScript, you can use the `PushProcessor` class to trivially implement this endpoint. Here’s an example in a [Hono](https://hono.dev/) app:

```ts
import {Hono} from 'hono';
import {handle} from 'hono/vercel';
import {
  PushProcessor,
  ZQLDatabase,
  PostgresJSConnection,
} from '@rocicorp/zero/pg';
import postgres from 'postgres';
import {schema} from '../shared/schema';
import {createMutators} from '../shared/mutators';

// PushProcessor is provided by Zero to encapsulate a standard
// implementation of the push protocol.
const processor = new PushProcessor(
  new ZQLDatabase(
    new PostgresJSConnection(postgres(process.env.ZERO_UPSTREAM_DB! as string)),
    schema,
  ),
);

export const app = new Hono().basePath('/api');

app.post('/push', async c => {
  const result = await processor.process(createMutators(), c.req.raw);
  return await c.json(result);
});

export default handle(app);
```

`PushProcessor` depends on an abstract `Database`. This allows it to implement the push algorithm against any database.

`@rocicorp/zero/pg` includes a `ZQLDatabase` implementation of this interface backed by Postgres. The implementation allows the same mutator functions to run on client and server, by providing an implementation of the ZQL APIs that custom mutators run on the client.

`ZQLDatabase` in turn relies on an abstract `DBConnection` that provides raw access to a Postgres database. This allows you to use any Postgres library you like, as long as you provide a `DBConnection` implementation for it. The `PostgresJSConnection` class implements `DBConnection` for the excellent [`postgres.js`](https://www.npmjs.com/package/postgres) library to connect to Postgres.

To reuse the client mutators exactly as-is on the server just pass the result of the same `createMutators` function to `PushProcessor`.

### Server Error Handling

The `PushProcessor` in `@rocicorp/zero/pg` skips any mutations that throw:

```ts
app.post('/push', async c => {
  const result = await processor.process({
    issue: {
      update: async (tx, data) => {
        // The mutation is skipped and the next mutation runs as normal.
        throw new Error('bonk');
      },
    },
  }, ...);
  return await c.json(result);
})
```

`PushProcessor` catches such errors and turns them into a structured response that gets sent back to the client. You can [recover the errors](#waiting-for-mutator-result) and show UI if you want.

It is also of course possible for the entire push endpoint to return an HTTP error, or to not reply at all:

```ts
app.post('/push', async c => {
  // This will cause the client to resend all queued mutations.
  throw new Error('zonk');
  const result = await processor.process({
    // ...
  }, ...);
  return await c.json(result);
})
```

If Zero recieves any response from the push endpoint other than HTTP 200, 401, or 403, it will disconnect, wait a few moments, reconnect, and then retry all unprocessed mutations.

If Zero receives HTTP 401 or 403, the client [refreshes the `auth` token](./auth#refresh) if possible, then retries all queued mutations.

If you want a different behavior, it is possible to [implement your own](#custom-push-implementation) `PushProcessor` and handle errors differently.

### Server-Specific Code

To implement server-specific code, just run different mutators in your push endpoint!

An approach we like is to create a separate `server-mutators.ts` file that wraps the client mutators:

```ts
// server-mutators.ts
import {CustomMutatorDefs} from '@rocicorp/zero';
import {schema} from './schema';

export function createMutators(
  clientMutators: CustomMutatorDefs<typeof schema>,
) {
  return {
    // Reuse all client mutators except the ones in `issue`
    ...clientMutators,

    issue: {
      // Reuse all issue mutators except `update`
      ...clientMutators.issue,

      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Call the shared mutator first
        await clientMutators.issue.update(tx, {id, title});

        // Record a history of this operation happening in an audit
        // log table.
        await tx.mutate.auditLog.insert({
          // Assuming you have an audit log table with fields for
          // `issueId`, `action`, and `timestamp`.
          issueId: id,
          action: 'update-title',
          timestamp: new Date().toISOString(),
        });
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

For simple things, we also expose a `location` field on the transaction object that you can use to branch your code:

```ts
myMutator: (tx) => {
  if (tx.location === 'client') {
    // Client-side code
  } else {
    // Server-side code
  }
},
```

### Permissions

Because custom mutators are just arbitrary TypeScript functions, there is no need for a special permissions system. Therefore, you won't use Zero's [write permissions](./permissions) when you use custom mutators.

<Note slug="read-permissions" emoji="💡" heading="You do still need *read* permissions">
  When using custom mutators you will have no [`insert`](permissions#insert-permissions), [`update`](permissions#update-permissions), or [`delete`](permissions#delete-permissions) permissions. You will still have [`select`](permissions#select-permissions) permissions, however.

We hope to build [custom queries](https://bugs.rocicorp.dev/issue/3453) next – a read analog to custom mutators. If we succeed, Zero's permission system will go away completely 🤯.

</Note>

In order to do permission checks, you'll need to know what user is making the request. You can pass this information to your mutators by adding a `AuthData` parameter to the `createMutators` function:

```ts
type AuthData = {
  sub: string;
};

export function createMutators(authData: AuthData | undefined) {
  return {
    issue: {
      launchMissiles: async (tx, args: {target: string}) => {
        if (!authData) {
          throw new Error('Users must be logged in to launch missiles');
        }

        const hasPermission = await tx.query.user
          .where('id', authData.sub)
          .whereExists('permissions', q => q.where('name', 'launch-missiles'))
          .one();
        if (!hasPermission) {
          throw new Error('User does not have permission to launch missiles');
        }
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

The `AuthData` parameter can be any data required for authorization, but is typically just the decoded JWT:

```ts
// app.tsx
const zero = new Zero({
  schema,
  auth: encodedJWT,
  mutators: createMutators(decodedJWT),
});

// hono-server.ts
const processor = new PushProcessor(
  schema,
  connectionProvider(postgres(process.env.ZERO_UPSTREAM_DB as string)),
);
processor.process(
  createMutators(decodedJWT),
  c.req.query(),
  await c.req.json(),
);
```

### Dropping Down to Raw SQL

The `ServerTransaction` interface has a `dbTransaction` property that exposes the underlying database connection. This allows you to run raw SQL queries directly against the database.

This is useful for complex queries, or for using Postgres features that Zero doesn't support yet:

```ts
markAllAsRead: async(tx: Transaction, {userId: string}) {
  // shared stuff ...

  if (tx.location === 'server') {
    // `tx` is now narrowed to `ServerTransaction`.
    // Do special server-only stuff with raw SQL.
    await tx.dbTransaction.query(
      `
      UPDATE notification
      SET read = true
      WHERE user_id = $1
    `,
      [userId],
    );
  }
}
```

As a convenience we also expose a special `CustomMutatorDefs` for use with `server-mutators.ts` that
sets all the mutators to `ServerTransaction` by default:

```ts
// server-mutators.ts
import type {
  CustomMutatorDefs,  // Special server-side CustomMutatorDefs
  PostgresJSTransaction,
  ServerTransaction,
} from '@rocicorp/zero/pg';
import {Schema} from './schema';

export function createMutators(clientMutators: CustomMutatorDefs<Schema>) {
  return {
    // Reuse all client mutators except the ones in `issue`
    ...clientMutators,

    issue: {
      // Reuse all issue mutators except `update`
      ...clientMutators.issue,

      markAllAsRead: async (tx, {userId: string}) {
        // No narrowing necessary – tx is already a `ServerTransaction`
        // assert(tx.location === 'server');
        await tx.dbTransaction.query(
          `
          UPDATE notification
          SET read = true
          WHERE user_id = $1
        `,
          [userId],
        );
      },
    }
  } as const satisfies CustomMutatorDefs<
    ServerTransaction<Schema, PostgresJSTransaction>
  >;
}

```

### Notifications and Async Work

It is bad practice to hold open database transactions while talking over the network, for example to send notifications. Instead, you should let the db transaction commit and do the work asynchronously.

There is no specific support for this in custom mutators, but since mutators are just code, it’s easy to do:

```ts
// server-mutators.ts
export function createMutators(
  authData: AuthData,
  asyncTasks: Array<() => Promise<void>>,
) {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        await tx.mutate.issue.update({id, title});

        asyncTasks.push(async () => {
          await sendEmailToSubscribers(args.id);
        });
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

Then in your push handler:

```ts
app.post('/push', async c => {
  const asyncTasks: Array<() => Promise<void>> = [];
  const result = await processor.process(
    createMutators(authData, asyncTasks),
    c.req.query(),
    await c.req.json(),
  );

  await Promise.all(asyncTasks.map(task => task()));
  return await c.json(result);
});
```

### Custom Database Connections

You can implement an adapter to a different Postgres library, or even a different database entirely.

To do so, provide a `connectionProvider` to `PushProcessor` that returns a different [`DBConnection`](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zql/src/mutate/custom.ts#L67) implementation. For an example implementation, [see the `postgres` implementation](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zero-pg/src/postgres-connection.ts#L4).

### Custom Push Implementation

You can manually implement the push protocol in any programming language.

This will be documented in the future, but you can refer to the [PushProcessor](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zero-pg/src/web.ts#L33) source code for an example for now.

## Examples

- Zbugs uses [custom mutators](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/shared/mutators.ts) for all mutations, [write permissions](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/shared/mutators.ts#L61), and [notifications](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/server/server-mutators.ts#L35).
- `hello-zero-solid` uses custom mutators for all [mutations](TODO), and for [permissions](TODO).


--- auth.mdx ---

Zero uses a [JWT](https://jwt.io/)-based flow to authenticate connections to zero-cache.

## Frontend

During login:

1. Your API server creates a `JWT` and sends it to your client.
2. Your client constructs a `Zero` instance with this token by passing it to the `auth` option.

<Note type="note" slug="user-id">
  When you set the `auth` option you must set the `userID` option to the same
  value that is present in the `sub` field of the token.
</Note>

```ts
const zero = new Zero({
  ...,
  auth: token, // your JWT
  userID, // this must match the `sub` field from `token`
});
```

## Server

For `zero-cache` to be able to verify the JWT, one of the following environment variables needs to be set:

1. `ZERO_AUTH_SECRET` - If your API server uses a symmetric key (secret) to create JWTs then this is that same key.
2. `ZERO_AUTH_JWK` - If your API server uses a private key to create JWTs then this is the corresponding public key, in [JWK](https://datatracker.ietf.org/doc/html/rfc7517) format.
3. `ZERO_AUTH_JWKS_URL` - Many auth providers host the public keys used to verify the JWTs they create at a public URL. If you use a provider that does this, or you publish your own keys publicly, set this to that URL.

## Refresh

The `auth` parameter to Zero can also be a function:

```ts
const zero = new Zero({
  ...,
  auth: async () => {
    const token = await fetchNewToken();
    return token;
  },
  userID,
});
```

In this case, Zero will call this function to get a new JWT if verification fails.

## Client-Side Data Storage

Zero stores client-side data in `IndexedDB` by default, but this is customizable with the `kvStore` parameter:

```ts
const zero = new Zero({
  // Store data in React Native's SQLite database
  // See https://github.com/Braden1996/react-native-replicache
  kvStore: createReplicacheReactNativeOPSQLiteKVStore,
});

const zero = new Zero({
  // Store data in memory, it disappears on refresh
  kvStore: 'mem',
});
```

Because multiple users can share the same browser, Zero requires that you provide a `userID` parameter on construction:

```ts
const zero = new Zero({
  ...,
  userID: "user-123",
});
```

Zero stores each user's data in a different IndexedDB instance. This allows users to quickly switch between multiple users and accounts without resyncing.

<Note emoji="🧑‍🏫" type="warning" heading="`userID` is not a security boundary" slug="user-id-is-not-a-security-boundary">
All users that have access to a browser profile have access to the same IndexedDB instances. There is nothing that Zero can do about this – users can just open the folder where the data is stored and look inside it.
</Note>

If your application is unauthenticated, or if you don't need fast user switching, you can just set `userID` to a constant like `anon` or `guest`:

```ts
const zero = new Zero({
  ...,
  userID: "anon",
});
```

Alternately, if you have more than one set of Zero data per-user (i.e., for different apps in the same domain), you can additonally use the `storageKey` parameter:

```ts
const zero = new Zero({
  ...,
  userID: "user-123",
  storageKey: "my-app",
});
```
 
If specified, `storageKey` is concatenated along with `userID` and other internal Zero information to form a unique IndexedDB database name.

<ImageLightbox src="/images/auth/indexeddb.png" caption="Zero's IndexedDB databases are prefixed with 'rep' or 'replicache' because ... well because we haven't fixed this yet." />

## Logging Out

When a user logs out, you should consider what should happen to the synced data.
 
If you do nothing, the synced data will be left on the device. The next login will be a little faster because Zero doesn't have to resync that dta from scratch. But also, the data will be left on the device indefinitely which could be undesirable for privacy and security.

 If you instead want to clear data on logout, Zero provides the `dropAllDatabases` function:

```ts
import {dropAllDatabases} from '@rocicorp/zero';

// Returns an object with:
// - The names of the successfully dropped databases
// - Any errors encountered while dropping
const {dropped, errors} = await dropAllDatabases();
```

## Permissions

Any data placed into your JWT (claims) can be used by permission rules on the backend.

```ts
const isAdminRule = (decodedJWT, {cmp}) => cmp(decodedJWT.role, '=', 'admin');
```

See the [permissions](permissions) section for more details.

## Examples

See [zbugs](samples#zbugs) or [hello-zero](samples#hello-zero).


--- permissions.mdx ---

Permissions are expressed using [ZQL](reading-data) and run automatically with every read and write.

## Define Permissions

Permissions are defined in [`schema.ts`](/docs/zero-schema) using the `definePermissions` function.

Here's an example of limiting reads to members of an organization and deletes to only the creator of an issue:

```ts
// The decoded value of your JWT.
type AuthData = {
  // The logged-in user.
  sub: string;
};

export const permissions = definePermissions<AuthData, Schema>(schema, () => {
  // Checks if the user exists in a related organization
  const allowIfInOrganization = (
    authData: AuthData,
    eb: ExpressionBuilder<Schema, 'issue'>,
  ) =>
    eb.exists('organization', q =>
      q.whereExists('user', q => q.where('id', authData.sub)),
    );

  // Checks if the user is the creator
  const allowIfIssueCreator = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('creatorID', authData.sub);

  return {
    issue: {
      row: {
        select: [allowIfInOrganization],
        delete: [allowIfIssueCreator],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

`definePermission` returns a _policy_ object for each table in the schema. Each policy defines a _ruleset_ for the _operations_ that are possible on a table: `select`, `insert`, `update`, and `delete`.

## Access is Denied by Default

If you don't specify any rules for an operation, it is denied by default. This is an important safety feature that helps ensure data isn't accidentally exposed.

To enable full access to an action (i.e., during development) use the `ANYONE_CAN` helper:

```ts
import {ANYONE_CAN} from '@rocicorp/zero';

const permissions = definePermissions<AuthData, Schema>(schema, () => {
  return {
    issue: {
      row: {
        select: ANYONE_CAN,
        // Other operations are denied by default.
      },
    },
    // Other tables are denied by default.
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

To do this for all actions, use `ANYONE_CAN_DO_ANYTHING`:

```ts
import {ANYONE_CAN_DO_ANYTHING} from '@rocicorp/zero';

const permissions = definePermissions<AuthData, Schema>(schema, () => {
  return {
    // All operations on issue are allowed to all users.
    issue: ANYONE_CAN_DO_ANYTHING,

    // Other tables are denied by default.
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Permission Evaluation

Zero permissions are "compiled" into a JSON-based format at build-time. This file is stored in the `{ZERO_APP_ID}.permissions` table of your upstream database. Like other tables, it replicates live down to `zero-cache`. `zero-cache` then parses this file, and applies the encoded rules to every read and write operation.

<Note type="warning" slug="rule-compilation-is-dumb">
The compilation process is very simple-minded (read: dumb). Despite looking like normal TypeScript functions that receive an `AuthData` parameter, rule functions are not actually invoked at runtime. Instead, they are invoked with a "placeholder" `AuthData` at build time. We track which fields of this placeholder are accessed and construct a ZQL expression that accesses the right field of `AuthData` at runtime.

The end result is that you can't really use most features of JS in these rules. Specifically you cannot:

- Iterate over properties or array elements in the auth token
- Use any JS features beyond property access of `AuthData`
- Use any conditional or global state

Basically only property access is allowed. This is really confusing and we're working on a better solution.

</Note>

## Permission Deployment

During development, permissions are compiled and uploaded to your database completely automatically as part of the `zero-cache-dev` script.

For production, you need to call `npx zero-deploy-permissions` within your app to update the permissions in the production database whenever they change. You would typically do this as part of your normal schema migration or CI process. For example, the SST deployment script for [zbugs](/docs/samples#zbugs) looks like this:

```ts
new command.local.Command(
  'zero-deploy-permissions',
  {
    create: `npx zero-deploy-permissions -p ../../src/schema.ts`,
    // Run the Command on every deploy ...
    triggers: [Date.now()],
    environment: {
      ZERO_UPSTREAM_DB: commonEnv.ZERO_UPSTREAM_DB,
      // If the application has a non-default App ID ...
      ZERO_APP_ID: commonEnv.ZERO_APP_ID,
    },
  },
  // after the view-syncer is deployed.
  {dependsOn: viewSyncer},
);
```

See the [SST Deployment Guide](deployment#guide-multi-node-on-sstaws) for more details.

## Rules

Each operation on a policy has a _ruleset_ containing zero or more _rules_.

A rule is just a TypeScript function that receives the logged in user's `AuthData` and generates a ZQL [where expression](reading-data#compound-filters). At least one rule in a ruleset must return a row for the operation to be allowed.

## Select Permissions

You can limit the data a user can read by specifying a `select` ruleset.

Select permissions act like filters. If a user does not have permission to read a row, it will be filtered out of the result set. It will not generate an error.

For example, imagine a select permission that restricts reads to only issues created by the user:

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueCreator = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('creatorID', authData.sub);

  return {
    issue: {
      row: {
        select: [allowIfIssueCreator],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

If the issue table has two rows, one created by the user and one by someone else, the user will only see the row they created in any queries.

## Insert Permissions

You can limit what rows can be inserted and by whom by specifying an `insert` ruleset.

Insert rules are evaluated after the entity is inserted. So if they query the database, they will see the inserted row present. If any rule in the insert ruleset returns a row, the insert is allowed.

Here's an example of an insert rule that disallows inserting users that have the role 'admin'.

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfNonAdmin = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'user'>,
  ) => cmp('role', '!=', 'admin');

  return {
    user: {
      row: {
        insert: [allowIfNonAdmin],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Update Permissions

There are two types of update rulesets: `preMutation` and `postMutation`. Both rulesets must pass for an update to be allowed.

`preMutation` rules see the version of a row _before_ the mutation is applied. This is useful for things like checking whether a user owns an entity before editing it.

`postMutation` rules see the version of a row _after_ the mutation is applied. This is useful for things like ensuring a user can only mark themselves as the creator of an entity and not other users.

Like other rulesets, `preMutation` and `postMutation` default to `NOBODY_CAN`. This means that every table must define both these rulesets in order for any updates to be allowed.

For example, the following ruleset allows an issue's owner to edit, but **not** re-assign the issue. The `postMutation` rule enforces that the current user still own the issue after edit.

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: [allowIfIssueOwner],
          postMutation: [allowIfIssueOwner],
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

This ruleset allows an issue's owner to edit and re-assign the issue:

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: [allowIfIssueOwner],
          postMutation: ANYONE_CAN,
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

And this allows anyone to edit an issue, but only if they also assign it to themselves. Useful for enforcing _"patches welcome"_? 🙃

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: ANYONE_CAN,
          postMutation: [allowIfIssueOwner],
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Delete Permissions

Delete permissions work in the same way as `insert` permissions except they run _before_ the delete is applied. So if a delete rule queries the database, it will see that the deleted row is present. If any rule in the ruleset returns a row, the delete is allowed.

## Debugging

See [Debugging Permissions](./debug/permissions).

## Examples

See [hello-zero](https://github.com/rocicorp/hello-zero/blob/main/src/schema.ts) for a simple example of write auth and [zbugs](https://github.com/rocicorp/mono/blob/main/apps/zbugs/shared/schema.ts#L217) for a much more involved one.


--- zql-on-the-server.mdx ---

[Custom Mutators](custom-mutators) use ZQL on the server as an implementation detail, but you can also use ZQL on the server directly, outside of Custom Mutators.

This is useful for a variety of reasons:

- You can use ZQL to implement standard REST endpoints, allowing you to share code with custom mutators.
- You can use ZQL as part of schema migrations.
- In the future ([but not yet implemented](#ssr)), this can support server-side rendering

Here's a basic example:

```ts
import {
  PushProcessor,
  ZQLDatabase,
  PostgresJSConnection,
  TransactionProviderInput,
} from '@rocicorp/zero/pg';

const db = new ZQLDatabase(
  new PostgresJSConnection(
    postgres(
      must(
        process.env.ZERO_UPSTREAM_DB as string,
        'required env var ZERO_UPSTREAM_DB',
      ),
    ),
  ),
  schema,
);

// This is needed temporarily and will be cleaned up in the future.
const dummyTransactionInput: TransactionProviderInput = {
  clientGroupID: 'unused',
  clientID: 'unused',
  mutationID: 42,
  upstreamSchema: 'unused',
};

db.transaction(async tx => {
  // await tx.mutate...
  // await tx.query...
  // await myMutator(tx, ...args);
}, dummyTransactionInput);
```

If ZQL does not have the featuers you need, you can use `tx.dbTransaction` to [drop down to raw SQL](custom-mutators#dropping-down-to-raw-sql).

<Note type="warning" slug="schema-read">
  `ZQLDatabase` currently does a read of your postgres schema before every
  transaction. This is fine for most usages, but for high scale it may become a
  problem. [Let us know](https://bugs.rocicorp.dev/issue/3799) if you need a fix
  for this.
</Note>

## Custom Database Connection

Zero only provides an adapter for `postgres.js`. It is possible to write your own adatapter by implementing `DBTransaction` and `DBConnection`.

### Node Postgres

Here is an example for `node-postgres` by Jökull Sólberg ([full example](https://gist.github.com/jokull/68a656b013a14912c1b3a851e3eec117))

```ts
import {Client, type ClientBase} from 'pg';

class PgConnection implements DBConnection<ClientBase> {
  readonly #client: ClientBase;

  constructor(client: ClientBase) {
    this.#client = client;
  }

  async query(sql: string, params: unknown[]): Promise<Row[]> {
    const result = await this.#client.query<Row>(sql, params as JSONValue[]);
    return result.rows;
  }

  async transaction<T>(
    fn: (tx: DBTransaction<ClientBase>) => Promise<T>,
  ): Promise<T> {
    if (!(this.#client instanceof Client)) {
      throw new Error('Transactions require a non-pooled Client instance');
    }

    const tx = new PgTransaction(this.#client);

    try {
      await this.#client.query('BEGIN');
      const result = await fn(tx);
      await this.#client.query('COMMIT');
      return result;
    } catch (error) {
      await this.#client.query('ROLLBACK');
      throw error;
    }
  }
}

class PgTransaction implements DBTransaction<ClientBase> {
  readonly wrappedTransaction: ClientBase;

  constructor(client: ClientBase) {
    this.wrappedTransaction = client;
  }

  async query(sql: string, params: unknown[]): Promise<Row[]> {
    const result = await this.wrappedTransaction.query<Row>(
      sql,
      params as JSONValue[],
    );
    return result.rows;
  }
}

// Then you can use it just like postgres.js
const client = new Client({
  connectionString: process.env.ZERO_UPSTREAM_DB,
});
await client.connect();

const db = new ZQLDatabase(new PgConnection(client), schema);
```

### Drizzle ORM

It is also possible to use ORMs like Drizzle. Wrap the drizzle transaction and now you can access drizzle's transaction within custom mutators.

[Blog post and full example](https://www.solberg.is/zero-custom-db-connection) by Jökull Sólberg (again 🙌)

```ts
// Assuming $client is the raw pg.PoolClient, this type matches how
// `drizzle()` inits when using `node-postgres`
type Drizzle = NodePgDatabase<typeof schema> & {$client: PoolClient};

// Extract the Drizzle-specific transaction type
type DrizzleTransaction = Parameters<Parameters<Drizzle['transaction']>[0]>[0];

class DrizzleConnection implements DBConnection<DrizzleTransaction> {
  drizzle: Drizzle;

  constructor(drizzle: Drizzle) {
    this.drizzle = drizzle;
  }

  // `query` is used by Zero's ZQLDatabase for ZQL reads on the server
  query(sql: string, params: unknown[]): Promise<Row[]> {
    return this.drizzle.$client
      .query<QueryResultRow>(sql, params)
      .then(({rows}) => rows);
  }

  // `transaction` wraps Drizzle's transaction
  transaction<T>(
    fn: (tx: DBTransaction<DrizzleTransaction>) => Promise<T>,
  ): Promise<T> {
    return this.drizzle.transaction(drizzleTx =>
      // Pass a new Zero DBTransaction wrapper around Drizzle's one
      fn(new ZeroDrizzleTransaction(drizzleTx)),
    );
  }
}

class ZeroDrizzleTransaction implements DBTransaction<DrizzleTransaction> {
  readonly wrappedTransaction: DrizzleTransaction;

  constructor(drizzleTx: DrizzleTransaction) {
    this.wrappedTransaction = drizzleTx;
  }

  // This `query` method would be used if ZQL reads happen *within*
  // a custom mutator that is itself running inside this wrapped transaction.
  query(sql: string, params: unknown[]): Promise<Row[]> {
    // Drizzle's transaction object might hide the raw client,
    // this is one way to get at it for `pg` driver. Adjust if needed.
    const session = this.wrappedTransaction._.session as unknown as {
      client: Drizzle['$client'];
    };
    return session.client
      .query<QueryResultRow>(sql, params)
      .then(({rows}) => rows);
  }
}
```

## SSR

Although you can run ZQL on the server, Zero does not yet have the wiring setup in its bindings layers to support server-side rendering ([patches welcome though!](https://bugs.rocicorp.dev/issue/3491)).

For now, you should use your framework's recommended pattern to prevent SSR execution.

### Next.js

Add the `use client`
[directive](https://nextjs.org/docs/app/api-reference/directives/use-client).

### SolidStart

Wrap components that use Zero with the
[`clientOnly`](https://docs.solidjs.com/solid-start/reference/client/client-only)
higher-order component.

The standard `clientOnly` pattern uses dynamic imports, but note that this
approach (similar to [React's `lazy`](https://react.dev/reference/react/lazy))
works with any function returning a `Promise<{default: () => JSX.Element}>`. If
code splitting is unnecessary, you can skip the dynamic import.

### TanStack Start

Use [React's `lazy`](https://react.dev/reference/react/lazy) for dynamic
imports.


--- offline.mdx ---

Zero currently supports offline reads, but not writes. We plan to support offline writes in the future, but we don't have a timeline for that yet.

<Note emoji="🤓" type="warning" heading="Well actually…">
Offline writes *are* currently accepted by Zero, and sent when you come back online. But we plan to disable this soon.
</Note>

The lack of offline writes is often surprising to people familiar with sync engines, because offline is usually touted as something that comes for free with these tools.

This page explains why Zero doesn't currently support offline writes, how we recommend you handle connectivity loss, and our future plans in this area.

## Offline Writes are a UX Problem

While Zero can technically queue offline writes and replay them when reconnected (this happens by default in any sync engine, and is what Zero does today), that fact doesn't make supporting offline writes much easier. That's because a really hard part of offline writes is in handling conflicts, and no software tool can make that problem go away.

For example, imagine two users are editing an article about cats. One goes offline and does a bunch of work on the article, while the other decides that the article should actually be about dogs and rewrites it. When the offline user reconnects, there is no way that any software algorithm can automatically resolve their conflict. One or the other of them is going to be upset.

And while the above example may sound extreme, you can construct similar situations with the majority of common applications. Just take your own application and ask yourself what should really happen if one user takes their device offline for a week and makes arbitrarily complex changes while other users are working online.

People who work on sync engines and related tools often say that *offline is just extreme lag*, but that's only true at a technical level. At a human level, being "offline" for a few seconds is very different from being offline for a few hours. The difference is how much knowledge you have about what your collaborators are doing, and how much of your work can be lost.

The only way to support offline writes in general is to either:

1. Make the logical datamodel append-only (i.e., users can create and mark tasks done, but cannot edit or delete them).
2. Support custom UX to allow users to fork and merge conflicts when they occur.
3. Only support editing from a single device.

None of these is *free*. Buiding a good offline UX is a lot of work, and most of that work is borne by application developers.

## … And a Schema Problem

But it's not just users that can diverge from each other. The server software and database schema can also diverge arbitrarily far from the client while the client is disconnected. When the client comes back online, the changes made may no longer be processable by the application, or may have a different effect than the user intended.

So to support long offline periods, the server must also maintain backward compatibility with clients indefinitely. Similarly, the server can never reject an offline write (i.e., due to validation) because that could lead to a user losing huge amounts of work.

## … And a Sync Engine Problem

Supporting offline writes also requires work in the sync engine.

In Zero, there are a few specific impacts:

1. The Zero client itself can get out of date while offline. On reconnect, the app might reload with a new version of the client. This new version must be able to read and process old data from arbitrarily long ago.
2. An arbitrarily large number of pending mutations can be built up. These mutations must be replayed on reconnect, which can take a long time.
3. When processing mutations on server we must consider what should happen if the database or application server are temporarily unavailable. We need to treat that kind of error differently from a validation error.

These problems are surmountable, but significant effort. Their solutions might also be in tension with other goals of the sync engine, like online performance and scalability. These tradeoffs will take time to work through.

## Zero's Position

For all of the above reasons, we plan to disable offline writes in Zero for beta.

When the Zero client loses connection to `zero-cache` for several minutes (or when `zero-cache` cannot reach the customer API server), it will enter a special *offline mode*. In this mode, all writes to Zero will throw.

While we recognize that offline writes would be useful for some applications, the reality is that for most of the apps we want to support, the user is online the vast majority of the time and the cost to support offline is extremely high. There is simply more value in making the online experience great first.

We would like to revisit this in the future and really think through how to design APIs and patterns that allow developers to make successful offline-enabled apps. But it's not our priority right now.

## Dealing with Offline Today

Until Zero disables offline writes automatically, we recomment using the `onOnlineChange` parameter to the `Zero` constructor to detect connection loss and disable editing manually in your UI.

## Even More Information

* [Lies I was Told About Collaborative Editing](https://www.moment.dev/blog/lies-i-was-told-pt-1): a detailed overview of the challenges around offline writes in any collaborative editing system.
* This [Zero Discord thread](https://discord.com/channels/830183651022471199/1363957701331390669/1364260786108633198) covers some challenges specifically in the context of Zero.
* [Patchwork](https://www.inkandswitch.com/patchwork/notebook/) by Ink & Switch is new and interesting research around how to support offline writes well in collaborative systems.


--- deployment.mdx ---

To deploy a Zero app, you need to:

1. Deploy your backend database. Most standard Postgres hosts [work with Zero](connecting-to-postgres).
1. Deploy `zero-cache`. We provide a [Docker image](https://hub.docker.com/r/rocicorp/zero) that can work with most Docker hosts.
1. Deploy your frontend. You can use any hosting service like Vercel or Netlify.

This page describes how to deploy `zero-cache`.

## Architecture

`zero-cache` is a horizontally scalable, stateful web service that maintains a SQLite replica of your Postgres database. It uses this replica to sync ZQL queries to clients over WebSockets.

You don't have to know the details of how `zero-cache` works to run it, but it helps to know the basic structure.

<ImageLightbox src="/images/deployment/arch.png" />

A running `zero-cache` is composed of a single `replication-manager` node and multiple `view-syncer` nodes. It also depends on Postgres, S3, and attached SSD storage.

**Upstream:** Your application's Postgres database.

**Change DB:** A Postgres DB used by Zero to store a recent subset of the Postgres replication log.

**CVR DB:** A Postgres DB used by Zero to store Client View Records (CVRs). CVRs track the state of each synced client.

<Note type="note" slug="pg-dbs">
The Change DB and CVR DBs are typically the same physical Postgres database as the Upstream DB. Zero stores their tables in separate Postgres schemas so they won't conflict with your application data.

We allow separate DBs so that they can be scaled and tuned independently if desired.

</Note>

**S3:** Stores a canonical copy of the SQLite replica.

**File System:** Used by both node types to store local copies of the SQLite replica. Can be ephemeral – Zero will re-initialize from S3 on startup. Recommended to use attached SSD storage for best performance.

**Replication Manager:** Serves as the single consumer of the Postgres replication log. Stores a recent subset of the Postgres changelog in the _Change DB_ for catching up ViewSyncers when they initialize. Also maintains the canonical replica, which ViewSyncers initialize from.

**View Syncers:** Handle WebSocket connections from clients and run ZQL queries. Updates CVR DB with the latest state of each client as queries run. Uses CVR DB on client connection to compute the initial diff to catch clients up.

## Topology

You should deploy `zero-cache` close to your database because the mutation implementation is chatty.

In the future, mutations will [move out of `zero-cache`](https://bugs.rocicorp.dev/issue/3045#comment-5a3BKxP8RfJ9njHLgx5e3).
When that happens you can deploy `zero-cache` geographically distributed and it will double as a read-replica.

## Updating

When run with multiple View Syncer nodes, `zero-cache` supports rolling, downtime-free updates. A new Replication Manager takes over the replication stream from the old Replication Manager, and connections from the old View Syncers are gradually drained and absorbed by active View Syncers.

## Client/Server Version Compatibility

Servers are compatible with any client of same major version, and with clients one major version back. So for example:

- Server `0.2.*` is compatible with client `0.2.*`
- Server `0.2.*` is compatible with client `0.1.*`
- Server `2.*.*` is compatible with client `2.*.*`
- Server `2.*.*` is compatible with client `1.*.*`

To upgrade Zero to a new major version, first deploy the new zero-cache, then the new frontend.

## Configuration

The `zero-cache` image is configured via environment variables. See [zero-cache Config](./zero-cache-config) for available options.

<Note heading="Custom mutators" slug="push-url">
  When using custom mutators, don't forget to set the push api url with
  `ZERO_PUSH_URL`
</Note>

## Guide: Multi-Node on SST+AWS

[SST](https://sst.dev/) is our recommended way to deploy Zero.

The setup below costs about $35/month. You can scale it up or down as needed by adjusting the amount of vCPUs and memory in each task.

### Setup Upstream

Create an upstream Postgres database server somewhere. See [Connecting to Postgres](connecting-to-postgres) for details. Populate the schema and any initial data for your application.

### Setup AWS

See [AWS setup guide](https://v2.sst.dev/setting-up-aws). The end result should be that you have a dev profile and SSO session defined in your `~/.aws/config` file.

### Initialize SST

```bash
npx sst init --yes
```

Choose "aws" for where to deploy.

Then overwite `/sst.config.ts` with the following code:

```ts
/* eslint-disable */
/// <reference path="./.sst/platform/config.d.ts" />
import {execSync} from 'child_process';

export default $config({
  app(input) {
    return {
      name: 'hello-zero',
      removal: input?.stage === 'production' ? 'retain' : 'remove',
      home: 'aws',
      region: process.env.AWS_REGION || 'us-east-1',
      providers: {
        command: true,
      },
    };
  },
  async run() {
    const zeroVersion = execSync('cat package.json | jq '.dependencies["@rocicorp/zero"]')
      .toString()
      .trim();

    // S3 Bucket
    const replicationBucket = new sst.aws.Bucket(`replication-bucket`);

    // VPC Configuration
    const vpc = new sst.aws.Vpc(`vpc`, {
      az: 2,
    });

    // ECS Cluster
    const cluster = new sst.aws.Cluster(`cluster`, {
      vpc,
    });

    const conn = new sst.Secret('PostgresConnectionString');
    const zeroAuthSecret = new sst.Secret('ZeroAuthSecret');

    // Common environment variables
    const commonEnv = {
      ZERO_PUSH_URL: '', // Your push api url when using custom mutators
      ZERO_UPSTREAM_DB: conn.value,
      ZERO_CVR_DB: conn.value,
      ZERO_CHANGE_DB: conn.value,
      ZERO_AUTH_SECRET: zeroAuthSecret.value,
      ZERO_REPLICA_FILE: 'sync-replica.db',
      ZERO_IMAGE_URL: `rocicorp/zero:${zeroVersion}`,
      ZERO_CVR_MAX_CONNS: '10',
      ZERO_UPSTREAM_MAX_CONNS: '10',
    };

    // Replication Manager Service
    const replicationManager = cluster.addService(`replication-manager`, {
      cpu: '0.5 vCPU',
      memory: '1 GB',
      architecture: 'arm64',
      image: commonEnv.ZERO_IMAGE_URL,
      link: [replicationBucket],
      wait: true,
      health: {
        command: ['CMD-SHELL', 'curl -f http://localhost:4849/ || exit 1'],
        interval: '5 seconds',
        retries: 3,
        startPeriod: '300 seconds',
      },
      environment: {
        ...commonEnv,
        ZERO_LITESTREAM_BACKUP_URL: $interpolate`s3://${replicationBucket.name}/backup`,
        ZERO_NUM_SYNC_WORKERS: '0',
      },
      transform: {
        target: {
          healthCheck: {
            enabled: true,
            path: '/keepalive',
            protocol: 'HTTP',
            interval: 5,
            healthyThreshold: 2,
            timeout: 3,
          },
        },
      },
    });

    // View Syncer Service
    const viewSyncer = cluster.addService(
      `view-syncer`,
      {
        cpu: '1 vCPU',
        memory: '2 GB',
        architecture: 'arm64',
        image: commonEnv.ZERO_IMAGE_URL,
        link: [replicationBucket],
        health: {
          command: ['CMD-SHELL', 'curl -f http://localhost:4848/ || exit 1'],
          interval: '5 seconds',
          retries: 3,
          startPeriod: '300 seconds',
        },
        environment: {
          ...commonEnv,
          ZERO_CHANGE_STREAMER_MODE: 'discover',
        },
        logging: {
          retention: '1 month',
        },
        transform: {
          target: {
            healthCheck: {
              enabled: true,
              path: '/keepalive',
              protocol: 'HTTP',
              interval: 5,
              healthyThreshold: 2,
              timeout: 3,
            },
            stickiness: {
              enabled: true,
              type: 'lb_cookie',
              cookieDuration: 120,
            },
          },
        },
      },
      {
        // Wait for replication-manager to come up first, for breaking changes
        // to replication-manager interface.
        dependsOn: [replicationManager],
      },
    );

    // Permissions deployment
    // Note: this setup requires your CI/CD pipeline to have access to your
    // Postgres database. If you do not want to do this, you can also use
    // `npx zero-deploy-permissions --output-format=sql` during build to
    // generate a permissions.sql file, then run that file as part of your
    // deployment within your VPC. See hello-zero-solid for an example:
    // https://github.com/rocicorp/hello-zero-solid/blob/main/sst.config.ts#L141
    new command.local.Command(
      'zero-deploy-permissions',
      {
        create: `npx zero-deploy-permissions -p ../../src/schema.ts`,
        // Run the Command on every deploy ...
        triggers: [Date.now()],
        environment: {
          ZERO_UPSTREAM_DB: commonEnv.ZERO_UPSTREAM_DB,
        },
      },
      // after the view-syncer is deployed.
      {dependsOn: viewSyncer},
    );
  },
});
```

### Set SST Secrets

Configure SST with your Postgres connection string and [Zero Auth Secret](/docs/auth#server).

Note that if you use JWT-based auth, you'll need to change the environment variables in the `sst.config.ts` file above, then set a different secret here.

```bash
npx sst secret set PostgresConnectionString "YOUR-PG-CONN-STRING"
npx sst secret set ZeroAuthSecret "YOUR-ZERO-AUTH-SECRET"
```

### Deploy

```bash
npx sst deploy
```

This takes about 5-10 minutes.

If successful, you should see a URL for the `view-syncer` service. This is the URL to pass to the `server` parameter of the `Zero` constructor on the client.

If unsuccessful, you can get detailed logs with `npx sst deploy --verbose`. [Come find us on Discord](https://discord.rocicorp.dev/) and we'll help get you sorted out.

## Guide: Single-Node on Fly.io

Let's deploy the [Quickstart](quickstart) app to [Fly.io](https://fly.io). We'll use Fly.io for both the database and `zero-cache`.

### Setup Quickstart

Go through the [Quickstart](quickstart) guide to get the app running locally.

### Setup Fly.io

Create an account on [Fly.io](https://fly.io) and [install the Fly CLI](https://fly.io/docs/flyctl/install/).

### Create Postgres app

<Note type="warning" slug="fly-app-names">
  **Note:** Fly.io requires app names to be unique across all Fly.io users.
  Change the `INITIALS` environment variable below to something unique.
</Note>

```bash
INITIALS=aa
PG_APP_NAME=$INITIALS-zstart-pg

PG_PASSWORD="$(head -c 256 /dev/urandom | od -An -t x1 | tr -d ' \n' | tr -dc 'a-zA-Z' | head -c 16)"

fly postgres create \
  --name $PG_APP_NAME \
  --region lax \
  --initial-cluster-size 1 \
  --vm-size shared-cpu-2x \
  --volume-size 40 \
  --password=$PG_PASSWORD
```

### Seed Upstream database

Populate the database with initial data and set its `wal_level` to `logical` to support replication to `zero-cache`. Then restart the database to apply the changes.

```bash
(cat ./docker/seed.sql; echo "\q") | fly pg connect -a $PG_APP_NAME
echo "ALTER SYSTEM SET wal_level = logical; \q" | fly pg connect -a $PG_APP_NAME
fly postgres restart --app $PG_APP_NAME
```

### Create `zero-cache` Fly.io app

```bash
CACHE_APP_NAME=$INITIALS-zstart-cache
fly app create $CACHE_APP_NAME
```

### Publish `zero-cache`

Create a `fly.toml` file.

```bash
CONNECTION_STRING="postgres://postgres:$PG_PASSWORD@$PG_APP_NAME.flycast:5432"
ZERO_VERSION=$(npm list @rocicorp/zero | grep @rocicorp/zero | cut -f 3 -d @)

cat <<EOF > fly.toml
app = "$CACHE_APP_NAME"
primary_region = 'lax'

[build]
image = "registry.hub.docker.com/rocicorp/zero:${ZERO_VERSION}"

[http_service]
internal_port = 4848
force_https = true
auto_stop_machines = 'off'
min_machines_running = 1

[[http_service.checks]]
grace_period = "10s"
interval = "30s"
method = "GET"
timeout = "5s"
path = "/"

[[vm]]
memory = '2gb'
cpu_kind = 'shared'
cpus = 2

[mounts]
source = "sqlite_db"
destination = "/data"

[env]
ZERO_REPLICA_FILE = "/data/sync-replica.db"
ZERO_UPSTREAM_DB="${CONNECTION_STRING}/zstart?sslmode=disable"
ZERO_CVR_DB="${CONNECTION_STRING}/zstart_cvr?sslmode=disable"
ZERO_CHANGE_DB="${CONNECTION_STRING}/zstart_cdb?sslmode=disable"
ZERO_PUSH_URL=""
ZERO_AUTH_SECRET="secretkey"
LOG_LEVEL = "debug"
EOF
```

Then publish `zero-cache`:

```bash
fly deploy
```

### Deploy Permissions

Now `zero-cache` is running on Fly.io, but there are no permissions. If you run the app against this `zero-cache`, you'll see that no data is returned from any query. To fix this, deploy your permissions:

```bash
npx zero-deploy-permissions --schema-path='./src/schema.ts'  --output-file='/tmp/permissions.sql'
(cat /tmp/permissions.sql; echo "\q") | fly pg connect -a $PG_APP_NAME -d zstart
```

You will need to redo this step every time you change your app's permissions, likely as part of your
CI/CD pipeline.

### Use Remote `zero-cache`

```bash
VITE_PUBLIC_SERVER="https://${CACHE_APP_NAME}.fly.dev/" npm run dev:ui
```

Now restart the frontend to pick up the env change, and refresh the app. You can stop your local database and `zero-cache` as we're not using them anymore. Open the web inspector to verify the app is talking to the remote `zero-cache`!

You can deploy the frontend to any standard hosting service like Vercel or Netlify, or even to Fly.io!

### Deploy Frontend to Vercel

If you've followed the above guide and deployed `zero-cache` to fly, you can simply run:

```sh
vercel deploy --prod \
  -e ZERO_AUTH_SECRET="secretkey" \
  -e VITE_PUBLIC_SERVER='https://${CACHE_APP_NAME}.fly.dev/'
```

to deploy your frontend to Vercel.

Explaining the arguments above --

- `ZERO_AUTH_SECRET` - The secret to create and verify JWTs. This is the same secret that was used when deploying zero-cache to fly.
- `VITE_PUBLIC_SERVER` - The URL the frontend will call to talk to the zero-cache server. This is the URL of the fly app.

## Guide: Multi-Node on Raw AWS

### S3 Bucket

Create an S3 bucket. `zero-cache` uses S3 to backup its SQLite replica so that it survives task restarts.

### Fargate Services

Run `zero-cache` as two Fargate services (using the same [rocicorp/zero](https://hub.docker.com/r/rocicorp/zero) docker image):

#### replication-manager

- `zero-cache` [config](https://zero.rocicorp.dev/docs/zero-cache-config):
  - `ZERO_LITESTREAM_BACKUP_URL=s3://{bucketName}/{generation}`
  - `ZERO_NUM_SYNC_WORKERS=0`
- Task count: **1**

#### view-syncer

- `zero-cache` config:
  - `ZERO_CHANGE_STREAMER_MODE=discover`
- Task count: **N**

### Notes

- Standard rolling restarts are fine for both services
- The `view-syncer` task count is static; update the service to change the count.
  - Support for dynamic resizing (i.e. Auto Scaling) is planned
  - Set `ZERO_CVR_MAX_CONNS` and `ZERO_UPSTREAM_MAX_CONNS` appropriately so that the total connections from both running and updating `view-syncers` (e.g. DesiredCount \* MaximumPercent) do not exceed your database’s `max_connections`.
- The `{generation}` component of the `s3://{bucketName}/{generation}` URL is an arbitrary path component that can be modified to reset the replica (e.g. a date, a number, etc.). Setting this to a new path is the multi-node equivalent of deleting the replica file to resync.
  - Note: `zero-cache` does not manage cleanup of old generations.
- The `replication-manager` serves requests on port **4849**. Routing from the `view-syncer` to the `http://{replication-manager}` is handled internally by storing data in the `changedb`.
- Fargate ephemeral storage is used for the replica.
  - The default size is 20GB. This can be increased up to 200GB
  - Allocate at least twice the size of the database to support the internal VACUUM operation.

## Guide: $PLATFORM

Where should we deploy Zero next?? Let us know on [Discord](https://discord.rocicorp.dev)!


--- zero-cache-config.mdx ---

`zero-cache` is configured either via CLI flag or environment variable. There is no separate `zero.config` file.

You can also see all available flags by running `zero-cache --help`.

## Required Flags

### Auth

One of [Auth JWK](#auth-jwk), [Auth JWK URL](#auth-jwk-url), or [Auth Secret](#auth-secret) must be specified. See [Authentication](/docs/auth/) for more details.

### Replica File

File path to the SQLite replica that zero-cache maintains. This can be lost, but if it is, zero-cache will have to re-replicate next time it starts up.

flag: `--replica-file`<br/>
env: `ZERO_REPLICA_FILE`<br/>
required: `true`

### Upstream DB

The "upstream" authoritative postgres database. In the future we will support other types of upstream besides PG.

flag: `--upstream-db`<br/>
env: `ZERO_UPSTREAM_DB`<br/>
required: `true`

## Optional Flags

### Admin Password

A password used to administer zero-cache server, for example to access the `/statz` endpoint.

flag: `--admin-password`<br/>
env: `ZERO_ADMIN_PASSWORD`<br/>
required: `false`

### App ID

Unique identifier for the app.

Multiple zero-cache apps can run on a single upstream database, each of which is isolated from the others, with its own permissions, sharding (future feature), and change/cvr databases.

The metadata of an app is stored in an upstream schema with the same name, e.g. `zero`, and the metadata for each app shard, e.g. client and mutation ids, is stored in the `{app-id}_{#}` schema. (Currently there is only a single "0" shard, but this will change with sharding).

The CVR and Change data are managed in schemas named `{app-id}_{shard-num}/cvr` and `{app-id}_{shard-num}/cdc`, respectively, allowing multiple apps and shards to share the same database instance (e.g. a Postgres "cluster") for CVR and Change management.

Due to constraints on replication slot names, an App ID may only consist of lower-case letters, numbers, and the underscore character.

Note that this option is used by both `zero-cache` and `zero-deploy-permissions`.

flag: `--app-id`<br/>
env: `ZERO_APP_ID`<br/>
default: `zero`

### App Publications

Postgres PUBLICATIONs that define the tables and columns to replicate. Publication names may not begin with an underscore, as zero reserves that prefix for internal use.

If unspecified, zero-cache will create and use an internal publication that publishes all tables in the public schema, i.e.:

```
CREATE PUBLICATION _{app-id}_public_0 FOR TABLES IN SCHEMA public;
```

Note that once an app has begun syncing data, this list of publications cannot be changed, and zero-cache will refuse to start if a specified value differs from what was originally synced. To use a different set of publications, a new app should be created.

flag: `--app-publications`<br/>
env: `ZERO_APP_PUBLICATIONS`<br/>
default: `[]`

### Auth JWK

A public key in JWK format used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-jwk`<br/>
env: `ZERO_AUTH_JWK`<br/>
required: `false`

### Auth JWK URL

A URL that returns a JWK set used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-jwks-url`<br/>
env: `ZERO_AUTH_JWKS_URL`<br/>
required: `false`

### Auto Reset

Automatically wipe and resync the replica when replication is halted. This situation can occur for configurations in which the upstream database provider prohibits event trigger creation, preventing the zero-cache from being able to correctly replicate schema changes. For such configurations, an upstream schema change will instead result in halting replication with an error indicating that the replica needs to be reset. When auto-reset is enabled, zero-cache will respond to such situations by shutting down, and when restarted, resetting the replica and all synced clients. This is a heavy-weight operation and can result in user-visible slowness or downtime if compute resources are scarce.

flag: `--auto-reset`<br/>
env: `ZERO_AUTO_RESET`<br/>
default: `true`

### Auth Secret

A symmetric key used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-secret`<br/>
env: `ZERO_AUTH_SECRET`<br/>
required: `false`

### Change DB

The Postgres database used to store recent replication log entries, in order to sync multiple view-syncers without requiring multiple replication slots on the upstream database. If unspecified, the upstream-db will be used.

flag: `--change-db`<br/>
env: `ZERO_CHANGE_DB`<br/>
required: `false`

### Change Max Connections

The maximum number of connections to open to the change database. This is used by the change-streamer for catching up zero-cache replication subscriptions.

flag: `--change-max-conns`<br/>
env: `ZERO_CHANGE_MAX_CONNS`<br/>
default: `5`

### Change Streamer Mode
The mode for running or connecting to the change-streamer:

* `dedicated`: runs the change-streamer and shuts down when another
  change-streamer takes over the replication slot. This is appropriate in a
  single-node configuration, or for the replication-manager in a
  multi-node configuration.
* `discover`: connects to the change-streamer as internally advertised in the
  change-db. This is appropriate for the view-syncers in a multi-node

flag: `--change-streamer-mode`<br/>
env: `ZERO_CHANGE_STREAMER_MODE`<br/>
default: `dedicated`

### Change Streamer Port

The port on which the change-streamer runs. This is an internal protocol between the replication-manager and zero-cache, which runs in the same process in local development. If unspecified, defaults to --port + 1.

flag: `--change-streamer-port`<br/>
env: `ZERO_CHANGE_STREAMER_PORT`<br/>
required: `false`

### CVR DB

The Postgres database used to store CVRs. CVRs (client view records) keep track of the data synced to clients in order to determine the diff to send on reconnect. If unspecified, the upstream-db will be used.

flag: `--cvr-db`<br/>
env: `ZERO_CVR_DB`<br/>
required: `false`

### CVR Max Connections

The maximum number of connections to open to the CVR database. This is divided evenly amongst sync workers.

Note that this number must allow for at least one connection per sync worker, or zero-cache will fail to start. See num-sync-workers.

flag: `--cvr-max-conns`<br/>
env: `ZERO_CVR_MAX_CONNS`<br/>
default: `30`

### Initial Sync Table Copy Workers

The number of parallel workers used to copy tables during initial sync. Each worker copies a single table at a time, fetching rows in batches of `initial-sync-row-batch-size`.

flag: `--initial-sync-table-copy-workers`<br/>
env: `ZERO_INITIAL_SYNC_TABLE_COPY_WORKERS`<br/>
default: `5`

### Lazy Startup

Delay starting the majority of zero-cache until first request.

This is mainly intended to avoid connecting to Postgres replication stream until the first request is received, which can be useful i.e., for preview instances.

Currently only supported in single-node mode.

flag: `--lazy-startup`<br/>
env: `ZERO_LAZY_STARTUP`<br/>
default: `false`

### Litestream Executable

Path to the litestream executable. This option has no effect if litestream-backup-url is unspecified.

flag: `--litestream-executable`<br/>
env: `ZERO_LITESTREAM_EXECUTABLE`<br/>
required: `false`

### Litestream Config Path

Path to the litestream yaml config file. zero-cache will run this with its environment variables, which can be referenced in the file via `${ENV}` substitution, for example:

- ZERO_REPLICA_FILE for the db Path
- ZERO_LITESTREAM_BACKUP_LOCATION for the db replica url
- ZERO_LITESTREAM_LOG_LEVEL for the log Level
- ZERO_LOG_FORMAT for the log type

flag: `--litestream-config-path`<br/>
env: `ZERO_LITESTREAM_CONFIG_PATH`<br/>
default: `./src/services/litestream/config.yml`

### Litestream Log Level

flag: `--litestream-log-level`<br/>
env: `ZERO_LITESTREAM_LOG_LEVEL`<br/>
default: `warn`
values: `debug`, `info`, `warn`, `error`

### Litestream Backup URL

The location of the litestream backup, usually an s3:// URL. This is only consulted by the replication-manager. view-syncers receive this information from the replication-manager. 

flag: `--litestream-backup-url`<br/>
env: `ZERO_LITESTREAM_BACKUP_URL`<br/>
required: `false`

### Litestream Checkpoint Threshold MB

The size of the WAL file at which to perform an SQlite checkpoint to apply the writes in the WAL to the main database file. Each checkpoint creates a new WAL segment file that will be backed up by litestream. Smaller thresholds may improve read performance, at the expense of creating more files to download when restoring the replica from the backup.

flag: `--litestream-checkpoint-threshold-mb`<br/>
env: `ZERO_LITESTREAM_CHECKPOINT_THRESHOLD_MB`<br/>
default: `40`

### Litestream Incremental Backup Interval Minutes

The interval between incremental backups of the replica. Shorter intervals reduce the amount of change history that needs to be replayed when catching up a new view-syncer, at the expense of increasing the number of files needed to download for the initial litestream restore.

flag: `--litestream-incremental-backup-interval-minutes`<br/>
env: `ZERO_LITESTREAM_INCREMENTAL_BACKUP_INTERVAL_MINUTES`<br/>
default: `15`

### Litestream Snapshot Backup Interval Hours

The interval between snapshot backups of the replica. Snapshot backups make a full copy of the database to a new litestream generation. This improves restore time at the expense of bandwidth. Applications with a large database and low write rate can increase this interval to reduce network usage for backups (litestream defaults to 24 hours).

flag: `--litestream-snapshot-backup-interval-hours`<br/>
env: `ZERO_LITESTREAM_SNAPSHOT_BACKUP_INTERVAL_HOURS`<br/>
default: `12`

### Litestream Restore Parallelism

The number of WAL files to download in parallel when performing the initial restore of the replica from the backup.

flag: `--litestream-restore-parallelism`<br/>
env: `ZERO_LITESTREAM_RESTORE_PARALLELISM`<br/>
default: `48`

### Log Format

Use text for developer-friendly console logging and json for consumption by structured-logging services.

flag: `--log-format`<br/>
env: `ZERO_LOG_FORMAT`<br/>
default: `"text"`<br/>
values: `text`, `json`

### Log IVM Sampling

How often to collect IVM metrics. 1 out of N requests will be sampled where N is this value.

flag: `--log-ivm-sampling`<br/>
env: `ZERO_LOG_IVM_SAMPLING`<br/>
default: `5000`

### Log Level

Sets the logging level for the application.

flag: `--log-level`<br/>
env: `ZERO_LOG_LEVEL`<br/>
default: `"info"`<br/>
values: `debug`, `info`, `warn`, `error`

### Log Slow Hydrate Threshold

The number of milliseconds a query hydration must take to print a slow warning.

flag: `--log-slow-hydrate-threshold`<br/>
env: `ZERO_LOG_SLOW_HYDRATE_THRESHOLD`<br/>
default: `100`

### Log Slow Row Threshold

The number of ms a row must take to fetch from table-source before it is considered slow.

flag: `--log-slow-row-threshold`<br/>
env: `ZERO_LOG_SLOW_ROW_THRESHOLD`<br/>
default: `2`

### Log Trace Collector

The URL of the trace collector to which to send trace data. Traces are sent over http. Port defaults to 4318 for most collectors.

flag: `--log-trace-collector`<br/>
env: `ZERO_LOG_TRACE_COLLECTOR`<br/>
required: `false`

### Number of Sync Workers

The number of processes to use for view syncing. Leave this unset to use the maximum available parallelism. If set to 0, the server runs without sync workers, which is the configuration for running the replication-manager.

flag: `--num-sync-workers`<br/>
env: `ZERO_NUM_SYNC_WORKERS`<br/>
required: `false`

### Per User Mutation Limit Max

The maximum mutations per user within the specified windowMs.

flag: `--per-user-mutation-limit-max`<br/>
env: `ZERO_PER_USER_MUTATION_LIMIT_MAX`<br/>
required: `false`

### Per User Mutation Limit Window (ms)

The sliding window over which the perUserMutationLimitMax is enforced.

flag: `--per-user-mutation-limit-window-ms`<br/>
env: `ZERO_PER_USER_MUTATION_LIMIT_WINDOW_MS`<br/>
default: `60000`

### Port

The port for sync connections.

flag: `--port`<br/>
env: `ZERO_PORT`<br/>
default: `4848`

### Push URL

The URL of the API server to which zero-cache will push mutations. Required if you use [custom mutators](/docs/custom-mutators).

flag: `--push-url`<br/>
env: `ZERO_PUSH_URL`<br/>
required: `false`

### Query Hydration Stats

Track and log the number of rows considered by query hydrations which take longer than **log-slow-hydrate-threshold** milliseconds.

This is useful for debugging and performance tuning.

flag: `--query-hydration-stats`<br/>
env: `ZERO_QUERY_HYDRATION_STATS`<br/>
required: `false`

### Replica Vacuum Interval Hours

Performs a VACUUM at server startup if the specified number of hours has elapsed since the last VACUUM (or initial-sync). The VACUUM operation is heavyweight and requires double the size of the db in disk space. If unspecified, VACUUM operations are not performed.

flag: `--replica-vacuum-interval-hours`<br/>
env: `ZERO_REPLICA_VACUUM_INTERVAL_HOURS`<br/>
required: `false`

### Server Version

The version string outputted to logs when the server starts up.

flag: `--server-version`<br/>
env: `ZERO_SERVER_VERSION`<br/>
required: `false`

### Storage DB Temp Dir

Temporary directory for IVM operator storage. Leave unset to use `os.tmpdir()`.

flag: `--storage-db-tmp-dir`<br/>
env: `ZERO_STORAGE_DB_TMP_DIR`<br/>
required: `false`

### Target Client Row Count

A soft limit on the number of rows Zero will keep on the client. 20k is a good default value for most applications, and we do not recommend exceeding 100k. See [Client Capacity Management](/docs/reading-data#client-capacity-management) for more details.

flag: `--target-client-row-count`<br/>
env: `ZERO_TARGET_CLIENT_ROW_COUNT`<br/>
default: `20000`

### Task ID

Globally unique identifier for the zero-cache instance. Setting this to a platform specific task identifier can be useful for debugging. If unspecified, zero-cache will attempt to extract the TaskARN if run from within an AWS ECS container, and otherwise use a random string.

flag: `--task-id`<br/>
env: `ZERO_TASK_ID`<br/>
required: `false`

### Upstream Max Connections

The maximum number of connections to open to the upstream database for committing mutations. This is divided evenly amongst sync workers. In addition to this number, zero-cache uses one connection for the replication stream.

Note that this number must allow for at least one connection per sync worker, or zero-cache will fail to start. See num-sync-workers.

flag: `--upstream-max-conns`<br/>
env: `ZERO_UPSTREAM_MAX_CONNS`<br/>
default: `20`


--- react.mdx ---

Zero has built-in support for React. Here’s what basic usage looks like:

```tsx
import {useQuery, useZero} from '@rocicorp/zero/react';
import {type Schema} from './schema.ts';
import {type Mutators} from './mutators.ts';

function IssueList() {
  const z = useZero<Schema, Mutators>();
  let issueQuery = z.query.issue
    .related('creator')
    .related('labels')
    .limit(100);

  const userID = selectedUserID();

  if (userID) {
    issueQuery = issueQuery.where('creatorID', '=', userID);
  }

  const [issues, issuesDetail] = useQuery(issueQuery);

  return (
    <>
      <div>
        {issuesDetail.type === 'complete' ? 'Complete' : 'Partial'}
        results
      </div>
      <div>
        {issues.map(issue => (
          <IssueRow issue={issue} />
        ))}
      </div>
    </>
  );
}
```

## ZeroProvider

The `useZero` hook must be used within a `ZeroProvider` component. The
`ZeroProvider` component is responsible for creating and destroying
`Zero` instances reactively.

```tsx
import {SessionProvider} from 'my-auth-provider';
import {ZeroProvider} from '@rocicorp/zero/react';
import {type Schema, schema} from './schema.ts';
import {type Mutators, createMutators} from './mutators.ts';

export function Root() {
  const session = useSession();
  const {userID, authToken: auth} = session;
  const server = import.meta.env.VITE_PUBLIC_SERVER;

  const mutators = useMemo(() => {
    return createMutators(auth);
  }, [auth]);

  return (
    // ZeroProvider will reactively create and destroy Zero instances
    // as needed when props change.
    <ZeroProvider {...{userID, auth, schema, mutators, server}}>
      <App />
    </ZeroProvider>
  );
}
```

You can also pass a `Zero` instance to the `ZeroProvider` if you want to control the lifecycle of the `Zero` instance yourself:

```tsx
// ZeroProvider just sets up the context, it doesn't manage
// the lifecycle of the Zero instance.
<ZeroProvider zero={zero}>
  <App />
</ZeroProvider>
```


Complete quickstart here:

https://github.com/rocicorp/hello-zero


--- solidjs.mdx ---

Zero has built-in support for Solid. Here’s what basic usage looks like:

```tsx
import {createQuery} from '@rocicorp/zero/solid';

const issues = createQuery(() => {
  let issueQuery = z.query.issue
    .related('creator')
    .related('labels')
    .limit(100);
  const userID = selectedUserID();

  if (userID) {
    issueQuery = issueQuery.where('creatorID', '=', userID);
  }
  return issueQuery;
});
```

Complete quickstart here:

https://github.com/rocicorp/hello-zero-solid


--- community.mdx ---

Integrations with various tools, built by the Zero dev community.

If you have made something that should be here, send us a [pull request](https://github.com/rocicorp/zero-docs/pulls).

## UI Frameworks

- [One](https://onestack.dev/) is a full-stack React (and React Native!) framework with built-in Zero support.
- [zero-svelte](https://github.com/stolinski/zero-svelte) and [zero-svelte-query](https://github.com/RobertoSnap/zero-svelte-query) are two different approaches to Zero bindings for Svelte.
- [zero-vue](https://github.com/danielroe/zero-vue) adds Zero bindings to Vue.
- [zero-astro](https://github.com/ferg-cod3s/zero-astro) adds Zero bindings to Astro.

## Database Tools

- [drizzle-zero](https://github.com/BriefHQ/drizzle-zero) generates Zero schemas from Drizzle.
- [prisma-generator-zero](https://github.com/passionfroot/prisma-generator-zero) generates Zero schemas from Prisma.

## Miscellaneous

- [undo](https://github.com/rocicorp/undo) is a simple undo/redo library that was originally built for Replicache, but works just as well with Zero.


--- debug/inspector.mdx ---

The Zero instance provides an API to gather information about the client's current state, such as:

- All active queries
- Query TTL
- Active clients
- Client database contents

This can help figuring out why you hit loading states, how many queries are active at a time, if you have any resource leaks due to failing to clean up queries or if expected data is missing on the client.

## Creating an Inspector

Each `Zero` instance has an `inspect` method that will return the inspector. The `inspect` method is asynchronous because it performs lazy loading of inspect-only related code.

For convenience, the active `Zero` instance is automatically exposed at `__zero`, so you can access the inspector in the console like this:

```ts
const inspector = await __zero.inspect();
```

<Note type="warning" slug="code-splitting">
  Ensure that code splitting is enabled when bundling your app to prevent
  loading inspect-related code by default. The `inspect` API is intended for
  debugging purposes only and should not be used in production applications. It
  is not efficient and communicates directly with the zero-cache via RPC over a
  web socket.

If you are using React, you can use
[`React.lazy`](https://react.dev/reference/react/lazy) to dynamically load
components that depend on the `inspect` API.

</Note>

Once you have an inspector you can inspect the current client and client group.

<Note heading="Clients and Client Groups">
  The client group represents a web browser profile. A client represents an
  instance of `Zero` within that profile. If a user has Chrome open with 5
  different tabs, each with one `Zero` instance, there will be 1 client group
  and 5 clients.
</Note>

For example to see active queries for the current client:

```ts
console.table(await inspector.client.queries());
```

To inspect other clients within the group:

```ts
const allClients = await inspector.clients();
```

## Dumping Data

In addition to information about queries, you can see the contents of the client side database.

```ts
const inspector = await zero.inspect();
const client = inspector.client;

// All raw k/v data currently synced to client
console.log('client map:');
console.log(await client.map());

// kv table extracted into tables
// This is same info that is in z.query[tableName].run()
for (const tableName of Object.keys(schema.tables)) {
  console.log(`table ${tableName}:`);
  console.table(await client.rows(tableName));
}
```


--- debug/permissions.mdx ---

Given that permissions are defined in their own file and internally applied to queries, it might be hard to figure out if or why a permission check is failing.

## Read Permissions

You can use the `analyze-query` utility with the `--apply-permissions` flag to see the complete query Zero runs, including read permissions.

```bash
npx analyze-query
  --schema-path='./shared/schema.ts'
  --query='issue.related("comments")'
  --apply-permissions
  --auth-data='{"userId":"user-123"}'
```

If the result looks right, the problem may be that Zero is not receiving the `AuthData` that you think it is. You can retrieve a query hash from websocket or server logs, then ask Zero for the details on that specific query.

Run this command with the same environment you run `zero-cache` with. It will use your `upstream` or `cvr` configuration to look up the query hash in the cvr database.

```bash
npx analyze-query
  --schema-path='./shared/schema.ts'
  --hash='3rhuw19xt9vry'
  --apply-permissions
  --auth-data='{"userId":"user-123"}'
```

<Note type="note" slug="decompilation">
  The printed query can be different than the source ZQL string, because it is rebuilt from the query AST. But it should be logically equivalent to the query you wrote.
</Note>

## Write Permissions

Look for a `WARN` level log in the output from `zero-cache` like this:

```
Permission check failed for {"op":"update","tableName":"message",...}, action update, phase preMutation, authData: {...}, rowPolicies: [...], cellPolicies: []
```

Zero prints the row, auth data, and permission policies that was applied to any failed writes.

<Note slug="ast-format">
  The ZQL query is printed in AST format. See [Query ASTs](./query-asts) to
  convert it to a more readable format.
</Note>


--- debug/slow-queries.mdx ---

In the `zero-cache` logs, you may see statements indicating a query is slow:

```shell
{
    "level": "DEBUG",
    "worker": "syncer",
    "component": "view-syncer",
    "hydrationTimeMs": 1339,
    "message": "Total rows considered: 146"
  },
```

or:

```shell
hash=3rhuw19xt9vry transformationHash=1nv7ot74gxfl7
Slow query materialization 325.46865100000286
```

Or, you may just notice queries taking longer than expected in the UI.

Here are some tips to help debug such slow queries.

## Check `ttl`

If you are seeing unexpected UI flicker when moving between views, it is likely that the queries backing these views have the default `ttl` of `never`. Set the `ttl` to some longer value to [keep data cached across navigations](https://zero.rocicorp.dev/docs/reading-data#background-queries).

You may alternately want to [preload some data](https://zero.rocicorp.dev/docs/reading-data#preloading) at app startup.

## Check Storage

`zero-cache` is effectively a database. It requires fast (low latency and high bandwidth) disk access to perform well. If you're running on network attached storage with high latency, or on AWS with low IOPS, then this is the most likely culprit.

The default deployment of Zero currently uses Fargate which scales IOPS with vCPU. Increasing the vCPU will increase storage throughput and likely resolve the issue.

Fly.io provides physically attached SSDs, even for their smallest VMs. Deploying zero-cache there (or any other provider that offers physically attached SSDs) is another option.

## Locality

If you see log lines like:

```shell
flushed cvr ... (124ms)
```

this indicates that `zero-cache` is likely deployed too far away from your [CVR database](../deployment#architecture). If you did not configure a CVR database URL then this will be your product's Postgres DB. A slow CVR flush can slow down Zero, since it must complete the flush before sending query result(s) to clients.

Try moving `zero-cache` to be deployed as close as possible to the CVR database.

## Query Plan

If neither (1) nor (2) is a problem, then the query itself is the most likely culprit. The `@rocicorp/zero` package ships with a query analyzer to help debug this.

The analyzer should be run in the directory that contains the `.env` file for `zero-cache` as it will use the `.env` file to find your replica.

Example:

```shell
npx analyze-query \
  --schema-path=./shared/schema.ts \
  --query='issue.related("comments")'
```

This will output the query plan and time to execute each phase of that plan.

Note that query performance can also be affected by read permissions. See [Debugging Permissions](./permissions) for information on how to analyze queries with read permissions applied.

## /statz

`zero-cache` makes some internal health statistics available via the `/statz` endpoint of `zero-cache`. In order to access this, you must configure an [admin password](/docs/zero-cache-config#admin-password).


--- debug/replication.mdx ---

## Resetting

During development we all do strange things (unsafely changing schemas, removing files, etc.). If the replica ever gets wedged (stops replicating, acts strange) you can wipe it and start over.

- If you copied your setup from `hello-zero` or `hello-zero-solid`, you can also run `npm run dev:clean`
- Otherwise you can run `rm /tmp/my-zero-replica.db*` (see your `.env` file for the replica file location) to clear the contents of the replica.

It is always safe to wipe the replica. Wiping will have no impact on your upstream database. Downstream zero-clients will get re-synced when they connect.

## Inspecting

For data to be synced to the client it must first be replicated to `zero-cache`. You can check the contents of `zero-cache` via:

```bash
$ npx @rocicorp/zero-sqlite3 /tmp/my-zero-replica.db
```

<Note slug="sqlite-version" emoji="💡" heading="Zero uses a the bedrock version of SQLite">
Zero uses a different version of SQLite that runs in WAL2 mode, which means the database files cannot be opened with standard SQLite tools.

To inspect your Zero database, you have two options:

1. Use our pre-compiled SQLite build `@rocicorp/zero-sqlite3` as described above
2. Build SQLite from the SQLite `bedrock` branch yourself
</Note>

This will drop you into a `sqlite3` shell with which you can use to explore the contents of the replica.

```sql
sqlite> .tables
_zero.changeLog          emoji                    viewState
_zero.replicationConfig  issue                    zero.permissions
_zero.replicationState   issueLabel               zero.schemaVersions
_zero.runtimeEvents      label                    zero_0.clients
_zero.versionHistory     user
comment                  userPref
sqlite> .mode qbox
sqlite> SELECT * FROM label;
┌─────────────────────────┬──────────────────────────┬────────────┐
│           id            │           name           │ _0_version │
├─────────────────────────┼──────────────────────────┼────────────┤
│ 'ic_g-DZTYDApZR_v7Cdcy' │ 'bug'                    │ '4ehreg'   │
...
```

## Miscellaneous

If you see `FATAL:  sorry, too many clients already` in logs, it’s because you have two zero-cache instances running against dev. One is probably in a background tab somewhere. In production, `zero-cache` can run horizontally scaled but on dev it doesn’t run in the config that allows that.


--- debug/query-asts.mdx ---

An AST (Abstract Syntax Tree) is a representation of a query that is used internally by Zero. It is not meant to be human readable, but it sometimes shows up in logs and other places.

If you need to read one of these, save the AST to a json file. Then run the following command:

```bash
cat ast.json | npx ast-to-zql
```

The returned ZQL query will be using server names, rather than client names, to identify columns and tables.
If you provide the schema file as an option you will get mapped back to client names:

```bash
cat ast.json | npx ast-to-zql --schema schema.ts
```

This comes into play if, in your schema.ts, you use the `from` feature to have different names on the client than your backend DB.

<Note slug="decompilation" type="note">
  The `ast-to-zql` process is a de-compilation of sorts. Given that, the ZQL
  string you get back will not be identical to the one you wrote in your
  application. Regardless, the queries will be semantically equivalent.
</Note>


--- debug/otel.mdx ---

The `zero-cache` service embeds the [JavaScript OTLP Exporter](https://opentelemetry.io/docs/languages/js/) and can send logs, traces, and metrics to any [standard otel collector](https://opentelemetry.io/).

To enable otel, set the following environment variables then run `zero-cache` as normal:

```sh
OTEL_EXPORTER_OTLP_ENDPOINT="<your otel endpoint>"
OTEL_EXPORTER_OTLP_HEADERS="<auth headers from your otel collector>"
OTEL_RESOURCE_ATTRIBUTES="<resource attributes from your otel collector>"
OTEL_NODE_RESOURCE_DETECTORS="env,host,os"
```

## Grafana Cloud Walkthrough

Here are instructions to setup [Grafana Cloud](https://grafana.com/oss/grafana/), but the setup for other otel collectors should be similar.

1. Sign up for [Grafana Cloud (Free Tier)](https://grafana.com/auth/sign-up/create-user?pg=login)
2. Click Connections > Add Connection in the left sidebar
  ![add-connection](/images/debugging/otel/add-connection.png)
3. Search for "OpenTelemetry" and select it
4. Click "Quickstart"
  ![quickstart](/images/debugging/otel/quickstart.png)
5. Select "JavaScript"
  ![javascript](/images/debugging/otel/javascript.png)
6. Create a new token
7. Copy the environment variables into your `.env` file or similar
  ![copy-env](/images/debugging/otel/env.png)
8. Start `zero-cache`
9. Look for logs under "Drilldown" > "Logs" in left sidebar


--- roadmap.mdx ---

## Alpha (EOY ‘24)

- ~~Schema migration~~
- ~~Write permissions~~
- ~~Solid support~~
- ~~Replica sqlite files not browsable with standard sqlite3 program~~
- ~~Relationship filters - currently you can put relationships in the ‘select’ part of the query, but not the ‘where’ part. Relationship filters commonly needed to, i.e., find all issues with particular label.~~
- ~~Multi-column primary keys~~
- ~~Read permissions~~
- ~~Docs for easily deploying Zero on your own AWS or Fly.io account~~
- ~~Up to 20MB client-side and 1GB server-side per-replica~~

## Beta (Q2 ‘25)

- ~~Custom mutators~~
- Cell-level read permissions (already exist for write)
- First-class support for React Native
- ~~Ability to wait for authoritative results~~
- Aggregations (count, sum, min, max, group-by, etc)
- Consistency.
  - See: [Consistency](/docs/reading-data#consistency).
  - This will also improve startup perf since apps won’t have to be so conservative in what they preload:
    ~~Cache size management: evict things from client-side cache to stay under size~~
- Reduce zero client bundle size to < 40KB
- Up to 20 MB client-side and 100 GB server-side per-replica

## GA

- Vector-based text search
- Extensive testing using randomized query generation and dst
- External audit of design and impl
- Ability to lock queries down to only expected forms for security
- Additional databases beside Postgres
- SaaS


--- release-notes/index.mdx ---

- [Zero 0.21: PG arrays, TanStack starter, and more](/docs/release-notes/0.21)
- [Zero 0.20: Full Supabase support, performance improvements](/docs/release-notes/0.20)
- [Zero 0.19: Many, many bugfixes and cleanups](/docs/release-notes/0.19)
- [Zero 0.18: Custom Mutators](/docs/release-notes/0.18)
- [Zero 0.17: Background Queries](/docs/release-notes/0.17)
- [Zero 0.16: Lambda-Based Permission Deployment](/docs/release-notes/0.16)
- [Zero 0.15: Live Permission Updates](/docs/release-notes/0.15)
- [Zero 0.14: Name Mapping and Multischema](/docs/release-notes/0.14)
- [Zero 0.13: Multinode and SST](/docs/release-notes/0.13)
- [Zero 0.12: Circular Relationships](/docs/release-notes/0.12)
- [Zero 0.11: Windows](/docs/release-notes/0.11)
- [Zero 0.10: Remove Top-Level Await](/docs/release-notes/0.10)
- [Zero 0.9: JWK Support](/docs/release-notes/0.9)
- [Zero 0.8: Schema Autobuild, Result Types, and Enums](/docs/release-notes/0.8)
- [Zero 0.7: Read Perms and Docker](/docs/release-notes/0.7)
- [Zero 0.6: Relationship Filters](/docs/release-notes/0.6)
- [Zero 0.5: JSON Columns](/docs/release-notes/0.5)
- [Zero 0.4: Compound Filters](/docs/release-notes/0.4)
- [Zero 0.3: Schema Migrations and Write Perms](/docs/release-notes/0.3)
- [Zero 0.2: Skip Mode and Computed PKs](/docs/release-notes/0.2)
- [Zero 0.1: First Release](/docs/release-notes/0.1)


--- reporting-bugs.mdx ---

## zbugs

You can use [zbugs](https://bugs.rocicorp.dev/)! (password: `zql`)

Our own bug tracker built from the ground up on Zero.

<ImageLightbox src="/images/reporting-bugs/zbugs.png" alt="zbugs" />

## Discord

Alternately just pinging us on Discord is great too.

<Button variant="outline" size="default" asChild>
  <a href="https://discord.rocicorp.dev">Join us on Discord</a>
</Button>


--- open-source.mdx ---

Specifically, the Zero client and server are Apache-2 licensed. You can use, modify, host, and distribute them freely:

https://github.com/rocicorp/mono/blob/main/LICENSE

## Business Model

We plan to commercialize Zero in the future by offering a hosted `zero-cache` service for people who do not want to run it themselves. We expect to charge prices for this rougly comparable to today's database hosting services. We'll also offer white-glove service to help enterprises run `zero-cache` within their own infrastructure.

These plans may change as we develop Zero further. For example, we may also build closed-source companion software – similar to how Docker, Inc. charges for team access to Docker Desktop.

But we have no plans to ever change the licensing of the core product: We're building a general-purpose sync engine for the entire web, and we can only do that if the core remains completely open.


--- llms.mdx ---

Are you an LLM?

Do you like long walks through vector space and late-night tokenization?

Or maybe you're a friend of an LLM, just trying to make life a little easier for the contextually challenged?

Either way, you're in the right place! Stream on over to [llms.txt](/llms.txt) for the text-only version of these docs.


--- custom-mutators.mdx ---

_Custom Mutators_ are a new way to write data in Zero that is much more powerful than the original ["CRUD" mutator API](./writing-data).

Instead of having only the few built-in `insert`/`update`/`delete` write operations for each table, custom mutators allow you to _create your own write operations_ using arbitrary code. This makes it possible to do things that are impossible or awkward with other sync engines.

For example, you can create custom mutators that:

- Perform arbitrary server-side validation
- Enforce fine-grained permissions
- Send email notifications
- Query LLMs
- Use Yjs for collaborative editing
- … and much, _much_ more – custom mutators are just code, and they can do anything code can do!

Despite their increased power, custom mutators still participate fully in sync. They execute instantly on the local device, immediately updating all active queries. They are then synced in the background to the server and to other clients.

<Note
  slug="deprecating-crud"
  heading="Custom mutators will eventually become Zero's only write API"
>
  We're still refining the design of custom mutators. During this phase, the old
  CRUD mutators will continue to work. But we do want to deprecate CRUD
  mutators, and eventually remove them. So please try out custom mutators and
  [let us know](https://discord.rocicorp.dev/) how they work for you, and what
  improvements you need before the cutover.
</Note>

## Understanding Custom Mutators

### Architecture

Custom mutators introduce a new _server_ component to the Zero architecture.

![diagram](/images/custom-mutators/overview.png)

This server is implemented by you, the developer. It's typically just your existing backend, where you already put auth or other server-side functionality.

The server can be a serverless function, a microservice, or a full stateful server. The only real requirement is that it expose a special _push endpoint_ that `zero-cache` can call to process mutations. This endpoint implements the [push protocol](#custom-push-implementation) and contains your custom logic for each mutation.

Zero provides utilities in `@rocicorp/zero` that make it really easy implement this endpoint in TypeScript. But you can also implement it yourself if you want. As long as your endpoint fulfills the push protocol, `zero-cache` doesn't care. You can even write it in a different programming language.

### What Even is a Mutator?

Zero's custom mutators are based on [_server reconciliation_](https://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html) – a technique for robust sync that has been used by the video game industry for decades.

<Note emoji="🌈" heading="The more you know">
  Our previous sync engine, [Replicache](https://replicache.dev/), also used
  server reconciliation. The ability to implement arbitrary mutators was one of
  Replicache's most popular features. Custom mutators bring this same power to
  Zero, but with a much better developer experience.
</Note>

A custom mutator is just a function that runs within a database transaction, and which can read and write to the database. Here's an example of a very simple custom mutator written in TypeScript:

```ts
async function updateIssue(
  tx: Transaction,
  {id, title}: {id: string; title: string},
) {
  // Validate title length.
  if (title.length > 100) {
    throw new Error(`Title is too long`);
  }

  await tx.mutate.issue.update({id, title});
}
```

Each custom mutator gets **two implementations**: one on the client and one on the server.

The client implementation must be written in TypeScript against the Zero `Transaction` interface, using [ZQL](#read-data-on-the-client) for reads and a [CRUD-style API](#write-data-on-the-client) for writes.

The server implementation runs on your server, in your push endpoint, against your database. In principle, it can be written in any language and use any data access library. For example you could have the following Go-based server implementation of the same mutator:

```go
func updateIssueOnServer(tx *sql.Tx, id string, title string) error {
  // Validate title length.
  if len(title) > 100 {
    return errors.New("Title is too long")
  }

  _, err := tx.Exec("UPDATE issue SET title = $1 WHERE id = $2", title, id)
  return err
}
```

In practice however, most Zero apps use TypeScript on the server. For these users we provide a handy `ServerTransaction` that implements ZQL against Postgres, so that you can share code between client and server mutators naturally.

So on a TypeScript server, that server mutator can just be:

```ts
async function updateIssueOnServer(
  tx: ServerTransaction,
  {id, title},
  {id: string, title: string},
) {
  // Delegate to client mutator.
  // The `ServerTransaction` here has a different implementation
  // that runs the same ZQL queries against Postgres!
  await updateIssue(tx, {id, title});
}
```

<Note slug="code-sharing" heading="Code sharing in mutators is optional">
Even in TypeScript, you can do as little or as much code sharing as you like. In your server mutator, you can [use raw SQL](#dropping-down-to-raw-sql), any data access libraries you prefer, or add as much extra server-specific logic as you need.

Reusing ZQL on the server is a handy – and we expect frequently used – option, but not a requirement.

</Note>

### Server Authority

You may be wondering what happens if the client and server mutators implementations don't match.

Zero is an example of a _server-authoritative_ sync engine. This means that the server mutator always takes precedence over the client mutator. The result from the client mutator is considered _speculative_ and is discarded as soon as the result from the server mutator is known. This is a very useful feature: it enables server-side validation, permissions, and other server-specific logic.

Imagine that you wanted to use an LLM to detect whether an issue update is spammy, rather than a simple length check. We can just add that to our server mutator:

```ts
async function updateIssueOnServer(
  tx: ServerTransaction,
  {id, title}: {id: string; title: string},
) {
  const response = await llamaSession.prompt(
    `Is this title update likely spam?\n\n${title}\n\nResponse "yes" or "no"`,
  );
  if (/yes/i.test(response)) {
    throw new Error(`Title is likely spam`);
  }

  // delegate rest of implementation to client mutator
  await updateIssue(tx, {id, title});
}
```

If the server detects that the mutation is spammy, the client will see the error message and the mutation will be rolled back. If the server mutator succeeds, the client mutator will be rolled back and the server result will be applied.

### Life of a Mutation

Now that we understand what client and server mutations are, let's walk through how they work together with Zero to sync changes from a source client to the server and then other clients:

1. When you call a custom mutator on the client, Zero runs your client-side mutator immediately on the local device, updating all active queries instantly.
2. In the background, Zero then sends a _mutation_ (a record of the mutator having run with certain arguments) to your server's push endpoint.
3. Your push endpoint runs the [push protocol](#custom-push-implementation), executing the server-side mutator in a transaction against your database and recording the fact that the mutation ran. Optionally, you use our `PushProcessor` class to handle this for you, but you can also implement it yourself.
4. The changes to the database are replicated to `zero-cache` as normal.
5. `zero-cache` calculates the updates to active queries and sends rows that have changed to each client. It also sends information about the mutations that have been applied to the database.
6. Clients receive row updates and apply them to their local cache. Any pending mutations which have been applied to the server have their local effects rolled back.
7. Client-side queries are updated and the user sees the changes.

## Using Custom Mutators

### Registering Client Mutators

By convention, the client mutators are defined with a function called `createMutators` in a file called `mutators.ts`:

```ts
// mutators.ts
import {CustomMutatorDefs} from '@rocicorp/zero';
import {schema} from './schema';

export function createMutators() {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Validate title length. Legacy issues are exempt.
        if (title.length > 100) {
          throw new Error(`Title is too long`);
        }
        await tx.mutate.issue.update({id, title});
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

The `mutators.ts` convention allows mutator implementations to be easily [reused server-side](#setting-up-the-server). The `createMutators` function convention is used so that we can pass authentication information in to [implement permissions](#permissions).

You are free to make different code layout choices – the only real requirement is that you register your map of mutators in the `Zero` constructor:

```ts
// main.tsx
import {Zero} from '@rocicorp/zero';
import {schema} from './schema';
import {createMutators} from './mutators';

const zero = new Zero({
  schema,
  mutators: createMutators(),
});
```

### Write Data on the Client

The `Transaction` interface passed to client mutators exposes the same `mutate` API as the existing [CRUD-style mutators](./writing-data):

```ts
async function myMutator(tx: Transaction) {
  // Insert a new issue
  await tx.mutate.issue.insert({
    id: 'issue-123',
    title: 'New title',
    description: 'New description',
  });

  // Upsert a new issue
  await tx.mutate.issue.upsert({
    id: 'issue-123',
    title: 'New title',
    description: 'New description',
  });

  // Update an issue
  await tx.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  });

  // Delete an issue
  await tx.mutate.issue.delete({
    id: 'issue-123',
  });
}
```

See [the CRUD docs](./writing-data) for detailed semantics on these methods.

### Read Data on the Client

You can read data within a client mutator using [ZQL](./reading-data):

```ts
export function createMutators() {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Read existing issue
        const prev = await tx.query.issue.where('id', id).one();

        // Validate title length. Legacy issues are exempt.
        if (!prev.isLegacy && title.length > 100) {
          throw new Error(`Title is too long`);
        }

        await tx.mutate.issue.update({id, title});
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

You have the full power of ZQL at your disposal, including relationships, filters, ordering, and limits.

Reads and writes within a mutator are transactional, meaning that the datastore is guaranteed to not change while your mutator is running. And if the mutator throws, the entire mutation is rolled back.

<Note type="note" heading="Run in Custom Mutators">
Outside of mutators, the `run()` method has a [`type` parameter](reading-data#running-queries-once) that can be used to wait for server results.

This parameter isn't supported within mutators, because waiting for server results makes no sense in an optimistic mutation – it defeats the purpose of running optimistically to begin with.

When a mutator runs on the client (`tx.location === "client"`), ZQL reads only return data already cached on the client. When mutators run on the server (`tx.location === "server"`), ZQL reads always return all data.

You can use `run()` within custom mutators, but the `type` argument does nothing. In the future, passing `type` in this situation will throw an error.

</Note>

### Invoking Client Mutators

Once you have registered your client mutators, you can call them from your client-side application:

```ts
zero.mutate.issue.update({
  id: 'issue-123',
  title: 'New title',
});
```

The result of a call to a mutator is a `Promise`. You do not usually need to `await` this promise as Zero mutators run very fast, usually completing in a tiny fraction of one frame.

However because mutators ocassionally need to access browser storage, they are technically `async`. Reading a row that was written by a mutator immediately after it is written may not return the new data, because the mutator may not have completed writing to storage yet.

### Waiting for Mutator Result

We typically recommend that you "fire and forget" mutators.

Optimistic mutations make sense when the common case is that a mutation succeeds. If a mutation frequently fails, then showing the user an optimistic result doesn't make sense, because it will likely be wrong.

That said there are cases where it is useful to know when a write succeeded on either the client or server.

One example is if you need to read a row directly after writing it. Zero's local writes are very fast (almost always < 1 frame), but because Zero is backed by IndexedDB, writes are still _technically_ asynchronous and reads directly after a write may not return the new data.

You can use the `.client` promise in this case to wait for a write to complete on the client side:

```ts
try {
  const write = zero.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  });

  // issue-123 not guaranteed to be present here. read1 may be undefined.
  const read1 = await zero.query.issue.where('id', 'issue-123').one();

  // Await client write – almost always less than 1 frame, and same
  // macrotask, so no browser paint will occur here.
  await write.client;

  // issue-123 definitely can be read now.
  const read2 = await zero.query.issue.where('id', 'issue-123').one();
} catch (e) {
  console.error('Mutator failed on client', e);
}
```

You can also wait for the server write to succeed:

```ts
try {
  await zero.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  }).server;

  // issue-123 is written to server
} catch (e) {
  console.error('Mutator failed on client or server', e);
}
```

If the client-side mutator fails, the `.server` promise is also rejected with the same error. You don't have to listen to both promises, the server promise covers both cases.

<Note heading="Returning data from mutators" slug="returning-data">
  There is not yet a way to return data from mutators in the success case – the type of `.clent` and `.server` is always `Promise<void>`. [Let us know](https://discord.rocicorp.dev/) if you need this.
</Note>

### Setting Up the Server

You will need a server somewhere you can run an endpoint on. This is typically a serverless function on a platform like Vercel or AWS but can really be anything.

Set the push URL with the [`ZERO_PUSH_URL` env var or `--push-url`](./zero-cache-config#push-url).

If there is per-client configuration you need to send to the push endpoint, you can do that with `push.queryParams`:

```ts
const z = new Zero({
  push: {
    queryParams: {
      workspaceID: '42',
    },
  },
});
```

The push endpoint receives a `PushRequest` as input describing one or more mutations to apply to the backend, and must return a `PushResponse` describing the results of those mutations.

If you are implementing your server in TypeScript, you can use the `PushProcessor` class to trivially implement this endpoint. Here’s an example in a [Hono](https://hono.dev/) app:

```ts
import {Hono} from 'hono';
import {handle} from 'hono/vercel';
import {
  PushProcessor,
  ZQLDatabase,
  PostgresJSConnection,
} from '@rocicorp/zero/pg';
import postgres from 'postgres';
import {schema} from '../shared/schema';
import {createMutators} from '../shared/mutators';

// PushProcessor is provided by Zero to encapsulate a standard
// implementation of the push protocol.
const processor = new PushProcessor(
  new ZQLDatabase(
    new PostgresJSConnection(postgres(process.env.ZERO_UPSTREAM_DB! as string)),
    schema,
  ),
);

export const app = new Hono().basePath('/api');

app.post('/push', async c => {
  const result = await processor.process(createMutators(), c.req.raw);
  return await c.json(result);
});

export default handle(app);
```

`PushProcessor` depends on an abstract `Database`. This allows it to implement the push algorithm against any database.

`@rocicorp/zero/pg` includes a `ZQLDatabase` implementation of this interface backed by Postgres. The implementation allows the same mutator functions to run on client and server, by providing an implementation of the ZQL APIs that custom mutators run on the client.

`ZQLDatabase` in turn relies on an abstract `DBConnection` that provides raw access to a Postgres database. This allows you to use any Postgres library you like, as long as you provide a `DBConnection` implementation for it. The `PostgresJSConnection` class implements `DBConnection` for the excellent [`postgres.js`](https://www.npmjs.com/package/postgres) library to connect to Postgres.

To reuse the client mutators exactly as-is on the server just pass the result of the same `createMutators` function to `PushProcessor`.

### Server Error Handling

The `PushProcessor` in `@rocicorp/zero/pg` skips any mutations that throw:

```ts
app.post('/push', async c => {
  const result = await processor.process({
    issue: {
      update: async (tx, data) => {
        // The mutation is skipped and the next mutation runs as normal.
        throw new Error('bonk');
      },
    },
  }, ...);
  return await c.json(result);
})
```

`PushProcessor` catches such errors and turns them into a structured response that gets sent back to the client. You can [recover the errors](#waiting-for-mutator-result) and show UI if you want.

It is also of course possible for the entire push endpoint to return an HTTP error, or to not reply at all:

```ts
app.post('/push', async c => {
  // This will cause the client to resend all queued mutations.
  throw new Error('zonk');
  const result = await processor.process({
    // ...
  }, ...);
  return await c.json(result);
})
```

If Zero recieves any response from the push endpoint other than HTTP 200, 401, or 403, it will disconnect, wait a few moments, reconnect, and then retry all unprocessed mutations.

If Zero receives HTTP 401 or 403, the client [refreshes the `auth` token](./auth#refresh) if possible, then retries all queued mutations.

If you want a different behavior, it is possible to [implement your own](#custom-push-implementation) `PushProcessor` and handle errors differently.

### Server-Specific Code

To implement server-specific code, just run different mutators in your push endpoint!

An approach we like is to create a separate `server-mutators.ts` file that wraps the client mutators:

```ts
// server-mutators.ts
import {CustomMutatorDefs} from '@rocicorp/zero';
import {schema} from './schema';

export function createMutators(
  clientMutators: CustomMutatorDefs<typeof schema>,
) {
  return {
    // Reuse all client mutators except the ones in `issue`
    ...clientMutators,

    issue: {
      // Reuse all issue mutators except `update`
      ...clientMutators.issue,

      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Call the shared mutator first
        await clientMutators.issue.update(tx, {id, title});

        // Record a history of this operation happening in an audit
        // log table.
        await tx.mutate.auditLog.insert({
          // Assuming you have an audit log table with fields for
          // `issueId`, `action`, and `timestamp`.
          issueId: id,
          action: 'update-title',
          timestamp: new Date().toISOString(),
        });
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

For simple things, we also expose a `location` field on the transaction object that you can use to branch your code:

```ts
myMutator: (tx) => {
  if (tx.location === 'client') {
    // Client-side code
  } else {
    // Server-side code
  }
},
```

### Permissions

Because custom mutators are just arbitrary TypeScript functions, there is no need for a special permissions system. Therefore, you won't use Zero's [write permissions](./permissions) when you use custom mutators.

<Note slug="read-permissions" emoji="💡" heading="You do still need *read* permissions">
  When using custom mutators you will have no [`insert`](permissions#insert-permissions), [`update`](permissions#update-permissions), or [`delete`](permissions#delete-permissions) permissions. You will still have [`select`](permissions#select-permissions) permissions, however.

We hope to build [custom queries](https://bugs.rocicorp.dev/issue/3453) next – a read analog to custom mutators. If we succeed, Zero's permission system will go away completely 🤯.

</Note>

In order to do permission checks, you'll need to know what user is making the request. You can pass this information to your mutators by adding a `AuthData` parameter to the `createMutators` function:

```ts
type AuthData = {
  sub: string;
};

export function createMutators(authData: AuthData | undefined) {
  return {
    issue: {
      launchMissiles: async (tx, args: {target: string}) => {
        if (!authData) {
          throw new Error('Users must be logged in to launch missiles');
        }

        const hasPermission = await tx.query.user
          .where('id', authData.sub)
          .whereExists('permissions', q => q.where('name', 'launch-missiles'))
          .one();
        if (!hasPermission) {
          throw new Error('User does not have permission to launch missiles');
        }
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

The `AuthData` parameter can be any data required for authorization, but is typically just the decoded JWT:

```ts
// app.tsx
const zero = new Zero({
  schema,
  auth: encodedJWT,
  mutators: createMutators(decodedJWT),
});

// hono-server.ts
const processor = new PushProcessor(
  schema,
  connectionProvider(postgres(process.env.ZERO_UPSTREAM_DB as string)),
);
processor.process(
  createMutators(decodedJWT),
  c.req.query(),
  await c.req.json(),
);
```

### Dropping Down to Raw SQL

The `ServerTransaction` interface has a `dbTransaction` property that exposes the underlying database connection. This allows you to run raw SQL queries directly against the database.

This is useful for complex queries, or for using Postgres features that Zero doesn't support yet:

```ts
markAllAsRead: async(tx: Transaction, {userId: string}) {
  // shared stuff ...

  if (tx.location === 'server') {
    // `tx` is now narrowed to `ServerTransaction`.
    // Do special server-only stuff with raw SQL.
    await tx.dbTransaction.query(
      `
      UPDATE notification
      SET read = true
      WHERE user_id = $1
    `,
      [userId],
    );
  }
}
```

As a convenience we also expose a special `CustomMutatorDefs` for use with `server-mutators.ts` that
sets all the mutators to `ServerTransaction` by default:

```ts
// server-mutators.ts
import type {
  CustomMutatorDefs,  // Special server-side CustomMutatorDefs
  PostgresJSTransaction,
  ServerTransaction,
} from '@rocicorp/zero/pg';
import {Schema} from './schema';

export function createMutators(clientMutators: CustomMutatorDefs<Schema>) {
  return {
    // Reuse all client mutators except the ones in `issue`
    ...clientMutators,

    issue: {
      // Reuse all issue mutators except `update`
      ...clientMutators.issue,

      markAllAsRead: async (tx, {userId: string}) {
        // No narrowing necessary – tx is already a `ServerTransaction`
        // assert(tx.location === 'server');
        await tx.dbTransaction.query(
          `
          UPDATE notification
          SET read = true
          WHERE user_id = $1
        `,
          [userId],
        );
      },
    }
  } as const satisfies CustomMutatorDefs<
    ServerTransaction<Schema, PostgresJSTransaction>
  >;
}

```

### Notifications and Async Work

It is bad practice to hold open database transactions while talking over the network, for example to send notifications. Instead, you should let the db transaction commit and do the work asynchronously.

There is no specific support for this in custom mutators, but since mutators are just code, it’s easy to do:

```ts
// server-mutators.ts
export function createMutators(
  authData: AuthData,
  asyncTasks: Array<() => Promise<void>>,
) {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        await tx.mutate.issue.update({id, title});

        asyncTasks.push(async () => {
          await sendEmailToSubscribers(args.id);
        });
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

Then in your push handler:

```ts
app.post('/push', async c => {
  const asyncTasks: Array<() => Promise<void>> = [];
  const result = await processor.process(
    createMutators(authData, asyncTasks),
    c.req.query(),
    await c.req.json(),
  );

  await Promise.all(asyncTasks.map(task => task()));
  return await c.json(result);
});
```

### Custom Database Connections

You can implement an adapter to a different Postgres library, or even a different database entirely.

To do so, provide a `connectionProvider` to `PushProcessor` that returns a different [`DBConnection`](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zql/src/mutate/custom.ts#L67) implementation. For an example implementation, [see the `postgres` implementation](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zero-pg/src/postgres-connection.ts#L4).

### Custom Push Implementation

You can manually implement the push protocol in any programming language.

This will be documented in the future, but you can refer to the [PushProcessor](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zero-pg/src/web.ts#L33) source code for an example for now.

## Examples

- Zbugs uses [custom mutators](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/shared/mutators.ts) for all mutations, [write permissions](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/shared/mutators.ts#L61), and [notifications](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/server/server-mutators.ts#L35).
- `hello-zero-solid` uses custom mutators for all [mutations](TODO), and for [permissions](TODO).


--- writing-data.mdx ---

Zero generates basic CRUD mutators for every table you sync. Mutators are available at `zero.mutate.<tablename>`:

```tsx
const z = new Zero(...);
z.mutate.user.insert({
	id: nanoid(),
	username: 'abby',
	language: 'en-us',
});
```

<Note slug="moar-power" heading="Need more power?">
  To build mutators with more complex logic or server-specific behavior, see the
  new [Custom Mutators API](./custom-mutators).
</Note>

## Insert

Create new records with `insert`:

```tsx
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: 'js',
});
```

Optional fields can be set to `null` to explicitly set the new field to `null`. They can also be set to `undefined` to take the default value (which is often `null` but can also be some generated value server-side).

```tsx
// schema.ts
import {createTableSchema} from '@rocicorp/zero';

const userSchema = createTableSchema({
  tableName: 'user',
  columns: {
    id: {type: 'string'},
    name: {type: 'string'},
    language: {type: 'string', optional: true},
  },
  primaryKey: ['id'],
  relationships: {},
});

// app.tsx

// Sets language to `null` specifically
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: null,
});

// Sets language to the default server-side value. Could be null, or some
// generated or constant default value too.
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
});

// Same as above
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: undefined,
});
```

## Upsert

Create new records or update existing ones with `upsert`:

```tsx
z.mutate.user.upsert({
  id: samID,
  username: 'sam',
  language: 'ts',
});
```

`upsert` supports the same `null` / `undefined` semantics for optional fields that `insert` does (see above).

## Update

Update an existing record. Does nothing if the specified record (by PK) does not exist.

You can pass a partial, leaving fields out that you don’t want to change. For example here we leave the username the same:

```tsx
// Leaves username field to previous value.
z.mutate.user.update({
  id: samID,
  language: 'golang',
});

// Same as above
z.mutate.user.update({
  id: samID,
  username: undefined,
  language: 'haskell',
});

// Reset language field to `null`
z.mutate.user.update({
  id: samID,
  language: null,
});
```

## Delete

Delete an existing record. Does nothing if specified record does not exist.

```tsx
z.mutate.user.delete({
  id: samID,
});
```

<Note type="note" heading="Cache Persistence" slug="cache-persistence">
If you want to remove cached data from the browser you can call the `dropAllDatabases` function provided by Zero.

You can read more about it in [Authentication](/docs/auth#data-storage).

</Note>

## Batch Mutate

You can do multiple CRUD mutates in a single _batch_. If any of the mutations fails, all will. They also all appear together atomically in a single transaction to other clients.

```tsx
z.mutateBatch(async tx => {
  const samID = nanoid();
  tx.user.insert({
    id: samID,
    username: 'sam',
  });

  const langID = nanoid();
  tx.language.insert({
    id: langID,
    userID: samID,
    name: 'js',
  });
});
```


--- reading-data.mdx ---

ZQL is Zero’s query language.

Inspired by SQL, ZQL is expressed in TypeScript with heavy use of the builder pattern. If you have used [Drizzle](https://orm.drizzle.team/) or [Kysely](https://kysely.dev/), ZQL will feel familiar.

ZQL queries are composed of one or more _clauses_ that are chained together into a _query_.

Unlike queries in classic databases, the result of a ZQL query is a _view_ that updates automatically and efficiently as the underlying data changes. You can call a query’s `materialize()` method to get a view, but more typically you run queries via some framework-specific bindings. For example see `useQuery` for [React](react) or [SolidJS](solidjs).

<Note type="note" emoji="🧑‍🏫" slug="immutability" heading="Data returned from ZQL should be considered immutable">
This means you should not modify the data directly. Instead, clone the data and modify the clone.

ZQL caches values and returns them multiple times. If you modify a value returned from ZQL, you will modify it everywhere it is used. This can lead to subtle bugs.

JavaScript and TypeScript lack true immutable types so we use `readonly` to help enforce it. But it's easy to cast away the `readonly` accidentally.

In the future, we'll [`freeze`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze) all returned data in `dev` mode to help prevent this.

</Note>

## Select

ZQL queries start by selecting a table. There is no way to select a subset of columns; ZQL queries always return the entire row (modulo column permissions).

```tsx
const z = new Zero(...);

// Returns a query that selects all rows and columns from the issue table.
z.query.issue;
```

This is a design tradeoff that allows Zero to better reuse the row locally for future queries. This also makes it easier to share types between different parts of the code.

## Ordering

You can sort query results by adding an `orderBy` clause:

```tsx
z.query.issue.orderBy('created', 'desc');
```

Multiple `orderBy` clauses can be present, in which case the data is sorted by those clauses in order:

```tsx
// Order by priority descending. For any rows with same priority,
// then order by created desc.
z.query.issue.orderBy('priority', 'desc').orderBy('created', 'desc');
```

All queries in ZQL have a default final order of their primary key. Assuming the `issue` table has a primary key on the `id` column, then:

```tsx
// Actually means: z.query.issue.orderBy('id', 'asc');
z.query.issue;

// Actually means: z.query.issue.orderBy('priority', 'desc').orderBy('id', 'asc');
z.query.issue.orderBy('priority', 'desc');
```

## Limit

You can limit the number of rows to return with `limit()`:

```tsx
z.query.issue.orderBy('created', 'desc').limit(100);
```

## Paging

You can start the results at or after a particular row with `start()`:

```tsx
let start: IssueRow | undefined;
while (true) {
  let q = z.query.issue.orderBy('created', 'desc').limit(100);
  if (start) {
    q = q.start(start);
  }
  const batch = await q.run();
  console.log('got batch', batch);

  if (batch.length < 100) {
    break;
  }
  start = batch[batch.length - 1];
}
```

By default `start()` is _exclusive_ - it returns rows starting **after** the supplied reference row. This is what you usually want for paging. If you want _inclusive_ results, you can do:

```tsx
z.query.issue.start(row, {inclusive: true});
```

## Getting a Single Result

If you want exactly zero or one results, use the `one()` clause. This causes ZQL to return `Row|undefined` rather than `Row[]`.

```tsx
const result = await z.query.issue.where('id', 42).one().run();
if (!result) {
  console.error('not found');
}
```

`one()` overrides any `limit()` clause that is also present.

## Relationships

You can query related rows using _relationships_ that are defined in your [Zero schema](/docs/zero-schema).

```tsx
// Get all issues and their related comments
z.query.issue.related('comments');
```

Relationships are returned as hierarchical data. In the above example, each row will have a `comments` field which is itself an array of the corresponding comments row.

You can fetch multiple relationships in a single query:

```tsx
z.query.issue.related('comments').related('reactions').related('assignees');
```

### Refining Relationships

By default all matching relationship rows are returned, but this can be refined. The `related` method accepts an optional second function which is itself a query.

```tsx
z.query.issue.related(
  'comments',
  // It is common to use the 'q' shorthand variable for this parameter,
  // but it is a _comment_ query in particular here, exactly as if you
  // had done z.query.comment.
  q => q.orderBy('modified', 'desc').limit(100).start(lastSeenComment),
);
```

This _relationship query_ can have all the same clauses that top-level queries can have.

<Note type="warning" heading="Order and limit not supported in junction relationships">
Using `orderBy` or `limit` in a relationship that goes through a junction table (i.e., a many-to-many relationship) is not currently supported and will throw a runtime error. See [bug 3527](https://bugs.rocicorp.dev/issue/3527).

You can sometimes work around this by making the junction relationship explicit, depending on your schema and usage.
</Note>

### Nested Relationships

You can nest relationships arbitrarily:

```tsx
// Get all issues, first 100 comments for each (ordered by modified,desc),
// and for each comment all of its reactions.
z.query.issue.related(
	'comments', q => q.orderBy('modified', 'desc').limit(100).related(
		'reactions')
	)
);
```

## Where

You can filter a query with `where()`:

```tsx
z.query.issue.where('priority', '=', 'high');
```

The first parameter is always a column name from the table being queried. Intellisense will offer available options (sourced from your [Zero Schema](/docs/zero-schema)).

### Comparison Operators

Where supports the following comparison operators:

| Operator                                 | Allowed Operand Types         | Description                                                              |
| ---------------------------------------- | ----------------------------- | ------------------------------------------------------------------------ |
| `=` , `!=`                               | boolean, number, string       | JS strict equal (===) semantics                                          |
| `<` , `<=`, `>`, `>=`                    | number                        | JS number compare semantics                                              |
| `LIKE`, `NOT LIKE`, `ILIKE`, `NOT ILIKE` | string                        | SQL-compatible `LIKE` / `ILIKE`                                          |
| `IN` , `NOT IN`                          | boolean, number, string       | RHS must be array. Returns true if rhs contains lhs by JS strict equals. |
| `IS` , `IS NOT`                          | boolean, number, string, null | Same as `=` but also works for `null`                                    |

TypeScript will restrict you from using operators with types that don’t make sense – you can’t use `>` with `boolean` for example.

<Note type="note" slug="adding-operators">
  If you don’t see the comparison operator you need, let us know, many are easy
  to add.
</Note>

### Equals is the Default Comparison Operator

Because comparing by `=` is so common, you can leave it out and `where` defaults to `=`.

```tsx
z.query.issue.where('priority', 'high');
```

### Comparing to `null`

As in SQL, ZQL’s `null` is not equal to itself (`null ≠ null`).

This is required to make join semantics work: if you’re joining `employee.orgID` on `org.id` you do **not** want an employee in no organization to match an org that hasn’t yet been assigned an ID.

When you purposely want to compare to `null` ZQL supports `IS` and `IS NOT` operators that work just like in SQL:

```tsx
// Find employees not in any org.
z.query.employee.where('orgID', 'IS', null);
```

TypeScript will prevent you from comparing to `null` with other operators.

### Compound Filters

The argument to `where` can also be a callback that returns a complex expression:

```tsx
// Get all issues that have priority 'critical' or else have both
// priority 'medium' and not more than 100 votes.
z.query.issue.where(({cmp, and, or, not}) =>
  or(
    cmp('priority', 'critical'),
    and(cmp('priority', 'medium'), not(cmp('numVotes', '>', 100))),
  ),
);
```

`cmp` is short for _compare_ and works the same as `where` at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below).

Note that chaining `where()` is also a one-level `and`:

```tsx
// Find issues with priority 3 or higher, owned by aa
z.query.issue.where('priority', '>=', 3).where('owner', 'aa');
```

### Relationship Filters

Your filter can also test properties of relationships. Currently the only supported test is existence:

```tsx
// Find all orgs that have at least one employee
z.query.organization.whereExists('employees');
```

The argument to `whereExists` is a relationship, so just like other relationships it can be refined with a query:

```tsx
// Find all orgs that have at least one cool employee
z.query.organization.whereExists('employees', q =>
  q.where('location', 'Hawaii'),
);
```

As with querying relationships, relationship filters can be arbitrarily nested:

```tsx
// Get all issues that have comments that have reactions
z.query.issue.whereExists('comments',
	q => q.whereExists('reactions'));
);
```

The `exists` helper is also provided which can be used with `and`, `or`, `cmp`, and `not` to build compound filters that check relationship existence:

```tsx
// Find issues that have at least one comment or are high priority
z.query.issue.where({cmp, or, exists} =>
  or(
    cmp('priority', 'high'),
    exists('comments'),
  ),
);
```

## Data Lifetime and Reuse

Zero reuses data synced from prior queries to answer new queries when possible. This is what enables instant UI transitions.

But what controls the lifetime of this client-side data? How can you know whether any partiular query will return instant results? How can you know whether those results will be up to date or stale?

The answer is that the data on the client is simply the union of rows returned from queries which are currently syncing. Once a row is no longer returned by any syncing query, it is removed from the client. Thus, there is never any stale data in Zero.

So when you are thinking about whether a query is going to return results instantly, you should think about _what other queries are syncing_, not about what data is local. Data exists locally if and only if there is a query syncing that returns that data.

<Note type="note" heading="Caches vs Replicas">
  This is why we often say that despite the name `zero-cache`, Zero is not technically a cache. It's a *replica*.

A cache has a random set of rows with a random set of versions. There is no expectation that the cache any particular rows, or that the rows' have matching versions. Rows are simply updated as they are fetched.

A replica by contrast is eagerly updated, whether or not any client has requested a row. A replica is always very close to up-to-date, and always self-consistent.

Zero is a _partial_ replica because it only replicates rows that are returned by syncing queries.

</Note>

## Query Lifecycle

<ImageLightbox
  src="/images/reading-data/query-lifecycle.png"
  caption="The lifecycle of a ZQL query."
/>

Queries can be either _active_ or _backgrounded_. An active query is one that is currently being used by the application. Backgrounded queries are not currently in use, but continue syncing in case they are needed again soon.

Active queries are created one of three ways:

1. The app calls `q.materialize()` to get a `View`.
2. The app uses a platform binding like React's `useQuery(q)`.
3. The app calls [`preload()`](#preloading) to sync larger queries without a view.

Active queries sync until they are _deactivated_. The way this happens depends on how the query was created:

1. For `materialize()` queries, the UI calls `destroy()` on the view.
2. For `useQuery()`, the UI unmounts the component (which calls `destroy()` under the covers).
3. For `preload()`, the UI calls `cleanup()` on the return value of `preload()`.

### Background Queries

By default a deactivated query stops syncing immediately.

But it's often useful to keep queries syncing beyond deactivation in case the UI needs the same or a similar query in the near future. This is accomplished with the `ttl` parameter:

```ts
const [user] = useQuery(z.query.user.where('id', userId), {ttl: '1d'});
```

The `ttl` parameter specifies how long the app developer wishes the query to run in the background. The following formats are allowed (where `%d` is a positive integer):

| Format    | Meaning                                                                              |
| --------- | ------------------------------------------------------------------------------------ |
| `none`    | No backgrounding. Query will immediately stop when deactivated. This is the default. |
| `%ds`     | Number of seconds.                                                                   |
| `%dm`     | Number of minutes.                                                                   |
| `%dh`     | Number of hours.                                                                     |
| `%dd`     | Number of days.                                                                      |
| `%dy`     | Number of years.                                                                     |
| `forever` | Query will never be stopped.                                                         |

If the UI re-requests a background query, it becomes an active query again. Since the query was syncing in the background, the very first synchronous result that the UI receives after reactivation will be up-to-date with the server (i.e., it will have `resultType` of `complete`).

Just like other types of queries, the data from background queries is available for use by new queries. A common pattern in to [preload](#preloading) a subset of most commonly needed data with `{ttl: 'forever'}` and then do more specific queries from the UI with, e.g., `{ttl: '1d'}`. Most often the preloaded data will be able to answer user queries, but if not, the new query will be answered by the server and backgrounded for a day in case the user revisits it.

### Client Capacity Management

Zero has a default soft limit of 20,000 rows on the client-side, or about 20MB of data assuming 1KB rows.

This limit can be increased with the [`--target-client-row-count`](./zero-cache-config#target-client-row-count) flag, but we do not recommend setting it higher than 100,000.

<Note slug="client-capacity" heading="Why does Zero store so little data client-side?">
Contrary to the design of other sync engines, we believe that storing tons of data client-side doesn't make sense. Here are some reasons why:

- Initial sync will be slow, slowing down initial app load.
- Because storage in browser tabs is unreliable, initial sync can occur surprisingly often.
- We want to answer queries _instantly_ as often as possible. This requires client-side data in memory on the main thread. If we have to page to disk, we may as well go to the network and reduce complexity.
- Even though Zero's queries are very efficient, they do still have some cost, expecially hydration. Massive client-side storage would result in hydrating tons of queries that are unlikely to be used every time the app starts.

Most importantly, no matter how much data you store on the client, there will be cases where you have to fallback to the server:

- Some users might have huge amounts of data.
- Some users might have tiny amounts of available client storage.
- You will likely want the app to start fast and sync in the background.

Because you have to be able to fallback to server the question becomes _what is the **right** amount of data to store on the client?_, not _how can I store the absolute max possible data on the client?_

The goal with Zero is to answer 99% of queries on the client from memory. The remaining 1% of queries can fallback gracefully to the server. 20,000 rows was chosen somewhat arbitrarily as a number of rows that was likely to be able to do this for many applications.

There is no hard limit at 20,000 or 100,000. Nothing terrible happens if you go above. The thing to keep in mind is that:

1. All those queries will revalidate everytime your app boots.
2. All data synced to the client is in memory in JS.

</Note>

Here is how this limit is managed:

1. Active queries are never destroyed, even if the limit is exceeded. Developers are expected to keep active queries well under the limit.
2. The `ttl` value counts from the moment a query deactivates. Backgrounded queries are destroyed immediately when the `ttl` is reached, even if the limit hasn't been reached.
3. If the client exceeds its limit, Zero will destroy backgrounded queries, least-recently-used first, until the store is under the limit again.

### Thinking in Queries

Although IVM is a very efficient way to keep queries up to date relative to re-running them, it isn't free. You still need to think about how many queries you are creating, how long they are kept alive, and how expensive they are.

This is why Zero defaults to _not_ backgrounding queries and doesn't try to aggressively fill its client datastore to capacity. You should put some thought into what queries you want to run in the background, and for how long.

Zero currently provides a few basic tools to understand the cost of your queries:

- The client logs a warning for slow query materializations. Look for `Slow query materialization` in your logs. The default threshold is `5s` (including network) but this is configurable with the `slowMaterializeThreshold` parameter.
- The client logs the materialization time of all queries at the `debug` level. Look for `Materialized query` in your logs.
- The server logs a warning for slow query materializations. Look for `Slow query materialization` in your logs. The default threshold is `5s` but this is configurable with the `log-slow-materialize-threshold` configuration parameter.

We will be adding more tools over time.

## Completeness

Zero returns whatever data it has on the client immediately for a query, then falls back to the server for any missing data. Sometimes it's useful to know the difference between these two types of results. To do so, use the `result` from `useQuery`:

```tsx
const [issues, issuesResult] = useQuery(z.query.issue);
if (issuesResult.type === 'complete') {
  console.log('All data is present');
} else {
  console.log('Some data is missing');
}
```

The possible values of `result.type` are currently `complete` and `unknown`.

The `complete` value is currently only returned when Zero has received the server result. But in the future, Zero will be able to return this result type when it _knows_ that all possible data for this query is already available locally. Additionally, we plan to add a `prefix` result for when the data is known to be a prefix of the complete result. See [Consistency](#consistency) for more information.

## Handling Missing Data

It is inevetable that there will be cases where the requested data cannot be found. Because Zero returns local results immediately, and server results asynchronously, displaying "not found" / 404 UI can be slightly tricky. If you just use a simple existence check, you will often see the 404 UI flicker while the server result loads:

```tsx
const [issue, issuesResult] = useQuery(
  z.query.issue.where('id', 'some-id').one(),
);

// ❌ This causes flickering of the UI
if (!issue) {
  return <div>404 Not Found</div>;
} else {
  return <div>{issue}</div>;
}
```

The way to do this correctly is to only display the "not found" UI when the result type is `complete`. This way the 404 page is slow but pages with data are still just as fast.

```tsx
const [issue, issuesResult] = useQuery(
  z.query.issue.where('id', 'some-id').one(),
);

if (!issue && issueResult.type === 'complete') {
  return <div>404 Not Found</div>;
}

if (!issue) {
  return null;
}

return <div>{issue}</div>;
```

## Listening to Changes

Currently, the way to listen for changes in query results is not ideal. You can add a listener to a materialized view which has the new data and result as parameters:

```ts
z.query.issue.materialize().addListener((issues, issuesResult) => {
  // do stuff...
});
```

However, using this method will maintain its own materialized view in memory which is wasteful. It also doesn't allow for granular listening to events like `add` and `remove` of rows.

A better way would be to create your own view without actually storing the data which will also allow you to listen to specific events. Again, the API is not good and will be improved in the future.

```ts
// Inside the View class
// Instead of storing the change, we invoke some callback
push(change: Change): void {
  switch (change.type) {
    case 'add':
      this.#onAdd?.(change)
      break
    case 'remove':
      this.#onRemove?.(change)
      break
    case 'edit':
      this.#onEdit?.(change)
      break
    case 'child':
      this.#onChild?.(change)
      break
    default:
      throw new Error(`Unknown change type: ${change['type']}`)
  }
}
```

(see View implementations in [`zero-vue`](https://github.com/danielroe/zero-vue/blob/f25808d4b7d1ef0b8e01a5670d7e3050d6a64bbf/src/view.ts#L77-L89) or [`zero-solid`](https://github.com/rocicorp/mono/blob/51995101d0657519207f1c4695a8765b9016e07c/packages/zero-solid/src/solid-view.ts#L119-L131))

## Preloading

Almost all Zero apps will want to preload some data in order to maximize the feel of instantaneous UI transitions.

In Zero, preloading is done via queries – the same queries you use in the UI and for auth.

However, because preload queries are usually much larger than a screenful of UI, Zero provides a special `preload()` helper to avoid the overhead of materializing the result into JS objects:

```tsx
// Preload the first 1k issues + their creator, assignee, labels, and
// the view state for the active user.
//
// There's no need to render this data, so we don't use `useQuery()`:
// this avoids the overhead of pulling all this data into JS objects.
z.query.issue
  .related('creator')
  .related('assignee')
  .related('labels')
  .related('viewState', q => q.where('userID', z.userID).one())
  .orderBy('created', 'desc')
  .limit(1000)
  .preload();
```

## Running Queries Once

Usually subscribing to a query is what you want in a reactive UI, but every so often you'll need to run a query just once. To do this, use the `run()` method:

```tsx
const results = await z.query.issue.where('foo', 'bar').run();
```

By default, `run()` only returns results that are currently available on the client. That is, it returns the data that would be given for [`result.type === 'unknown'`](#completeness).

If you want to wait for the server to return results, pass `{type: 'complete'}` to `run`:

```tsx
const results = await z.query.issue.where('foo', 'bar').run({type: 'complete'});
```

<Note slug="default-run">
As a convenience you can also directly await queries:

```ts
await z.query.issue.where('foo', 'bar');
```

This is the same as saying `run()` or `run({type: 'unknown'})`.

</Note>

## Consistency

Zero always syncs a consistent partial replica of the backend database to the client. This avoids many common consistency issues that come up in classic web applications. But there are still some consistency issues to be aware of when using Zero.

For example, imagine that you have a bug database w/ 10k issues. You preload the first 1k issues sorted by created.

The user then does a query of issues assigned to themselves, sorted by created. Among the 1k issues that were preloaded imagine 100 are found that match the query. Since the data we preloaded is in the same order as this query, we are guaranteed that any local results found will be a _prefix_ of the server results.

The UX that result is nice: the user will see initial results to the query instantly. If more results are found server-side, those results are guaranteed to sort below the local results. There's no shuffling of results when the server response comes in.

Now imagine that the user switches the sort to ‘sort by modified’. This new query will run locally, and will again find some local matches. But it is now unlikely that the local results found are a prefix of the server results. When the server result comes in, the user will probably see the results shuffle around.

To avoid this annoying effect, what you should do in this example is also preload the first 1k issues sorted by modified desc. In general for any query shape you intend to do, you should preload the first `n` results for that query shape with no filters, in each sort you intend to use.

<Note slug="no-duplicate-rows" heading="Zero does not sync duplicate rows">
Zero syncs the *union* of all active queries' results. You don't have to worry about syncing many sorts of the same query when it's likely the results will overlap heavily.

</Note>

In the future, we will be implementing a consistency model that fixes these issues automatically. We will prevent Zero from returning local data when that data is not known to be a prefix of the server result. Once the consistency model is implemented, preloading can be thought of as purely a performance thing, and not required to avoid unsightly flickering.


--- permissions.mdx ---

Permissions are expressed using [ZQL](reading-data) and run automatically with every read and write.

## Define Permissions

Permissions are defined in [`schema.ts`](/docs/zero-schema) using the `definePermissions` function.

Here's an example of limiting reads to members of an organization and deletes to only the creator of an issue:

```ts
// The decoded value of your JWT.
type AuthData = {
  // The logged-in user.
  sub: string;
};

export const permissions = definePermissions<AuthData, Schema>(schema, () => {
  // Checks if the user exists in a related organization
  const allowIfInOrganization = (
    authData: AuthData,
    eb: ExpressionBuilder<Schema, 'issue'>,
  ) =>
    eb.exists('organization', q =>
      q.whereExists('user', q => q.where('id', authData.sub)),
    );

  // Checks if the user is the creator
  const allowIfIssueCreator = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('creatorID', authData.sub);

  return {
    issue: {
      row: {
        select: [allowIfInOrganization],
        delete: [allowIfIssueCreator],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

`definePermission` returns a _policy_ object for each table in the schema. Each policy defines a _ruleset_ for the _operations_ that are possible on a table: `select`, `insert`, `update`, and `delete`.

## Access is Denied by Default

If you don't specify any rules for an operation, it is denied by default. This is an important safety feature that helps ensure data isn't accidentally exposed.

To enable full access to an action (i.e., during development) use the `ANYONE_CAN` helper:

```ts
import {ANYONE_CAN} from '@rocicorp/zero';

const permissions = definePermissions<AuthData, Schema>(schema, () => {
  return {
    issue: {
      row: {
        select: ANYONE_CAN,
        // Other operations are denied by default.
      },
    },
    // Other tables are denied by default.
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

To do this for all actions, use `ANYONE_CAN_DO_ANYTHING`:

```ts
import {ANYONE_CAN_DO_ANYTHING} from '@rocicorp/zero';

const permissions = definePermissions<AuthData, Schema>(schema, () => {
  return {
    // All operations on issue are allowed to all users.
    issue: ANYONE_CAN_DO_ANYTHING,

    // Other tables are denied by default.
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Permission Evaluation

Zero permissions are "compiled" into a JSON-based format at build-time. This file is stored in the `{ZERO_APP_ID}.permissions` table of your upstream database. Like other tables, it replicates live down to `zero-cache`. `zero-cache` then parses this file, and applies the encoded rules to every read and write operation.

<Note type="warning" slug="rule-compilation-is-dumb">
The compilation process is very simple-minded (read: dumb). Despite looking like normal TypeScript functions that receive an `AuthData` parameter, rule functions are not actually invoked at runtime. Instead, they are invoked with a "placeholder" `AuthData` at build time. We track which fields of this placeholder are accessed and construct a ZQL expression that accesses the right field of `AuthData` at runtime.

The end result is that you can't really use most features of JS in these rules. Specifically you cannot:

- Iterate over properties or array elements in the auth token
- Use any JS features beyond property access of `AuthData`
- Use any conditional or global state

Basically only property access is allowed. This is really confusing and we're working on a better solution.

</Note>

## Permission Deployment

During development, permissions are compiled and uploaded to your database completely automatically as part of the `zero-cache-dev` script.

For production, you need to call `npx zero-deploy-permissions` within your app to update the permissions in the production database whenever they change. You would typically do this as part of your normal schema migration or CI process. For example, the SST deployment script for [zbugs](/docs/samples#zbugs) looks like this:

```ts
new command.local.Command(
  'zero-deploy-permissions',
  {
    create: `npx zero-deploy-permissions -p ../../src/schema.ts`,
    // Run the Command on every deploy ...
    triggers: [Date.now()],
    environment: {
      ZERO_UPSTREAM_DB: commonEnv.ZERO_UPSTREAM_DB,
      // If the application has a non-default App ID ...
      ZERO_APP_ID: commonEnv.ZERO_APP_ID,
    },
  },
  // after the view-syncer is deployed.
  {dependsOn: viewSyncer},
);
```

See the [SST Deployment Guide](deployment#guide-multi-node-on-sstaws) for more details.

## Rules

Each operation on a policy has a _ruleset_ containing zero or more _rules_.

A rule is just a TypeScript function that receives the logged in user's `AuthData` and generates a ZQL [where expression](reading-data#compound-filters). At least one rule in a ruleset must return a row for the operation to be allowed.

## Select Permissions

You can limit the data a user can read by specifying a `select` ruleset.

Select permissions act like filters. If a user does not have permission to read a row, it will be filtered out of the result set. It will not generate an error.

For example, imagine a select permission that restricts reads to only issues created by the user:

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueCreator = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('creatorID', authData.sub);

  return {
    issue: {
      row: {
        select: [allowIfIssueCreator],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

If the issue table has two rows, one created by the user and one by someone else, the user will only see the row they created in any queries.

## Insert Permissions

You can limit what rows can be inserted and by whom by specifying an `insert` ruleset.

Insert rules are evaluated after the entity is inserted. So if they query the database, they will see the inserted row present. If any rule in the insert ruleset returns a row, the insert is allowed.

Here's an example of an insert rule that disallows inserting users that have the role 'admin'.

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfNonAdmin = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'user'>,
  ) => cmp('role', '!=', 'admin');

  return {
    user: {
      row: {
        insert: [allowIfNonAdmin],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Update Permissions

There are two types of update rulesets: `preMutation` and `postMutation`. Both rulesets must pass for an update to be allowed.

`preMutation` rules see the version of a row _before_ the mutation is applied. This is useful for things like checking whether a user owns an entity before editing it.

`postMutation` rules see the version of a row _after_ the mutation is applied. This is useful for things like ensuring a user can only mark themselves as the creator of an entity and not other users.

Like other rulesets, `preMutation` and `postMutation` default to `NOBODY_CAN`. This means that every table must define both these rulesets in order for any updates to be allowed.

For example, the following ruleset allows an issue's owner to edit, but **not** re-assign the issue. The `postMutation` rule enforces that the current user still own the issue after edit.

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: [allowIfIssueOwner],
          postMutation: [allowIfIssueOwner],
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

This ruleset allows an issue's owner to edit and re-assign the issue:

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: [allowIfIssueOwner],
          postMutation: ANYONE_CAN,
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

And this allows anyone to edit an issue, but only if they also assign it to themselves. Useful for enforcing _"patches welcome"_? 🙃

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: ANYONE_CAN,
          postMutation: [allowIfIssueOwner],
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Delete Permissions

Delete permissions work in the same way as `insert` permissions except they run _before_ the delete is applied. So if a delete rule queries the database, it will see that the deleted row is present. If any rule in the ruleset returns a row, the delete is allowed.

## Debugging

See [Debugging Permissions](./debug/permissions).

## Examples

See [hello-zero](https://github.com/rocicorp/hello-zero/blob/main/src/schema.ts) for a simple example of write auth and [zbugs](https://github.com/rocicorp/mono/blob/main/apps/zbugs/shared/schema.ts#L217) for a much more involved one.


--- debug/permissions.mdx ---

Given that permissions are defined in their own file and internally applied to queries, it might be hard to figure out if or why a permission check is failing.

## Read Permissions

You can use the `analyze-query` utility with the `--apply-permissions` flag to see the complete query Zero runs, including read permissions.

```bash
npx analyze-query
  --schema-path='./shared/schema.ts'
  --query='issue.related("comments")'
  --apply-permissions
  --auth-data='{"userId":"user-123"}'
```

If the result looks right, the problem may be that Zero is not receiving the `AuthData` that you think it is. You can retrieve a query hash from websocket or server logs, then ask Zero for the details on that specific query.

Run this command with the same environment you run `zero-cache` with. It will use your `upstream` or `cvr` configuration to look up the query hash in the cvr database.

```bash
npx analyze-query
  --schema-path='./shared/schema.ts'
  --hash='3rhuw19xt9vry'
  --apply-permissions
  --auth-data='{"userId":"user-123"}'
```

<Note type="note" slug="decompilation">
  The printed query can be different than the source ZQL string, because it is rebuilt from the query AST. But it should be logically equivalent to the query you wrote.
</Note>

## Write Permissions

Look for a `WARN` level log in the output from `zero-cache` like this:

```
Permission check failed for {"op":"update","tableName":"message",...}, action update, phase preMutation, authData: {...}, rowPolicies: [...], cellPolicies: []
```

Zero prints the row, auth data, and permission policies that was applied to any failed writes.

<Note slug="ast-format">
  The ZQL query is printed in AST format. See [Query ASTs](./query-asts) to
  convert it to a more readable format.
</Note>


--- zero-cache-config.mdx ---

`zero-cache` is configured either via CLI flag or environment variable. There is no separate `zero.config` file.

You can also see all available flags by running `zero-cache --help`.

## Required Flags

### Auth

One of [Auth JWK](#auth-jwk), [Auth JWK URL](#auth-jwk-url), or [Auth Secret](#auth-secret) must be specified. See [Authentication](/docs/auth/) for more details.

### Replica File

File path to the SQLite replica that zero-cache maintains. This can be lost, but if it is, zero-cache will have to re-replicate next time it starts up.

flag: `--replica-file`<br/>
env: `ZERO_REPLICA_FILE`<br/>
required: `true`

### Upstream DB

The "upstream" authoritative postgres database. In the future we will support other types of upstream besides PG.

flag: `--upstream-db`<br/>
env: `ZERO_UPSTREAM_DB`<br/>
required: `true`

## Optional Flags

### Admin Password

A password used to administer zero-cache server, for example to access the `/statz` endpoint.

flag: `--admin-password`<br/>
env: `ZERO_ADMIN_PASSWORD`<br/>
required: `false`

### App ID

Unique identifier for the app.

Multiple zero-cache apps can run on a single upstream database, each of which is isolated from the others, with its own permissions, sharding (future feature), and change/cvr databases.

The metadata of an app is stored in an upstream schema with the same name, e.g. `zero`, and the metadata for each app shard, e.g. client and mutation ids, is stored in the `{app-id}_{#}` schema. (Currently there is only a single "0" shard, but this will change with sharding).

The CVR and Change data are managed in schemas named `{app-id}_{shard-num}/cvr` and `{app-id}_{shard-num}/cdc`, respectively, allowing multiple apps and shards to share the same database instance (e.g. a Postgres "cluster") for CVR and Change management.

Due to constraints on replication slot names, an App ID may only consist of lower-case letters, numbers, and the underscore character.

Note that this option is used by both `zero-cache` and `zero-deploy-permissions`.

flag: `--app-id`<br/>
env: `ZERO_APP_ID`<br/>
default: `zero`

### App Publications

Postgres PUBLICATIONs that define the tables and columns to replicate. Publication names may not begin with an underscore, as zero reserves that prefix for internal use.

If unspecified, zero-cache will create and use an internal publication that publishes all tables in the public schema, i.e.:

```
CREATE PUBLICATION _{app-id}_public_0 FOR TABLES IN SCHEMA public;
```

Note that once an app has begun syncing data, this list of publications cannot be changed, and zero-cache will refuse to start if a specified value differs from what was originally synced. To use a different set of publications, a new app should be created.

flag: `--app-publications`<br/>
env: `ZERO_APP_PUBLICATIONS`<br/>
default: `[]`

### Auth JWK

A public key in JWK format used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-jwk`<br/>
env: `ZERO_AUTH_JWK`<br/>
required: `false`

### Auth JWK URL

A URL that returns a JWK set used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-jwks-url`<br/>
env: `ZERO_AUTH_JWKS_URL`<br/>
required: `false`

### Auto Reset

Automatically wipe and resync the replica when replication is halted. This situation can occur for configurations in which the upstream database provider prohibits event trigger creation, preventing the zero-cache from being able to correctly replicate schema changes. For such configurations, an upstream schema change will instead result in halting replication with an error indicating that the replica needs to be reset. When auto-reset is enabled, zero-cache will respond to such situations by shutting down, and when restarted, resetting the replica and all synced clients. This is a heavy-weight operation and can result in user-visible slowness or downtime if compute resources are scarce.

flag: `--auto-reset`<br/>
env: `ZERO_AUTO_RESET`<br/>
default: `true`

### Auth Secret

A symmetric key used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-secret`<br/>
env: `ZERO_AUTH_SECRET`<br/>
required: `false`

### Change DB

The Postgres database used to store recent replication log entries, in order to sync multiple view-syncers without requiring multiple replication slots on the upstream database. If unspecified, the upstream-db will be used.

flag: `--change-db`<br/>
env: `ZERO_CHANGE_DB`<br/>
required: `false`

### Change Max Connections

The maximum number of connections to open to the change database. This is used by the change-streamer for catching up zero-cache replication subscriptions.

flag: `--change-max-conns`<br/>
env: `ZERO_CHANGE_MAX_CONNS`<br/>
default: `5`

### Change Streamer Mode
The mode for running or connecting to the change-streamer:

* `dedicated`: runs the change-streamer and shuts down when another
  change-streamer takes over the replication slot. This is appropriate in a
  single-node configuration, or for the replication-manager in a
  multi-node configuration.
* `discover`: connects to the change-streamer as internally advertised in the
  change-db. This is appropriate for the view-syncers in a multi-node

flag: `--change-streamer-mode`<br/>
env: `ZERO_CHANGE_STREAMER_MODE`<br/>
default: `dedicated`

### Change Streamer Port

The port on which the change-streamer runs. This is an internal protocol between the replication-manager and zero-cache, which runs in the same process in local development. If unspecified, defaults to --port + 1.

flag: `--change-streamer-port`<br/>
env: `ZERO_CHANGE_STREAMER_PORT`<br/>
required: `false`

### CVR DB

The Postgres database used to store CVRs. CVRs (client view records) keep track of the data synced to clients in order to determine the diff to send on reconnect. If unspecified, the upstream-db will be used.

flag: `--cvr-db`<br/>
env: `ZERO_CVR_DB`<br/>
required: `false`

### CVR Max Connections

The maximum number of connections to open to the CVR database. This is divided evenly amongst sync workers.

Note that this number must allow for at least one connection per sync worker, or zero-cache will fail to start. See num-sync-workers.

flag: `--cvr-max-conns`<br/>
env: `ZERO_CVR_MAX_CONNS`<br/>
default: `30`

### Initial Sync Table Copy Workers

The number of parallel workers used to copy tables during initial sync. Each worker copies a single table at a time, fetching rows in batches of `initial-sync-row-batch-size`.

flag: `--initial-sync-table-copy-workers`<br/>
env: `ZERO_INITIAL_SYNC_TABLE_COPY_WORKERS`<br/>
default: `5`

### Lazy Startup

Delay starting the majority of zero-cache until first request.

This is mainly intended to avoid connecting to Postgres replication stream until the first request is received, which can be useful i.e., for preview instances.

Currently only supported in single-node mode.

flag: `--lazy-startup`<br/>
env: `ZERO_LAZY_STARTUP`<br/>
default: `false`

### Litestream Executable

Path to the litestream executable. This option has no effect if litestream-backup-url is unspecified.

flag: `--litestream-executable`<br/>
env: `ZERO_LITESTREAM_EXECUTABLE`<br/>
required: `false`

### Litestream Config Path

Path to the litestream yaml config file. zero-cache will run this with its environment variables, which can be referenced in the file via `${ENV}` substitution, for example:

- ZERO_REPLICA_FILE for the db Path
- ZERO_LITESTREAM_BACKUP_LOCATION for the db replica url
- ZERO_LITESTREAM_LOG_LEVEL for the log Level
- ZERO_LOG_FORMAT for the log type

flag: `--litestream-config-path`<br/>
env: `ZERO_LITESTREAM_CONFIG_PATH`<br/>
default: `./src/services/litestream/config.yml`

### Litestream Log Level

flag: `--litestream-log-level`<br/>
env: `ZERO_LITESTREAM_LOG_LEVEL`<br/>
default: `warn`
values: `debug`, `info`, `warn`, `error`

### Litestream Backup URL

The location of the litestream backup, usually an s3:// URL. This is only consulted by the replication-manager. view-syncers receive this information from the replication-manager. 

flag: `--litestream-backup-url`<br/>
env: `ZERO_LITESTREAM_BACKUP_URL`<br/>
required: `false`

### Litestream Checkpoint Threshold MB

The size of the WAL file at which to perform an SQlite checkpoint to apply the writes in the WAL to the main database file. Each checkpoint creates a new WAL segment file that will be backed up by litestream. Smaller thresholds may improve read performance, at the expense of creating more files to download when restoring the replica from the backup.

flag: `--litestream-checkpoint-threshold-mb`<br/>
env: `ZERO_LITESTREAM_CHECKPOINT_THRESHOLD_MB`<br/>
default: `40`

### Litestream Incremental Backup Interval Minutes

The interval between incremental backups of the replica. Shorter intervals reduce the amount of change history that needs to be replayed when catching up a new view-syncer, at the expense of increasing the number of files needed to download for the initial litestream restore.

flag: `--litestream-incremental-backup-interval-minutes`<br/>
env: `ZERO_LITESTREAM_INCREMENTAL_BACKUP_INTERVAL_MINUTES`<br/>
default: `15`

### Litestream Snapshot Backup Interval Hours

The interval between snapshot backups of the replica. Snapshot backups make a full copy of the database to a new litestream generation. This improves restore time at the expense of bandwidth. Applications with a large database and low write rate can increase this interval to reduce network usage for backups (litestream defaults to 24 hours).

flag: `--litestream-snapshot-backup-interval-hours`<br/>
env: `ZERO_LITESTREAM_SNAPSHOT_BACKUP_INTERVAL_HOURS`<br/>
default: `12`

### Litestream Restore Parallelism

The number of WAL files to download in parallel when performing the initial restore of the replica from the backup.

flag: `--litestream-restore-parallelism`<br/>
env: `ZERO_LITESTREAM_RESTORE_PARALLELISM`<br/>
default: `48`

### Log Format

Use text for developer-friendly console logging and json for consumption by structured-logging services.

flag: `--log-format`<br/>
env: `ZERO_LOG_FORMAT`<br/>
default: `"text"`<br/>
values: `text`, `json`

### Log IVM Sampling

How often to collect IVM metrics. 1 out of N requests will be sampled where N is this value.

flag: `--log-ivm-sampling`<br/>
env: `ZERO_LOG_IVM_SAMPLING`<br/>
default: `5000`

### Log Level

Sets the logging level for the application.

flag: `--log-level`<br/>
env: `ZERO_LOG_LEVEL`<br/>
default: `"info"`<br/>
values: `debug`, `info`, `warn`, `error`

### Log Slow Hydrate Threshold

The number of milliseconds a query hydration must take to print a slow warning.

flag: `--log-slow-hydrate-threshold`<br/>
env: `ZERO_LOG_SLOW_HYDRATE_THRESHOLD`<br/>
default: `100`

### Log Slow Row Threshold

The number of ms a row must take to fetch from table-source before it is considered slow.

flag: `--log-slow-row-threshold`<br/>
env: `ZERO_LOG_SLOW_ROW_THRESHOLD`<br/>
default: `2`

### Log Trace Collector

The URL of the trace collector to which to send trace data. Traces are sent over http. Port defaults to 4318 for most collectors.

flag: `--log-trace-collector`<br/>
env: `ZERO_LOG_TRACE_COLLECTOR`<br/>
required: `false`

### Number of Sync Workers

The number of processes to use for view syncing. Leave this unset to use the maximum available parallelism. If set to 0, the server runs without sync workers, which is the configuration for running the replication-manager.

flag: `--num-sync-workers`<br/>
env: `ZERO_NUM_SYNC_WORKERS`<br/>
required: `false`

### Per User Mutation Limit Max

The maximum mutations per user within the specified windowMs.

flag: `--per-user-mutation-limit-max`<br/>
env: `ZERO_PER_USER_MUTATION_LIMIT_MAX`<br/>
required: `false`

### Per User Mutation Limit Window (ms)

The sliding window over which the perUserMutationLimitMax is enforced.

flag: `--per-user-mutation-limit-window-ms`<br/>
env: `ZERO_PER_USER_MUTATION_LIMIT_WINDOW_MS`<br/>
default: `60000`

### Port

The port for sync connections.

flag: `--port`<br/>
env: `ZERO_PORT`<br/>
default: `4848`

### Push URL

The URL of the API server to which zero-cache will push mutations. Required if you use [custom mutators](/docs/custom-mutators).

flag: `--push-url`<br/>
env: `ZERO_PUSH_URL`<br/>
required: `false`

### Query Hydration Stats

Track and log the number of rows considered by query hydrations which take longer than **log-slow-hydrate-threshold** milliseconds.

This is useful for debugging and performance tuning.

flag: `--query-hydration-stats`<br/>
env: `ZERO_QUERY_HYDRATION_STATS`<br/>
required: `false`

### Replica Vacuum Interval Hours

Performs a VACUUM at server startup if the specified number of hours has elapsed since the last VACUUM (or initial-sync). The VACUUM operation is heavyweight and requires double the size of the db in disk space. If unspecified, VACUUM operations are not performed.

flag: `--replica-vacuum-interval-hours`<br/>
env: `ZERO_REPLICA_VACUUM_INTERVAL_HOURS`<br/>
required: `false`

### Server Version

The version string outputted to logs when the server starts up.

flag: `--server-version`<br/>
env: `ZERO_SERVER_VERSION`<br/>
required: `false`

### Storage DB Temp Dir

Temporary directory for IVM operator storage. Leave unset to use `os.tmpdir()`.

flag: `--storage-db-tmp-dir`<br/>
env: `ZERO_STORAGE_DB_TMP_DIR`<br/>
required: `false`

### Target Client Row Count

A soft limit on the number of rows Zero will keep on the client. 20k is a good default value for most applications, and we do not recommend exceeding 100k. See [Client Capacity Management](/docs/reading-data#client-capacity-management) for more details.

flag: `--target-client-row-count`<br/>
env: `ZERO_TARGET_CLIENT_ROW_COUNT`<br/>
default: `20000`

### Task ID

Globally unique identifier for the zero-cache instance. Setting this to a platform specific task identifier can be useful for debugging. If unspecified, zero-cache will attempt to extract the TaskARN if run from within an AWS ECS container, and otherwise use a random string.

flag: `--task-id`<br/>
env: `ZERO_TASK_ID`<br/>
required: `false`

### Upstream Max Connections

The maximum number of connections to open to the upstream database for committing mutations. This is divided evenly amongst sync workers. In addition to this number, zero-cache uses one connection for the replication stream.

Note that this number must allow for at least one connection per sync worker, or zero-cache will fail to start. See num-sync-workers.

flag: `--upstream-max-conns`<br/>
env: `ZERO_UPSTREAM_MAX_CONNS`<br/>
default: `20`


--- release-notes/0.21.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.21
```

## Upgrading

There is one [breaking change](#breaking-changes) in this release, but we think it is unlikely to affect anyone since the results were wrong already – the change just makes the error explicit.

See [hello-zero](https://github.com/rocicorp/hello-zero/pull/44) for an example of using arrays and the new `ZeroProvider` features.

## Features

- New *"ztunes"* sample using TanStack, Drizzle, Better Auth, and Fly.io ([docs](/samples#ztunes)).
- Add initial support for Postgres arrays ([docs](/docs/postgres-support#column-types), [bug](https://bugs.rocicorp.dev/issue/3617)).
- Improved React lifecycle management with `ZeroProvider` ([docs](/docs/react#zero-provider), [PR](https://github.com/rocicorp/mono/pull/4527)).
- Expose `Zero` instances automatically at `__zero` ([docs](/docs/debug/inspector#creating-an-inspector), [PR](https://github.com/rocicorp/mono/pull/4526)).
- Add `--output-{synced|vended}-rows` to `analyze-query` ([PR](https://github.com/rocicorp/mono/pull/4463)).
- Technically a bug fix, but this was so annoying I'm calling it a feature: <nobr>`zero-sqlite3`</nobr> now correctly supports the up/down arrow keys ([commit](https://github.com/rocicorp/zero-sqlite3/commit/50c7a850572d9d3d4ac26ffb2e1cca2e8c5e08b0)).
- Another super annoying fix: logs from `zero-cache` are now level-colored ([PR](https://github.com/rocicorp/mono/pull/4533)).

## Fixes

- Lazy-load otel. This was causing problems with `pnpm` ([PR](https://github.com/rocicorp/mono/pull/4492)).
- Initial replication is now memory-bounded ([PR](https://github.com/rocicorp/mono/pull/4408)).
- Change the way otel starts up in `zero-cache-dev` to not rely on `npx` ([PR](https://github.com/rocicorp/mono/pull/4448)).
- Use existing `--log-slow-hydrate-threshold` as the threshold for `--query-hydration-stats` rather than hardcoded 200ms.
- Fix race condition starting up in multinode deployments ([PR](https://github.com/rocicorp/mono/pull/4400)).
- Avoid site-local IPv6 addresses in auto-discovery ([PR](https://github.com/rocicorp/mono/pull/4414)).
- Many z2s fixes found by fuzzing (PRs: [4415](https://github.com/rocicorp/mono/pull/4415), [4416](https://github.com/rocicorp/mono/pull/4416), [4417](https://github.com/rocicorp/mono/pull/4417), [4421](https://github.com/rocicorp/mono/pull/4421), [4422](https://github.com/rocicorp/mono/pull/4422), [4423](https://github.com/rocicorp/mono/pull/4423)).
- Don't load prettier in `analyze-query`. This was causing problems when prettier config was cjs. ([PR](https://github.com/rocicorp/mono/pull/4441)).
- Don't hydrate system relationships in `analyze-query`. This was causing incorrect results. ([PR](https://github.com/rocicorp/mono/pull/4458)).
- Fix memory leaks from not cleaning up `pusher` and `mutagen` ([PR](https://github.com/rocicorp/mono/pull/4481)).
- Fix handling of invalid websocket requests that were crashing server. ([PR](https://github.com/rocicorp/mono/pull/4490)).
- Remove red error text when `.env` missing ([PR](https://github.com/rocicorp/mono/pull/4523)).
- Allow `zero-cache` to startup without schema file, but print a warning ([PR](https://github.com/rocicorp/mono/pull/4531)).
- Log a warning when auth token exceeds max allowed header size ([PR](https://github.com/rocicorp/mono/pull/4535)).

## Breaking Changes

- Using `order by` and `limit` in many-to-many relationships now throws an error. It didn't work before but did the wrong thing silently. Now it throws a runtime error. See [docs](/docs/reading-data#order-and-limit-not-supported-in-junction-relationships), [bug](https://bugs.rocicorp.dev/issue/3527).


--- release-notes/0.20.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.20
```

## Upgrading

There are two config changes for multinode deployments:

* **Required:** Remove `view-syncer`'s `ZERO_CHANGE_STREAMER_URI` env var and replace it with `ZERO_CHANGE_STREAMER_MODE: "discover"`.
* **Optional:** Change the `ZERO_LITESTREAM_BACKUP_URL` env var from being passed to both `replication-manager` and `view-syncer` nodes to being passed only to `replication-manager`. This config is no longer needed by `view-syncer` (and is ignored by it).

See [hello-zero](https://github.com/rocicorp/hello-zero/pull/40/files#diff-6ae5efcb80db51f4b137515e1674caaf950405317e84e2961cf38f9d3a1804b9) for an upgrade example using SST.

Additionally, the `ZERO_TENANTS_JSON`, feature was removed. We do not think anyone was using it, but if you were please [reach out to us](https://discord.rocicorp.dev/) for options.

## Features

- Supabase is now fully supported. After upgrading, you should see that schema changes are incremental and don't reset the replica ([docs](/docs/connecting-to-postgres)).
- Improve performance of single-key reads on client. Scale depends on size of data but 100x improvement is common ([PR](https://github.com/rocicorp/mono/pull/4327)).
- Implement short-circuiting for `or` queries. Because of permissions, one or more branches of `or` would often be empty, turning the entire `or` into a full-table scan. 100x improvement on chinook test dataset ([PR](https://github.com/rocicorp/mono/pull/4339)).
- Remove DNF conversion. This was intended to make consistency easier in the future, but was resulting in some queries exploding in size ([PR](https://github.com/rocicorp/mono/pull/4386), [bug](https://bugs.rocicorp.dev/issue/3569)).
- Autodiscovery for `replication-manager`. `view-syncer` nodes now find `replication-manager` using the Postgres `changedb` database, and no longer need an internal load balancer. See the new `ZERO_CHANGE_STREAMER_MODE: "discover"` config in the [deployment docs](/docs/deployment) ([PR](https://github.com/rocicorp/mono/pull/4335)).
- Make `ZERO_LITESTREAM_BACKUP_URL` specific to `replication-manager`. `view-syncer` nodes now ignore this config and learn it from `replication-manager` instead. This makes restarting replication less error-prone ([PR](https://github.com/rocicorp/mono/pull/4344), [discussion](https://discord.com/channels/830183651022471199/1370898291721506876/1371685592152150057)).
- OpenTelemetry support ([docs](/docs/debug/otel)).

## Fixes

- Allow dots in column names (only works with custom mutators) ([PR](https://github.com/rocicorp/mono/pull/4323)).
- Fix websocket liveness check to avoid false negatives when busy ([PR](https://github.com/rocicorp/mono/pull/4321)).
- Fix unhandled exception in `zero-cache` when processing query eviction ([PR](https://github.com/rocicorp/mono/pull/4322)).
- Keep microsecond precision across timezones ([PR](https://github.com/rocicorp/mono/pull/4257)).
- Fix unhandled exception in `zero-cache` during `handleClose` ([PR](https://github.com/rocicorp/mono/pull/4326)).
- Fix `NOT IN` in `z2s` ([PR](https://github.com/rocicorp/mono/pull/4329)).
- Mutators: assert provided columns actually exist ([PR](https://github.com/rocicorp/mono/pull/4337)).
- Fix ordering of columns in replicated index ([PR](https://github.com/rocicorp/mono/pull/4362)).
- Use a shorter keepalive for replication stream for compat with Neon ([PR](https://github.com/rocicorp/mono/pull/4363)).
- Allow destructuring `where` in `query.related` ([PR](https://github.com/rocicorp/mono/pull/4366)).
- Add flow control for large change DB transactions ([PR](https://github.com/rocicorp/mono/pull/4371)).
- Fix `z2s` handling of pg types with params (char, varchar, numeric, etc) ([PR](https://github.com/rocicorp/mono/pull/4361)).
- Support `from` and `--schema-path` in `analyze-query` ([PR](https://github.com/rocicorp/mono/pull/4368)).

## Breaking Changes

- The autodiscovery feature for `replication-manager` is a breaking change for multinode deployments. See the [upgrade instructions](#upgrading) for details.
- The `ZERO_TENANTS_JSON` config was removed 🫗.
- The `ZERO_INITIAL_SYNC_ROW_BATCH_SIZE` config was removed. It is no longer needed because initial sync now adapts to available memory automatically.


--- release-notes/0.19.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.19
```

## Upgrading

* If you use custom mutators, please see [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/pull/18/files) for how to update your push endpoint.
* If you use SolidJS, please switch to [`createQuery`](https://github.com/rocicorp/hello-zero-solid/pull/18/files).
* If you are `awaiting z.mutate.foo.bar()`, you should [switch to `await z.mutate.foo.bar().client`](/docs/custom-mutators#waiting-for-mutator-result) to be consistent with `.server`.
* If you were using a 0.19 canary, the `.server` property [returns error by rejection again](/docs/custom-mutators#waiting-for-mutator-result) (like 0.18 did). Sorry about the thrash here.

## Features

- Add a `type` param to `query.run()` so it can wait for server results ([doc](/docs/reading-data#running-queries-once), [bug](https://bugs.rocicorp.dev/issue/3243))
- `await z.mutate.foo.bar()` is now `await z.mutate.foo.bar().client` for consistency with `.server`, old API still works but deprecated ([doc](/docs/custom-mutators#waiting-for-mutator-result))
- Improve speed of litestream restore by about 7x
- Increase replication speed when using JSON by about 25%
- Add options to `analyze-query` to apply permissions and auth data ([doc](/docs/debug/permissions#read-permissions)).
- Add option to `--lazy-startup` to `zero-cache` to delay connecting to upstram until first connection ([doc](/docs/zero-cache-config#lazy-startup))
- Add `/statz` endpoint for getting some health statistics from a running Zero instance ([doc](/docs/debug/slow-queries#statz))

## Fixes

- Support passing `Request` to `PushProccesor.process()` ([PR](https://github.com/rocicorp/mono/pull/4214))
- Fix layering in `PushProcessor` to better support custom db implementations (thanks Erik Munson!) ([PR](https://github.com/rocicorp/mono/pull/4251))
- Fix socket disconnects in GCP ([PR](https://github.com/rocicorp/mono/pull/4173))
- Quote Postgres enum types to preserve casing ([report](https://discord.com/channels/830183651022471199/1358217995188437074/1358218))
- `z2s`: Return `undefined` for empty result set when using `query.one()`
- `z2s`: Allow accessing tables in non-public schemas
- `z2s`: Allow `tx.foo.update({bar: undefined})` where `bar` is `optional` to match client behavior
- Fix broken replication when updating a key that is part of a unique (but non-PK) index
- `solid`: Rename `useQuery` to `createQuery` to fit Solid naming conventions (old name deprecated)
- Resync when publications are missing ([PR](https://github.com/rocicorp/mono/pull/4205))
- Fix missing `NOT LIKE` in `query.where()` ([PR](https://github.com/rocicorp/mono/pull/4217))
- Fix timezone shift when writing to `timestamp`/`timestamptz` and server is non-UTC timezone (thanks Tom Jenkinson!) ([PR](https://github.com/rocicorp/mono/pull/4216))
- Bound time spent in incremental updates to 1/2 hydration time
- Fix `ttl` being off by 1000 in some cases 😬 ([PR](https://github.com/rocicorp/mono/pull/4225))
- `z2s`: Relationships nested in a junction relationship were not working correctly ([PR](https://github.com/rocicorp/mono/pull/4221))
- Custom mutators: Due to multitab, client can receive multiple responses for same mutation
- Fix deadlock that could happen when pushing on a closed websocket ([PR](https://github.com/rocicorp/mono/pull/4256))
- Fix incorrect shutdown under heavy CPU load  (thanks Erik Munson!) ([PR](https://github.com/rocicorp/mono/pull/4252))
- Fix case where deletes were getting reverted (thanks for reproduction Marc MacLeod!) ([PR](https://github.com/rocicorp/mono/pull/4282))
- `z2s`: Incorrect handling of self-join, and not exists
- `not(exists())` is not supported on the client
- re-auth on 401s returned by push endpoint
- Added `push.queryParams` constructor parameter to allow passing query params to the push endpoint ([doc](/docs/custom-mutators#setting-up-the-server))

## Breaking Changes

- The structure of setting up a `PushProcesor` has changed slightly. See [push endpoint setup](/docs/custom-mutators#setting-up-the-server) or [upgrade guide](#upgrading).
- Not technically a breaking change from 0.18, but if you were using 0.19 canaries, the `.server` property returns error by rejection again (like 0.18 did) ([doc](/docs/custom-mutators#waiting-for-mutator-result)).


--- release-notes/0.18.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.18
```

## Upgrading

To try out custom mutators, see the changes to [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/pull/17).

## Features

- Custom Mutators! Finally! Define arbitrary write operations in code ([doc](/docs/custom-mutators)).
- Added inspector API for debugging sync, queries, and client storage ([doc](/docs/debug/inspector)).
- Added `analyze-query` tool to debug query performance ([doc](/docs/debug/slow-queries#query-plan)).
- Added `transform-query` tool to debug permissions ([doc](/docs/debug/permissions#read-permissions)).
- Added `ast-to-zql` script to prettify Zero's internal AST format ([doc](/docs/debug/query-asts)).

## Fixes

- Added backpressure to `replication-manager` to protect against Postgres moving faster than we can push to clients ([PR](https://github.com/rocicorp/mono/pull/4089)).
- `@rocicorp/zero/advanced` has been deprecated. `AdvancedQuery` got folded into `Query` and `ZeroAdvancedOptions` got folded into `ZeroOptions` ([PR](https://github.com/rocicorp/mono/pull/4086)).
- Support `ALTER SCHEMA` DDL changes ([PR](https://github.com/rocicorp/mono/pull/4098))
- Allow `replication-manager` to continue running while a new one re-replicates. ([PR](https://github.com/rocicorp/mono/pull/4124)).
- Improve replication performance for some schema changes ([PR](https://github.com/rocicorp/mono/pull/4151)).
- Make the log level of `zero-deploy-permissions` configurable ([PR](https://github.com/rocicorp/mono/pull/4002))
- Bind `exists` to the expression builder ([PR](https://github.com/rocicorp/mono/pull/4010))
- Fix `single output already exists` error ([PR](https://github.com/rocicorp/mono/pull/4020))
- Fix `getBrowserGlobal('window')?.addEventListener not a function` in Expo (thanks `@andrewcoelho`!) ([PR](https://github.com/rocicorp/mono/pull/4037)).
- Fix Vue bindings ref counting bug. Bindings no longer need to pass `RefCountMap` ([PR](https://github.com/rocicorp/mono/pull/4013)).
- Fix CVR ownership takeover race conditions ([PR](https://github.com/rocicorp/mono/pull/4071)).
- Support `REPLICA IDENTITY FULL` in degraded-mode pg providers ([PR](https://github.com/rocicorp/mono/pull/4131)).
- Handle corrupt sqlite db by re-replicating ([PR](https://github.com/rocicorp/mono/pull/4133)).
- Don't send useless pokes to clients that are unchanged ([PR](https://github.com/rocicorp/mono/pull/4149)).
- Add `limit(1)` to queries using a relation that is marked `one()` ([PR](https://github.com/rocicorp/mono/pull/4154)).
- Export `UseQueryOptions`

## Breaking Changes

None.


--- release-notes/0.17.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.17
```

## Upgrading

See the upgrade from [hello-zero](https://github.com/rocicorp/hello-zero/pull/31) or [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/pull/16) for an example.

## Features

- Queries now take an optional `ttl` argument. This argument _backgrounds_ queries for some time after the app stops using them. Background queries continue syncing so they are instantly ready if the UI re-requests them. The data from background queries is also available to be used by new queries where possible ([doc](/docs/reading-data#query-lifecycle)).
- Structural schema versioning. This is TypeScript, why are we versioning with numbers like cave-people?? We got rid of `schemaVersion` concept entirely and now determine schema compatibility completely automatically, TS-stylie ([doc](/docs/zero-schema/#migrations)).
- Permissions now scoped to _"apps"_. You can now have different Zero "apps" talking to the same upstream database. Each app gets completely separate configuration and permissions. This should also enable previewing `zero-cache` (each preview would be its own app). Apps replace the existing "shard" concept ([doc](/docs/zero-cache-config#app-id)).
- Initial replication is over 5x faster, up to about 50MB/second or 15k row/second in our tests.
- Added warnings for slow hydration in both client and server ([doc](/docs/reading-data#thinking-in-queries)).
- `auto-reset` is now enabled by default for databases that don't support event triggers ([doc](/docs/connecting-to-postgres#schema-changes)).
- Default `cvr` and `change` databases to `upstream`, so that you don't have to specify them in the common case where they are the same as upstream.
- This docs site now has search!

## Fixes

- Certain kinds of many:many joins were causing `node already exists` assertions
- Certain kinds of `or` queries were causing consistency issues
- Support `replica identity full` for PostgreSQL tables
- We now print a stack trace during close at `debug` level to enable debugging errors where Zero is accessed after close.
- We now print a warning when `IndexedDB` is missing rather than throwing. This makes it a little easier to use Zero in SSR setups.
- We now reset `zero-cache` implicitly in a few edge cases rather than halting replication.
- Fixed a deadlock in `change-streamer`.

## Breaking Changes

- `query.run()` now returns its result via promise. This is required for compatibility with upcoming custom mutators, but also will allow us to wait for server results in the future (though that (still 😢) doesn't exist yet).


--- release-notes/0.16.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.16
```

## Upgrading

See the upgrade from [hello-zero](https://github.com/rocicorp/hello-zero/commit/156e829eef91ff8b92e189c800ec6eba7213c383) or [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/commit/4f8d2b055b32efa9434df6f514007dce5f1d2c0b) for an example.

## Features

- Documented how to use lambdas to deploy permissions in SST, rather than needing CI/CD to have access to Postgres. ([doc](/docs/deployment#initialize-sst) – search for "`permissionsDeployer`").
- Added simple debugging logs for read and write permissions ([doc](/docs/debug/permissions)).

## Fixes

- Improve performance of initial sync about 2x ([PR 1](https://github.com/rocicorp/mono/pull/3836), [PR 2](https://github.com/rocicorp/mono/pull/3835)).
- `IN` should allow `readonly` array arguments ([Report](https://discord.com/channels/830183651022471199/1288232858795769917/1340464538704937082), [PR](https://github.com/rocicorp/mono/pull/3819)).
- Export `ANYONE_CAN_DO_ANYTHING` ([Report](https://discord.com/channels/830183651022471199/1340088459544756276/1340674831347355711)).
- Fix false-positive in schema change detection ([Report](https://discord.com/channels/830183651022471199/1288232858795769917/1341135548944744448), [PR](https://github.com/rocicorp/mono/pull/3828)).
- Fix writes of numeric types ([Report](https://discord.com/channels/830183651022471199/1288232858795769917/1341076949749071955), [PR](https://github.com/rocicorp/mono/pull/3750))
- Fix bug where litestream was creating way too many files in s3 ([PR](https://github.com/rocicorp/mono/pull/3839))
- Fix memory leak in change-streamer noticeable under high write load ([PR](https://github.com/rocicorp/mono/pull/3859))
- Fix `query already registered` error ([PR](https://github.com/rocicorp/mono/pull/3840))
- Correctly handle optional booleans ([PR](https://github.com/rocicorp/mono/pull/3863))
- Ignore indexes with unpublished columns ([PR](https://github.com/rocicorp/mono/pull/3862))

## Breaking Changes

None.


--- release-notes/0.15.mdx ---

<Video src="https://d2nzlypqdo5xbb.cloudfront.net/video/release-notes/roci-releasenotes-0.15-be-sweet-v5.mp4" />

## Install

```bash
npm install @rocicorp/zero@0.15
```

## Upgrade Guide

This release changes the way that permissions are sent to the server. Before, permissions were sent to the server by setting the `ZERO_SCHEMA_JSON` or `ZERO_SCHEMA_FILE` environment variables, which include the permissions.

In 0.15, these variables go away and are replaced by a new command: `npx zero-deploy-permissions`. This command writes the permissions to a new table in the upstream database. This design allows live permission updates, without restarting the server. It also solves problems with max env var size that users were seeing.

This release also flips the default permission from `allow` to `deny` for all rules.

To upgrade your app:

1. See the changes to [hello-zero](https://github.com/rocicorp/hello-zero/pull/26) or [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/pull/14) for how to update your permissions.
2. Remove the `ZERO_SCHEMA_JSON` and `ZERO_SCHEMA_FILE` environment variables from your setup. They aren't used anymore.
3. Use [`npx zero-deploy-permissions`](/docs/permissions#permission-deployment) to deploy permissions when necessary. You can hook this up to your CI to automate it. See the [zbugs implementation](https://github.com/rocicorp/mono/blob/86ab73122a0532e4ec516badc1d8fb82b3465b49/prod/sst/sst.config.ts#L178) as an example.

## Features

- Live-updating permissions ([docs](/docs/permissions#permission-deployment)).
- Permissions now default to **deny** rather than **allow** ([docs](/docs/permissions#access-is-denied-by-default)).

## Fixes

- Multiple `whereExists` in same query not working ([PR](https://github.com/rocicorp/mono/pull/3746))
- Allow overlapped mutators ([bug](https://bugs.rocicorp.dev/issue/3529))
- "Immutable type too deep" error ([PR](https://github.com/rocicorp/mono/pull/3758))
- Log server version at startup ([PR](https://github.com/rocicorp/mono/pull/3737))
- Eliminate quadratic CVR writes ([PR](https://github.com/rocicorp/mono/pull/3736))
- Handle `numeric` in the replication stream ([PR](https://github.com/rocicorp/mono/pull/3750))
- Make the auto-reset required error more prominent ([PR](https://github.com/rocicorp/mono/pull/3794))
- Add `"type":"module"` recommendation when schema load fails ([PR](https://github.com/rocicorp/mono/pull/3797))
- Throw error if multiple auth options set ([PR](https://github.com/rocicorp/mono/pull/3807))
- Handle NULL characters in JSON columns ([PR](https://github.com/rocicorp/mono/pull/3810))

## Breaking Changes

- Making permissions deny by default breaks existing apps. To fix add `ANYONE_CAN` or other appropriate permissions for your tables. See [docs](/docs/permissions#access-is-denied-by-default).
- The `ZERO_SCHEMA_JSON` and `ZERO_SCHEMA_FILE` environment variables are no longer used. Remove them from your setup and use [`npx zero-deploy-permissions`](/docs/permissions#permission-deployment) instead.


--- release-notes/0.14.mdx ---

<Video src="https://d2nzlypqdo5xbb.cloudfront.net/video/release-notes/zero-release-0.14-call-me-al-v6.mp4" />

## Install

```bash
npm install @rocicorp/zero@0.14
```

## Features

- Use `from()` to map column or tables to a different name ([docs](../zero-schema#name-mapping)).
- Sync from muliple Postgres schemas ([docs](../zero-schema#multiple-schemas))

## Fixes

- `useQuery` not working when `server` unset ([bug](https://bugs.rocicorp.dev/issue/3497))
- Error: "single output already exists" in hello-zero-solid ([bug](https://bugs.rocicorp.dev/issue/3488))
- `Row` helper doesn't work with query having `one()` ([bug](https://bugs.rocicorp.dev/issue/3503))
- Partitioned Postgres tables not replicating

## Breaking Changes

None.


--- release-notes/0.13.mdx ---

<Video src="https://d2nzlypqdo5xbb.cloudfront.net/video/release-notes/zero-releasenotes-0.13-thumb.mp4" />

## Install

```bash
npm install @rocicorp/zero@0.13
```

## Features

- Multinode deployment for horizontal scalability and zero-downtime deploys ([docs](/docs/deployment#architecture)).
- SST Deployment Guide ([docs](/docs/deployment#guide-multi-node-on-sstaws)).
- Plain AWS Deployment Guide ([docs](/docs/deployment#guide-multi-node-on-raw-aws)).
- Various exports for external libraries
- Remove build hash from docker version for consistency with npm ([discussion](https://discord.com/channels/830183651022471199/1325165395015110688/1333906735060226161))

## Fixes

- Move heartbeat monitoring to separate path, not port
- Type instantiation is excessively deep and possibly infinite ([bug](https://bugs.rocicorp.dev/issue/3477)).
- 20x improvement to `whereExists` performance ([discussion](https://github.com/rocicorp/mono/pull/3629#issuecomment-2621976119))

## Breaking Changes

- Removing the hash from the version is a breaking change if you had scripts relying on that.
- Moving the heartbeat monitor to a path is a breaking change for deployments that were using that.


--- release-notes/0.12.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.12
```

## Features

- Schemas now support circular relationships ([docs](/docs/zero-schema#circular-relationships)).
- Added `one()` and `many()` schema helpers to default relationship type ([docs](/docs/zero-schema#table-schemas)).
- Support for syncing tables without a primary key as long as there is a unique index. This enables Prisma's [implicit many-to-many relations](https://www.prisma.io/docs/orm/prisma-schema/data-model/relations/many-to-many-relations#implicit-many-to-many-relations) ([docs](/docs/postgres-support#primary-keys)).
- Zero has been confirmed to work with Aurora and Google Cloud SQL ([docs](/docs/connecting-to-postgres))
- Client bundle size reduced from 55kb to 47kb (-15%).

## Fixes

- Windows: `zero-cache` was spawning emptying terminals and leaving listeners connected on exit.
- Incorrect warning in `zero-cache` about enums not being supported.
- Failure to handle the primary key of Postgres tables changing.
- Incorrect results when `whereExists()` is before `where()` in query ([bug](https://bugs.rocicorp.dev/issue/3417)).
- Error: _The inferred type of '...' cannot be named without a reference to ..._.
- Error: _insufficient upstream connections_.
- Several causes of flicker in React.
- Incorrect values for `ResultType` when unloading and loading a query quickly ([bug](https://bugs.rocicorp.dev/issue/3456)).
- Error: _Postgres is missing the column '...' but that column was part of a row_.
- Pointless initial empty render in React when data is already available in memory.
- Error: _Expected string at ... Got array_ during auth.
- `where()` incorrectly allows comparing to `null` with the `=` operator ([bug](https://bugs.rocicorp.dev/issue/3426)).
- SolidJS: Only call `setState` once per transaction.

## Breaking Changes

- The schema definition syntax has changed to support circular relationships. See the changes to [`hello-zero`](https://github.com/rocicorp/hello-zero/commit/70cd15d5631436c058518d154acb3495b718970e) and [`hello-zero-solid`](https://github.com/rocicorp/hello-zero-solid/commit/c8932a7ff06cbdc02759b3b48592ed61055d4cd3) for upgrade examples.


--- release-notes/0.11.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.11
```

## Features

- Windows should work a lot better now. Thank you very much to [aexylus](https://aexylus.com/) and [Sergio Leon](https://www.cbnsndwch.io/) for the testing and contributions here.
- Support nested property access in JWT auth tokens ([docs](/docs/permissions#rules)).
- Make initial sync configurable ([docs](/docs/zero-cache-config#initial-sync-table-copy-workers)).
- Add query result type to SolidJS ([docs](/docs/reading-data#completeness))
- Docker image now contains native amd64 and arm64 binaries.
- Add `storageKey` constructor parameter to enable multiple `Zero` instances for same `userID`.

## Fixes

Many, many fixes, including:

- Fix downstream replication of primitive values
- Fix replication of `TRUNCATE` messages
- Fix large storage use for idle pg instances
- Add runtime sanity checks for when a table is referenced but not synced
- Fix `zero-cache-dev` for multitenant

## Breaking Changes

- The addition of result types to SolidJS is a breaking API change on SolidJS only. See the changes to [`hello-zero-solid`](https://github.com/rocicorp/hello-zero-solid/commit/7c6c3a47479f037f8323b102013244881c74fe9e) for upgrade example.


--- release-notes/0.10.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.10
```

## Features

- None.

## Fixes

- Remove top-level await from `zero-client`.
- Various logging improvements.
- Don't throw error when `WebSocket` unavailable on server.
- Support building on Windows (running on Windows still doesn't work)

## Breaking Changes

- None.


--- release-notes/0.9.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.9
```

See the changes to [hello-zero](https://github.com/rocicorp/hello-zero/pull/8) or [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/pull/5) for example updates.

## Features

- **JWK Support**. For auth, you can now specify a JWK containing a public key, or a JWKS url to support autodiscovery of keys. ([docs](../auth/))
- **UUID column**. Zero now supports the `uuid` Postgres column type. ([docs](../postgres-support#column-types))

## Fixes

- **Readonly Values**. Type of values returned from Zero queries are marked `readonly`. The system always considered them readonly, but now the types reflect that. ([docs](../reading-data/))

## Breaking Changes

- The `zero-cache` config `ZERO_JWT_SECRET` has been renamed to `ZERO_AUTH_SECRET` for consistency with the new JWK-related keys. If you were using the old name, you'll need to update your `.env` file.
- All values returned by Zero are now `readonly`. You'll probably have to add this TS modifier various places. If you find yourself casting away `readonly` you probably should be cloing the value instead.


--- release-notes/0.8.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.8
```

See the changes to [hello-zero](https://github.com/rocicorp/hello-zero/pull/8) or [hello-zero-solid](https://github.com/rocicorp/hello-zero-solid/pull/5) for example updates.

## Features

- **Schema Autobuild**. There's now a `zero-cache-dev` script that automatically rebuilds the schema and restarts `zero-cache` on changes to `schema.ts`. ([docs](../zero-schema#building-the-zero-schema))
- **Result Type.** You can now tell whether a query is complete or partial. ([docs](/docs/reading-data#completeness))
- **Enums**. Enums are now supported in Postgres schemas and on client. ([docs](../postgres-support#column-types))
- **Custom Types**. You can define custom JSON types in your schema. ([docs](../zero-schema#custom-json-types))
- **OTEL Tracing.** Initial tracing support. ([docs](/docs/zero-cache-config#log-trace-collector))
- **timestampz.** Add support for `timestampz` Postgres column type. ([docs](../postgres-support#column-types))
- **SSLMode**. You can disable TLS when `zero-cache` connects to DB with `sslmode=disable`. ([docs](../connecting-to-postgres#ssl-mode))
- **Permission Helpers**. `ANYONE_CAN` and `NOBODY_CAN` helpers were added to make these cases more readable. ([docs](../permissions#permissions-denied-by-default))
- **Multitenant Support**. A single `zero-cache` can now front separate Postgres databases. This is useful for customers that have one "dev" database in production per-developer. ([docs](../zero-cache-config#tenants-json))

## Fixes

- **Crash with JSON Columns**. Fixed a crash when a JSON column was used in a Zero app with write permissions ([bug](https://bugs.rocicorp.dev/issue/3215))
- **Better Connection Error Reporting**. Some connection errors would cause `zero-cache` to exit silently. Now they are returned to client and logged.

## Breaking Changes

- `useQuery` in React now returns a 2-tuple of `[rows, result]` where `result` is an object with a `type` field.
- `postProposedMutation` in write permissions for `update` renamed to `postMutation` for consistency.
- `TableScheamToRow` renamed to `Row` to not be so silly long.


--- release-notes/0.7.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.7
```

## Features

- **Read permissions.** You can now control read access to data using ZQL ([docs](../permissions#select-permissions)).
- **Deployment.** We now have a single-node Docker container ([docs](../deployment/)). Future work will add multinode support.
- **Compound FKs.** Zero already supported compound _primary_ keys, but now it also supports compound _foreign_ keys ([docs](../zero-schema#relationships-and-compound-keys)).
- **Schema DX**:
  - Columns types can use bare strings now if `optional` is not needed ([example](https://github.com/rocicorp/mono/commit/212379241e27e717f1237946f3384127d06661c3#diff-01e627d4886ffc106a9f60c5ea65f35b3868ad4de898cecf7ae60329b11c22e7R13)).
  - PK can be a single string in the common case where it’s non-compound ([example](https://github.com/rocicorp/mono/commit/212379241e27e717f1237946f3384127d06661c3#diff-01e627d4886ffc106a9f60c5ea65f35b3868ad4de898cecf7ae60329b11c22e7R19)).

## Breaking Changes

- Several changes to `schema.ts`. See [update](https://github.com/rocicorp/hello-zero/commits/main/) to `hello-zero` for overview. Details:
  - `defineAuthorization` was renamed to `definedPermissions` to avoid confusion with _authentication_.
  - The way that many:many relationships are defined has changed to be more general and easy to remember. See example.
  - The signature of `definePermissions` and the related rule functions have changed:
    - Now rules return an _expression_ instead of full query. This was required to make read permissions work and we did it for write permissions for consitency (see example).
    - The `update` policy now has two child policies: `preMutation` and `postMutation`. The rules we used to have were `preMutation`. They run before a change and can be used to validate a user has permission to change a row. The `postMutation` rules run after and can be used to limit the changes a user is allowed to make.
  - The `schema.ts` file should export an object having two fields: `schema` and `permissions`.
- The way that `schema.ts` is consumed has also changed. Rather than `zero-cache` directly reading the typescript source, we compile it to JSON and read that.
  - `ZERO_SCHEMA_FILE` should now point to a JSON file, not `.ts`. It defaults to `./zero-schema.json` which we’ve found to be pretty useful so you’ll probably just remove this key from your `.env` entirely.
  - Use `npx zero-build-schema` to generate the JSON. You must currently do this manually each time you change the schema, we will automate it soon.

<Note type="note">
  We compile the schema to JSON so that we can use it on the server without
  needing a TS toolchain there. Also so that we can run a SaaS in the future
  without needing to run user code.
</Note>

## zbugs

- Comments [now have permalinks](https://bugs.rocicorp.dev/issue/3067#comment-qt7YPQxXsBMBqcOkkO1pY). Implementing permalinks in a synced SPA [is fun](https://github.com/rocicorp/mono/commit/384d0955a3998d68d293985b0de89c5302076ec5)!
- Private issues. Zbugs now supports private (to team only) issues. I wonder what’s in them … 👀.

## Docs

- [The docs have moved](https://zero.rocicorp.dev/). Please don’t use Notion anymore, they won’t be updated.


--- release-notes/0.6.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.6
```

## Upgrade Guide

This release is a bit harder to upgrade to than previous alphas. For a step-by-step guide, please refer to the commits that upgrade the React and Solid quickstart apps:

- [Upgrading hello-zero from Zero 0.5 to 0.6](https://github.com/rocicorp/hello-zero/compare/ee837552be8419fbcbe4c5887609c89a7b1c4e07...8a0d29149bac0ab10aa25de3ebdb25ab70bc0d96)
- [Upgrading hello-zero-solid from Zero 0.5 to 0.6](https://github.com/rocicorp/hello-zero-solid/compare/79405b2da06b059a184abec69fdc20f071c58c4d...f4fed1ab7555bdd3bc131536863d60e799de571a)

## Breaking Changes

- Totally new configuration system.
  - `zero.config.ts` is no more – config is now via env vars ([documentation](/docs/zero-cache-config)).
  - Permissions rules moved into schema ([documentation](/docs/auth#permissions)).
- Renamed CRUD mutators to be consistent with SQL naming ([bug](https://bugs.rocicorp.dev/issue/3144), [documentation](/docs/writing-data)).
  - `z.mutate.<table>.create -> insert`
  - `z.mutate.<table>.put -> upsert`
- Removed `select` from ZQL. It wasn’t doing anything ([documentation](/docs/reading-data))
- Moved batch mutation to its own `mutateBatch` method. Before the `mutate` field also doubled as a method. This made intellisense hard to understand since `z.mutate` had all the tables as fields but also all the fields of a function.

## Features

- Relationship filters. Queries can now include `whereExists` ([bug](https://bugs.rocicorp.dev/issue/3039), [documentation](/docs/reading-data#relationship-filters)).
- Reworked syntax for compound `where` filters, including ergonomically building `or` expressions with dynamic number of clauses ([bug](https://bugs.rocicorp.dev/issue/3104), [documentation](/docs/reading-data#compound-filters)).
- Support using Postgres databases without superuser access for smaller apps ([documentation](/docs/connecting-to-postgres)).
- Support for running `Zero` client under Cloudflare Durable Objects ([documentation](/docs/samples#hello-zero-do)).
- Reworked support for `null` / `undefined` to properly support optional fields ([bug](https://bugs.rocicorp.dev/issue/3114), [documentation](/docs/zero-schema#optional-columns)).
- Added `IS` / `IS NOT` to ZQL to support checking for null ([bug](https://bugs.rocicorp.dev/issue/3028), [documentation](/docs/reading-data#comparing-to-null)).
- Improved intellisense for mutators.
- Added `--port` flag and `ZERO_PORT` environment variable ([bug](https://bugs.rocicorp.dev/issue/3031), [documentation](/docs/zero-cache-config)).
- Default max connections of zero-cache more conservatively so that it should fit with even common small Postgres configurations.
- `zero-cache` now accepts requests with any base path, not just `/api`. The `server` parameter to the `Zero` client constructor can now be a host (`https://myapp-myteam.zero.ms`) or a host with a single path component (`https://myapp-myteam.zero.ms/zero`). These two changes together allow hosting `zero-cache` on same domain with an app that already uses the `/api` prefix ([bug](https://bugs.rocicorp.dev/issue/3115)).
- Allow Postgres columns with default values, but don’t sync them ([documentation](/docs/postgres-support#column-defaults)).
- The `npx zero-sqlite` utility now accepts all the same flags and arguments that `sqlite3` does ([documentation](/docs/debugging/replication)).

## zbugs

- Added tooltip describing who submitted which emoji reactions
- Updated implementation of label, assignee, and owner filters to use relationship filters
- Updated text filter implementation to use `or` to search description and comments too

## Docs

- Added new [ZQL reference](/docs/reading-data)
- Added new [mutators reference](/docs/writing-data)
- Added new [config reference](/docs/zero-cache-config)


--- release-notes/0.5.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.5
```

## Breaking changes

- `createTableSchema` and `createSchema` moved to `@rocicorp/zero/schema` subpackage. This is in preparation to moving authorization into the schema file.
- `SchemaToRow` helper type was renamed `TableSchemaToRow` and moved into `@rocicorp/zero/schema`.

Basically:

```diff
- import { createSchema, createTableSchema, SchemaToRow } from "@rocicorp/zero";
+ import { createSchema, createTableSchema, TableSchemaToRow } from "@rocicorp/zero/schema";
```

## Features

- Added support for JSON columns in Postgres ([documentation](/docs/postgres-support)).
- Zero pacakage now includes `zero-sqlite3`, which can be used to explore our sqlite files ([documentation](/docs/recipes)).

## Fixes

- We were not correctly replicating the `char(n)` type, despite documenting that we were.

## Docs

_nothing notable_

## zbugs

_nothing notable_


--- release-notes/0.4.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.4
```

## Breaking changes

The `or` changes modified the client/server protocol. You’ll need to restart zero-cache and clear browser data after updating.

## Added `or` , `and` , and `not` to ZQL ([documentation](/docs/reading-data)).

- Added `query.run()` method ([documentation](/docs/reading-data#running-queries-once)).

## Fixes

- Use `batch()` method in zero-solid to improve performance when multiple updates happen in same frame. To take advantage of this you must use the `createZero` helper from `@rocicorp/zero/solid`, instead of instantiating Zero directly. See the solid [sample app](https://github.com/rocicorp/hello-zero-solid/blob/main/src/main.tsx#L16).
- Postgres tables that were reserved words in SQLite but not Postgres caused crash during replication.
- `LIKE` was not matching correctly in the case of multiline subjects.
- Upstream database and zero database can now be same Postgres db (don’t need separate ports).

## Docs

_nothing notable_

## zbugs

- Use `or` to run text search over both titles and bodies
- prevent j/k in emoji
- preload emojis


--- release-notes/0.3.mdx ---

## Install

```bash
npm install @rocicorp/zero@0.3
```

## Breaking changes

- zero.config file is now TypeScript, not JSON. See: https://github.com/rocicorp/hello-zero/blob/07c08b1f86b526a96e281ee65af672f52a59bcee/zero.config.ts.

## Features

- **Schema Migrations:** Zero now has first-class support for schema migration ([documentation](/docs/zero-schema/#migrations)).
- **Write Permissions:** First-class write permissions based on ZQL ([documentation](/docs/auth)).
- **Date/Time related types:** Zero now natively supports the TIMESTAMP and DATE Postgres types ([sample app](https://github.com/rocicorp/hello-zero/blob/main/src/schema.ts), [documentation](/docs/postgres-support)).
- **SolidJS:** We now have first-class support for SolidJS ([documentation](/docs/solidjs)).
- **Intellisense for Schema Definition:** Introduce `createSchema` and `createTableSchema` helper functions to enable intellisense when defining shemas. See [Sample App](https://github.com/rocicorp/hello-zero/blob/main/src/schema.ts#L10).
- **`escapeLike()` :** Add helper to properly escape strings for use in `LIKE` filters. See [Sample App](https://github.com/rocicorp/hello-zero/blob/main/src/App.tsx#L37).
- **New QuickStart App:** Entirely rewrote the [setup/sample flow](/docs/quickstart) to (a) make it much faster to get started playing with Zero, and (b) demonstrate more features.

## Fixes

- The `@rocicorp/zero` package now downloads a prebuilt sqlite instead of compiling it locally. This significantly speeds up install.
- Support `rds.force_ssl=1` RDS configuration.
- Fixed bug where sibling subqueries could be lost on edit changes.
- Fixes to error handling to ensure zero-cache prints errors when crashing in multiprocess mode.
- If zero-cache hears from a client with an unknown CVR/cookie, zero-cache forces that client to reset itself and reload automatically. Useful during development when server-state is frequently getting cleared.

## Docs

- Started work to make [real docs](/docs/introduction). Not quite done yet.

## zbugs

https://bugs.rocicorp.dev/ (pw: zql)

- Improve startup perf: ~3s → ~1.5s Hawaii ↔ US East. More work to do here but good progress.
- Responsive design for mobile.
- “Short IDs”: Bugs now have a short numeric ID, not a random hash. See [Demo Video](https://discord.com/channels/830183651022471199/1288232858795769917/1298114323272568852).
- First-class label picker.
- Unread indicators.
- Finish j/k support for paging through issues. It’s now “search-aware”, it pages through issues in order of search you clicked through to detail page in.
- Text search (slash to activate — needs better discoverability)
- Emojis on issues and comments
- Sort controls on list view
- remove fps meter temporarily
- numerous other UI polish


--- release-notes/0.2.mdx ---

## Breaking changes

- None

## Features

- “Skip mode”: zero-cache now skips columns with unsupported datatypes. A warning is printed out when this happens:
  <ImageLightbox
    src="/images/release-notes/0.2/skip-mode.png"
    alt="Skip mode warning"
  />
  This makes it easy to use zero-cache with existing schemas that have columns
  Zero can’t handle. You can [pair this with Postgres
  triggers](/docs/postgres-support#column-types) to easily translate unsupported
  types into something Zero can sync.
- Zero now supports compound primary keys. You no longer need to include an extraneous `id` column on the junction tables.

## Fixes

- Change the way Zero detects unsupported environments to work in One (and any other supported env). Before, Zero was looking for WebSocket and indexedDB early on, but indexedDB won’t be present on RN as SQLite will be used. Instead look for indexedDB only at use.
- Require Node v20 explicitly in package.json to prevent accidentally compiling better-sqlite3 with different Node version than running with.
- Ensure error messages early in startup get printed out before shutting down in multiprocess mode.

## Docs

- [Factored out the sample app](https://github.com/rocicorp/my-first-zapp) from the docs into its own Github repo so you can just download it and poke around if you prefer that.

## Source tree fixes

- Run zero-cache from source. You no longer have to build `zero` before running `zbugs`, it picks up the changes automatically.

## zbugs

- Numerous polish/styling fixes
- Change default to ‘open’ bugs
- Add ‘assignee’ field


--- release-notes/0.1.mdx ---

## Breaking changes

- The name of some config keys in `zero.config.json` changed:
  - `upstreamUri` → `upstreamDBConnStr`
  - `cvrDbUri` → `cvrDBConnStr`
  - `changeDbUri` → `changeDBConnStr`
  - `replicaDbFile` → `replicaDBFile`
- Changed default port of `zero-cache` to `4848` . So your app startup should look like `VITE_PUBLIC_SERVER="http://localhost:4848"`.

## Features

- Print a warning to js console when Zero constructor `server` param is `null` or `undefined`
- zero-cache should now correctly bind to both ipv4 and ipv6 loopback addresses. This should fix the issue where using `localhost` to connect to zero-cache on some systems did not work.
- Check for presence of `WebSocket` early in startup of `Zero`. Print a clear error to catch people accidentally running Zero under SSR.
- Fix annoying error in js console in React strict mode from constructing and closing Replicache in quick succession.

## Source tree fixes

These only apply if you were working in the Rocicorp monorepo.

- Fixed issue where zbugs didn’t rebuild when zero dependency changed - generally zbugs build normally again
- The zero binary has the right permissions bit so you don’t have to chmod u+x after build
- Remove overloaded name `snapshot` in use-query.tsx (thanks Scott 🙃)


--- custom-mutators.mdx ---

_Custom Mutators_ are a new way to write data in Zero that is much more powerful than the original ["CRUD" mutator API](./writing-data).

Instead of having only the few built-in `insert`/`update`/`delete` write operations for each table, custom mutators allow you to _create your own write operations_ using arbitrary code. This makes it possible to do things that are impossible or awkward with other sync engines.

For example, you can create custom mutators that:

- Perform arbitrary server-side validation
- Enforce fine-grained permissions
- Send email notifications
- Query LLMs
- Use Yjs for collaborative editing
- … and much, _much_ more – custom mutators are just code, and they can do anything code can do!

Despite their increased power, custom mutators still participate fully in sync. They execute instantly on the local device, immediately updating all active queries. They are then synced in the background to the server and to other clients.

<Note
  slug="deprecating-crud"
  heading="Custom mutators will eventually become Zero's only write API"
>
  We're still refining the design of custom mutators. During this phase, the old
  CRUD mutators will continue to work. But we do want to deprecate CRUD
  mutators, and eventually remove them. So please try out custom mutators and
  [let us know](https://discord.rocicorp.dev/) how they work for you, and what
  improvements you need before the cutover.
</Note>

## Understanding Custom Mutators

### Architecture

Custom mutators introduce a new _server_ component to the Zero architecture.

![diagram](/images/custom-mutators/overview.png)

This server is implemented by you, the developer. It's typically just your existing backend, where you already put auth or other server-side functionality.

The server can be a serverless function, a microservice, or a full stateful server. The only real requirement is that it expose a special _push endpoint_ that `zero-cache` can call to process mutations. This endpoint implements the [push protocol](#custom-push-implementation) and contains your custom logic for each mutation.

Zero provides utilities in `@rocicorp/zero` that make it really easy implement this endpoint in TypeScript. But you can also implement it yourself if you want. As long as your endpoint fulfills the push protocol, `zero-cache` doesn't care. You can even write it in a different programming language.

### What Even is a Mutator?

Zero's custom mutators are based on [_server reconciliation_](https://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html) – a technique for robust sync that has been used by the video game industry for decades.

<Note emoji="🌈" heading="The more you know">
  Our previous sync engine, [Replicache](https://replicache.dev/), also used
  server reconciliation. The ability to implement arbitrary mutators was one of
  Replicache's most popular features. Custom mutators bring this same power to
  Zero, but with a much better developer experience.
</Note>

A custom mutator is just a function that runs within a database transaction, and which can read and write to the database. Here's an example of a very simple custom mutator written in TypeScript:

```ts
async function updateIssue(
  tx: Transaction,
  {id, title}: {id: string; title: string},
) {
  // Validate title length.
  if (title.length > 100) {
    throw new Error(`Title is too long`);
  }

  await tx.mutate.issue.update({id, title});
}
```

Each custom mutator gets **two implementations**: one on the client and one on the server.

The client implementation must be written in TypeScript against the Zero `Transaction` interface, using [ZQL](#read-data-on-the-client) for reads and a [CRUD-style API](#write-data-on-the-client) for writes.

The server implementation runs on your server, in your push endpoint, against your database. In principle, it can be written in any language and use any data access library. For example you could have the following Go-based server implementation of the same mutator:

```go
func updateIssueOnServer(tx *sql.Tx, id string, title string) error {
  // Validate title length.
  if len(title) > 100 {
    return errors.New("Title is too long")
  }

  _, err := tx.Exec("UPDATE issue SET title = $1 WHERE id = $2", title, id)
  return err
}
```

In practice however, most Zero apps use TypeScript on the server. For these users we provide a handy `ServerTransaction` that implements ZQL against Postgres, so that you can share code between client and server mutators naturally.

So on a TypeScript server, that server mutator can just be:

```ts
async function updateIssueOnServer(
  tx: ServerTransaction,
  {id, title},
  {id: string, title: string},
) {
  // Delegate to client mutator.
  // The `ServerTransaction` here has a different implementation
  // that runs the same ZQL queries against Postgres!
  await updateIssue(tx, {id, title});
}
```

<Note slug="code-sharing" heading="Code sharing in mutators is optional">
Even in TypeScript, you can do as little or as much code sharing as you like. In your server mutator, you can [use raw SQL](#dropping-down-to-raw-sql), any data access libraries you prefer, or add as much extra server-specific logic as you need.

Reusing ZQL on the server is a handy – and we expect frequently used – option, but not a requirement.

</Note>

### Server Authority

You may be wondering what happens if the client and server mutators implementations don't match.

Zero is an example of a _server-authoritative_ sync engine. This means that the server mutator always takes precedence over the client mutator. The result from the client mutator is considered _speculative_ and is discarded as soon as the result from the server mutator is known. This is a very useful feature: it enables server-side validation, permissions, and other server-specific logic.

Imagine that you wanted to use an LLM to detect whether an issue update is spammy, rather than a simple length check. We can just add that to our server mutator:

```ts
async function updateIssueOnServer(
  tx: ServerTransaction,
  {id, title}: {id: string; title: string},
) {
  const response = await llamaSession.prompt(
    `Is this title update likely spam?\n\n${title}\n\nResponse "yes" or "no"`,
  );
  if (/yes/i.test(response)) {
    throw new Error(`Title is likely spam`);
  }

  // delegate rest of implementation to client mutator
  await updateIssue(tx, {id, title});
}
```

If the server detects that the mutation is spammy, the client will see the error message and the mutation will be rolled back. If the server mutator succeeds, the client mutator will be rolled back and the server result will be applied.

### Life of a Mutation

Now that we understand what client and server mutations are, let's walk through how they work together with Zero to sync changes from a source client to the server and then other clients:

1. When you call a custom mutator on the client, Zero runs your client-side mutator immediately on the local device, updating all active queries instantly.
2. In the background, Zero then sends a _mutation_ (a record of the mutator having run with certain arguments) to your server's push endpoint.
3. Your push endpoint runs the [push protocol](#custom-push-implementation), executing the server-side mutator in a transaction against your database and recording the fact that the mutation ran. Optionally, you use our `PushProcessor` class to handle this for you, but you can also implement it yourself.
4. The changes to the database are replicated to `zero-cache` as normal.
5. `zero-cache` calculates the updates to active queries and sends rows that have changed to each client. It also sends information about the mutations that have been applied to the database.
6. Clients receive row updates and apply them to their local cache. Any pending mutations which have been applied to the server have their local effects rolled back.
7. Client-side queries are updated and the user sees the changes.

## Using Custom Mutators

### Registering Client Mutators

By convention, the client mutators are defined with a function called `createMutators` in a file called `mutators.ts`:

```ts
// mutators.ts
import {CustomMutatorDefs} from '@rocicorp/zero';
import {schema} from './schema';

export function createMutators() {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Validate title length. Legacy issues are exempt.
        if (title.length > 100) {
          throw new Error(`Title is too long`);
        }
        await tx.mutate.issue.update({id, title});
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

The `mutators.ts` convention allows mutator implementations to be easily [reused server-side](#setting-up-the-server). The `createMutators` function convention is used so that we can pass authentication information in to [implement permissions](#permissions).

You are free to make different code layout choices – the only real requirement is that you register your map of mutators in the `Zero` constructor:

```ts
// main.tsx
import {Zero} from '@rocicorp/zero';
import {schema} from './schema';
import {createMutators} from './mutators';

const zero = new Zero({
  schema,
  mutators: createMutators(),
});
```

### Write Data on the Client

The `Transaction` interface passed to client mutators exposes the same `mutate` API as the existing [CRUD-style mutators](./writing-data):

```ts
async function myMutator(tx: Transaction) {
  // Insert a new issue
  await tx.mutate.issue.insert({
    id: 'issue-123',
    title: 'New title',
    description: 'New description',
  });

  // Upsert a new issue
  await tx.mutate.issue.upsert({
    id: 'issue-123',
    title: 'New title',
    description: 'New description',
  });

  // Update an issue
  await tx.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  });

  // Delete an issue
  await tx.mutate.issue.delete({
    id: 'issue-123',
  });
}
```

See [the CRUD docs](./writing-data) for detailed semantics on these methods.

### Read Data on the Client

You can read data within a client mutator using [ZQL](./reading-data):

```ts
export function createMutators() {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Read existing issue
        const prev = await tx.query.issue.where('id', id).one();

        // Validate title length. Legacy issues are exempt.
        if (!prev.isLegacy && title.length > 100) {
          throw new Error(`Title is too long`);
        }

        await tx.mutate.issue.update({id, title});
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

You have the full power of ZQL at your disposal, including relationships, filters, ordering, and limits.

Reads and writes within a mutator are transactional, meaning that the datastore is guaranteed to not change while your mutator is running. And if the mutator throws, the entire mutation is rolled back.

<Note type="note" heading="Run in Custom Mutators">
Outside of mutators, the `run()` method has a [`type` parameter](reading-data#running-queries-once) that can be used to wait for server results.

This parameter isn't supported within mutators, because waiting for server results makes no sense in an optimistic mutation – it defeats the purpose of running optimistically to begin with.

When a mutator runs on the client (`tx.location === "client"`), ZQL reads only return data already cached on the client. When mutators run on the server (`tx.location === "server"`), ZQL reads always return all data.

You can use `run()` within custom mutators, but the `type` argument does nothing. In the future, passing `type` in this situation will throw an error.

</Note>

### Invoking Client Mutators

Once you have registered your client mutators, you can call them from your client-side application:

```ts
zero.mutate.issue.update({
  id: 'issue-123',
  title: 'New title',
});
```

The result of a call to a mutator is a `Promise`. You do not usually need to `await` this promise as Zero mutators run very fast, usually completing in a tiny fraction of one frame.

However because mutators ocassionally need to access browser storage, they are technically `async`. Reading a row that was written by a mutator immediately after it is written may not return the new data, because the mutator may not have completed writing to storage yet.

### Waiting for Mutator Result

We typically recommend that you "fire and forget" mutators.

Optimistic mutations make sense when the common case is that a mutation succeeds. If a mutation frequently fails, then showing the user an optimistic result doesn't make sense, because it will likely be wrong.

That said there are cases where it is useful to know when a write succeeded on either the client or server.

One example is if you need to read a row directly after writing it. Zero's local writes are very fast (almost always < 1 frame), but because Zero is backed by IndexedDB, writes are still _technically_ asynchronous and reads directly after a write may not return the new data.

You can use the `.client` promise in this case to wait for a write to complete on the client side:

```ts
try {
  const write = zero.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  });

  // issue-123 not guaranteed to be present here. read1 may be undefined.
  const read1 = await zero.query.issue.where('id', 'issue-123').one();

  // Await client write – almost always less than 1 frame, and same
  // macrotask, so no browser paint will occur here.
  await write.client;

  // issue-123 definitely can be read now.
  const read2 = await zero.query.issue.where('id', 'issue-123').one();
} catch (e) {
  console.error('Mutator failed on client', e);
}
```

You can also wait for the server write to succeed:

```ts
try {
  await zero.mutate.issue.update({
    id: 'issue-123',
    title: 'New title',
  }).server;

  // issue-123 is written to server
} catch (e) {
  console.error('Mutator failed on client or server', e);
}
```

If the client-side mutator fails, the `.server` promise is also rejected with the same error. You don't have to listen to both promises, the server promise covers both cases.

<Note heading="Returning data from mutators" slug="returning-data">
  There is not yet a way to return data from mutators in the success case – the type of `.clent` and `.server` is always `Promise<void>`. [Let us know](https://discord.rocicorp.dev/) if you need this.
</Note>

### Setting Up the Server

You will need a server somewhere you can run an endpoint on. This is typically a serverless function on a platform like Vercel or AWS but can really be anything.

Set the push URL with the [`ZERO_PUSH_URL` env var or `--push-url`](./zero-cache-config#push-url).

If there is per-client configuration you need to send to the push endpoint, you can do that with `push.queryParams`:

```ts
const z = new Zero({
  push: {
    queryParams: {
      workspaceID: '42',
    },
  },
});
```

The push endpoint receives a `PushRequest` as input describing one or more mutations to apply to the backend, and must return a `PushResponse` describing the results of those mutations.

If you are implementing your server in TypeScript, you can use the `PushProcessor` class to trivially implement this endpoint. Here’s an example in a [Hono](https://hono.dev/) app:

```ts
import {Hono} from 'hono';
import {handle} from 'hono/vercel';
import {
  PushProcessor,
  ZQLDatabase,
  PostgresJSConnection,
} from '@rocicorp/zero/pg';
import postgres from 'postgres';
import {schema} from '../shared/schema';
import {createMutators} from '../shared/mutators';

// PushProcessor is provided by Zero to encapsulate a standard
// implementation of the push protocol.
const processor = new PushProcessor(
  new ZQLDatabase(
    new PostgresJSConnection(postgres(process.env.ZERO_UPSTREAM_DB! as string)),
    schema,
  ),
);

export const app = new Hono().basePath('/api');

app.post('/push', async c => {
  const result = await processor.process(createMutators(), c.req.raw);
  return await c.json(result);
});

export default handle(app);
```

`PushProcessor` depends on an abstract `Database`. This allows it to implement the push algorithm against any database.

`@rocicorp/zero/pg` includes a `ZQLDatabase` implementation of this interface backed by Postgres. The implementation allows the same mutator functions to run on client and server, by providing an implementation of the ZQL APIs that custom mutators run on the client.

`ZQLDatabase` in turn relies on an abstract `DBConnection` that provides raw access to a Postgres database. This allows you to use any Postgres library you like, as long as you provide a `DBConnection` implementation for it. The `PostgresJSConnection` class implements `DBConnection` for the excellent [`postgres.js`](https://www.npmjs.com/package/postgres) library to connect to Postgres.

To reuse the client mutators exactly as-is on the server just pass the result of the same `createMutators` function to `PushProcessor`.

### Server Error Handling

The `PushProcessor` in `@rocicorp/zero/pg` skips any mutations that throw:

```ts
app.post('/push', async c => {
  const result = await processor.process({
    issue: {
      update: async (tx, data) => {
        // The mutation is skipped and the next mutation runs as normal.
        throw new Error('bonk');
      },
    },
  }, ...);
  return await c.json(result);
})
```

`PushProcessor` catches such errors and turns them into a structured response that gets sent back to the client. You can [recover the errors](#waiting-for-mutator-result) and show UI if you want.

It is also of course possible for the entire push endpoint to return an HTTP error, or to not reply at all:

```ts
app.post('/push', async c => {
  // This will cause the client to resend all queued mutations.
  throw new Error('zonk');
  const result = await processor.process({
    // ...
  }, ...);
  return await c.json(result);
})
```

If Zero recieves any response from the push endpoint other than HTTP 200, 401, or 403, it will disconnect, wait a few moments, reconnect, and then retry all unprocessed mutations.

If Zero receives HTTP 401 or 403, the client [refreshes the `auth` token](./auth#refresh) if possible, then retries all queued mutations.

If you want a different behavior, it is possible to [implement your own](#custom-push-implementation) `PushProcessor` and handle errors differently.

### Server-Specific Code

To implement server-specific code, just run different mutators in your push endpoint!

An approach we like is to create a separate `server-mutators.ts` file that wraps the client mutators:

```ts
// server-mutators.ts
import {CustomMutatorDefs} from '@rocicorp/zero';
import {schema} from './schema';

export function createMutators(
  clientMutators: CustomMutatorDefs<typeof schema>,
) {
  return {
    // Reuse all client mutators except the ones in `issue`
    ...clientMutators,

    issue: {
      // Reuse all issue mutators except `update`
      ...clientMutators.issue,

      update: async (tx, {id, title}: {id: string; title: string}) => {
        // Call the shared mutator first
        await clientMutators.issue.update(tx, {id, title});

        // Record a history of this operation happening in an audit
        // log table.
        await tx.mutate.auditLog.insert({
          // Assuming you have an audit log table with fields for
          // `issueId`, `action`, and `timestamp`.
          issueId: id,
          action: 'update-title',
          timestamp: new Date().toISOString(),
        });
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

For simple things, we also expose a `location` field on the transaction object that you can use to branch your code:

```ts
myMutator: (tx) => {
  if (tx.location === 'client') {
    // Client-side code
  } else {
    // Server-side code
  }
},
```

### Permissions

Because custom mutators are just arbitrary TypeScript functions, there is no need for a special permissions system. Therefore, you won't use Zero's [write permissions](./permissions) when you use custom mutators.

<Note slug="read-permissions" emoji="💡" heading="You do still need *read* permissions">
  When using custom mutators you will have no [`insert`](permissions#insert-permissions), [`update`](permissions#update-permissions), or [`delete`](permissions#delete-permissions) permissions. You will still have [`select`](permissions#select-permissions) permissions, however.

We hope to build [custom queries](https://bugs.rocicorp.dev/issue/3453) next – a read analog to custom mutators. If we succeed, Zero's permission system will go away completely 🤯.

</Note>

In order to do permission checks, you'll need to know what user is making the request. You can pass this information to your mutators by adding a `AuthData` parameter to the `createMutators` function:

```ts
type AuthData = {
  sub: string;
};

export function createMutators(authData: AuthData | undefined) {
  return {
    issue: {
      launchMissiles: async (tx, args: {target: string}) => {
        if (!authData) {
          throw new Error('Users must be logged in to launch missiles');
        }

        const hasPermission = await tx.query.user
          .where('id', authData.sub)
          .whereExists('permissions', q => q.where('name', 'launch-missiles'))
          .one();
        if (!hasPermission) {
          throw new Error('User does not have permission to launch missiles');
        }
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

The `AuthData` parameter can be any data required for authorization, but is typically just the decoded JWT:

```ts
// app.tsx
const zero = new Zero({
  schema,
  auth: encodedJWT,
  mutators: createMutators(decodedJWT),
});

// hono-server.ts
const processor = new PushProcessor(
  schema,
  connectionProvider(postgres(process.env.ZERO_UPSTREAM_DB as string)),
);
processor.process(
  createMutators(decodedJWT),
  c.req.query(),
  await c.req.json(),
);
```

### Dropping Down to Raw SQL

The `ServerTransaction` interface has a `dbTransaction` property that exposes the underlying database connection. This allows you to run raw SQL queries directly against the database.

This is useful for complex queries, or for using Postgres features that Zero doesn't support yet:

```ts
markAllAsRead: async(tx: Transaction, {userId: string}) {
  // shared stuff ...

  if (tx.location === 'server') {
    // `tx` is now narrowed to `ServerTransaction`.
    // Do special server-only stuff with raw SQL.
    await tx.dbTransaction.query(
      `
      UPDATE notification
      SET read = true
      WHERE user_id = $1
    `,
      [userId],
    );
  }
}
```

As a convenience we also expose a special `CustomMutatorDefs` for use with `server-mutators.ts` that
sets all the mutators to `ServerTransaction` by default:

```ts
// server-mutators.ts
import type {
  CustomMutatorDefs,  // Special server-side CustomMutatorDefs
  PostgresJSTransaction,
  ServerTransaction,
} from '@rocicorp/zero/pg';
import {Schema} from './schema';

export function createMutators(clientMutators: CustomMutatorDefs<Schema>) {
  return {
    // Reuse all client mutators except the ones in `issue`
    ...clientMutators,

    issue: {
      // Reuse all issue mutators except `update`
      ...clientMutators.issue,

      markAllAsRead: async (tx, {userId: string}) {
        // No narrowing necessary – tx is already a `ServerTransaction`
        // assert(tx.location === 'server');
        await tx.dbTransaction.query(
          `
          UPDATE notification
          SET read = true
          WHERE user_id = $1
        `,
          [userId],
        );
      },
    }
  } as const satisfies CustomMutatorDefs<
    ServerTransaction<Schema, PostgresJSTransaction>
  >;
}

```

### Notifications and Async Work

It is bad practice to hold open database transactions while talking over the network, for example to send notifications. Instead, you should let the db transaction commit and do the work asynchronously.

There is no specific support for this in custom mutators, but since mutators are just code, it’s easy to do:

```ts
// server-mutators.ts
export function createMutators(
  authData: AuthData,
  asyncTasks: Array<() => Promise<void>>,
) {
  return {
    issue: {
      update: async (tx, {id, title}: {id: string; title: string}) => {
        await tx.mutate.issue.update({id, title});

        asyncTasks.push(async () => {
          await sendEmailToSubscribers(args.id);
        });
      },
    },
  } as const satisfies CustomMutatorDefs<typeof schema>;
}
```

Then in your push handler:

```ts
app.post('/push', async c => {
  const asyncTasks: Array<() => Promise<void>> = [];
  const result = await processor.process(
    createMutators(authData, asyncTasks),
    c.req.query(),
    await c.req.json(),
  );

  await Promise.all(asyncTasks.map(task => task()));
  return await c.json(result);
});
```

### Custom Database Connections

You can implement an adapter to a different Postgres library, or even a different database entirely.

To do so, provide a `connectionProvider` to `PushProcessor` that returns a different [`DBConnection`](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zql/src/mutate/custom.ts#L67) implementation. For an example implementation, [see the `postgres` implementation](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zero-pg/src/postgres-connection.ts#L4).

### Custom Push Implementation

You can manually implement the push protocol in any programming language.

This will be documented in the future, but you can refer to the [PushProcessor](https://github.com/rocicorp/mono/blob/1a3741fbdad6dbdd56aa1f48cc2cc83938a61b16/packages/zero-pg/src/web.ts#L33) source code for an example for now.

## Examples

- Zbugs uses [custom mutators](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/shared/mutators.ts) for all mutations, [write permissions](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/shared/mutators.ts#L61), and [notifications](https://github.com/rocicorp/mono/blob/a76c9a61670cc09e1a9fe7ab795749f3eef25577/apps/zbugs/server/server-mutators.ts#L35).
- `hello-zero-solid` uses custom mutators for all [mutations](TODO), and for [permissions](TODO).


--- writing-data.mdx ---

Zero generates basic CRUD mutators for every table you sync. Mutators are available at `zero.mutate.<tablename>`:

```tsx
const z = new Zero(...);
z.mutate.user.insert({
	id: nanoid(),
	username: 'abby',
	language: 'en-us',
});
```

<Note slug="moar-power" heading="Need more power?">
  To build mutators with more complex logic or server-specific behavior, see the
  new [Custom Mutators API](./custom-mutators).
</Note>

## Insert

Create new records with `insert`:

```tsx
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: 'js',
});
```

Optional fields can be set to `null` to explicitly set the new field to `null`. They can also be set to `undefined` to take the default value (which is often `null` but can also be some generated value server-side).

```tsx
// schema.ts
import {createTableSchema} from '@rocicorp/zero';

const userSchema = createTableSchema({
  tableName: 'user',
  columns: {
    id: {type: 'string'},
    name: {type: 'string'},
    language: {type: 'string', optional: true},
  },
  primaryKey: ['id'],
  relationships: {},
});

// app.tsx

// Sets language to `null` specifically
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: null,
});

// Sets language to the default server-side value. Could be null, or some
// generated or constant default value too.
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
});

// Same as above
z.mutate.user.insert({
  id: nanoid(),
  username: 'sam',
  language: undefined,
});
```

## Upsert

Create new records or update existing ones with `upsert`:

```tsx
z.mutate.user.upsert({
  id: samID,
  username: 'sam',
  language: 'ts',
});
```

`upsert` supports the same `null` / `undefined` semantics for optional fields that `insert` does (see above).

## Update

Update an existing record. Does nothing if the specified record (by PK) does not exist.

You can pass a partial, leaving fields out that you don’t want to change. For example here we leave the username the same:

```tsx
// Leaves username field to previous value.
z.mutate.user.update({
  id: samID,
  language: 'golang',
});

// Same as above
z.mutate.user.update({
  id: samID,
  username: undefined,
  language: 'haskell',
});

// Reset language field to `null`
z.mutate.user.update({
  id: samID,
  language: null,
});
```

## Delete

Delete an existing record. Does nothing if specified record does not exist.

```tsx
z.mutate.user.delete({
  id: samID,
});
```

<Note type="note" heading="Cache Persistence" slug="cache-persistence">
If you want to remove cached data from the browser you can call the `dropAllDatabases` function provided by Zero.

You can read more about it in [Authentication](/docs/auth#data-storage).

</Note>

## Batch Mutate

You can do multiple CRUD mutates in a single _batch_. If any of the mutations fails, all will. They also all appear together atomically in a single transaction to other clients.

```tsx
z.mutateBatch(async tx => {
  const samID = nanoid();
  tx.user.insert({
    id: samID,
    username: 'sam',
  });

  const langID = nanoid();
  tx.language.insert({
    id: langID,
    userID: samID,
    name: 'js',
  });
});
```


--- reading-data.mdx ---

ZQL is Zero’s query language.

Inspired by SQL, ZQL is expressed in TypeScript with heavy use of the builder pattern. If you have used [Drizzle](https://orm.drizzle.team/) or [Kysely](https://kysely.dev/), ZQL will feel familiar.

ZQL queries are composed of one or more _clauses_ that are chained together into a _query_.

Unlike queries in classic databases, the result of a ZQL query is a _view_ that updates automatically and efficiently as the underlying data changes. You can call a query’s `materialize()` method to get a view, but more typically you run queries via some framework-specific bindings. For example see `useQuery` for [React](react) or [SolidJS](solidjs).

<Note type="note" emoji="🧑‍🏫" slug="immutability" heading="Data returned from ZQL should be considered immutable">
This means you should not modify the data directly. Instead, clone the data and modify the clone.

ZQL caches values and returns them multiple times. If you modify a value returned from ZQL, you will modify it everywhere it is used. This can lead to subtle bugs.

JavaScript and TypeScript lack true immutable types so we use `readonly` to help enforce it. But it's easy to cast away the `readonly` accidentally.

In the future, we'll [`freeze`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze) all returned data in `dev` mode to help prevent this.

</Note>

## Select

ZQL queries start by selecting a table. There is no way to select a subset of columns; ZQL queries always return the entire row (modulo column permissions).

```tsx
const z = new Zero(...);

// Returns a query that selects all rows and columns from the issue table.
z.query.issue;
```

This is a design tradeoff that allows Zero to better reuse the row locally for future queries. This also makes it easier to share types between different parts of the code.

## Ordering

You can sort query results by adding an `orderBy` clause:

```tsx
z.query.issue.orderBy('created', 'desc');
```

Multiple `orderBy` clauses can be present, in which case the data is sorted by those clauses in order:

```tsx
// Order by priority descending. For any rows with same priority,
// then order by created desc.
z.query.issue.orderBy('priority', 'desc').orderBy('created', 'desc');
```

All queries in ZQL have a default final order of their primary key. Assuming the `issue` table has a primary key on the `id` column, then:

```tsx
// Actually means: z.query.issue.orderBy('id', 'asc');
z.query.issue;

// Actually means: z.query.issue.orderBy('priority', 'desc').orderBy('id', 'asc');
z.query.issue.orderBy('priority', 'desc');
```

## Limit

You can limit the number of rows to return with `limit()`:

```tsx
z.query.issue.orderBy('created', 'desc').limit(100);
```

## Paging

You can start the results at or after a particular row with `start()`:

```tsx
let start: IssueRow | undefined;
while (true) {
  let q = z.query.issue.orderBy('created', 'desc').limit(100);
  if (start) {
    q = q.start(start);
  }
  const batch = await q.run();
  console.log('got batch', batch);

  if (batch.length < 100) {
    break;
  }
  start = batch[batch.length - 1];
}
```

By default `start()` is _exclusive_ - it returns rows starting **after** the supplied reference row. This is what you usually want for paging. If you want _inclusive_ results, you can do:

```tsx
z.query.issue.start(row, {inclusive: true});
```

## Getting a Single Result

If you want exactly zero or one results, use the `one()` clause. This causes ZQL to return `Row|undefined` rather than `Row[]`.

```tsx
const result = await z.query.issue.where('id', 42).one().run();
if (!result) {
  console.error('not found');
}
```

`one()` overrides any `limit()` clause that is also present.

## Relationships

You can query related rows using _relationships_ that are defined in your [Zero schema](/docs/zero-schema).

```tsx
// Get all issues and their related comments
z.query.issue.related('comments');
```

Relationships are returned as hierarchical data. In the above example, each row will have a `comments` field which is itself an array of the corresponding comments row.

You can fetch multiple relationships in a single query:

```tsx
z.query.issue.related('comments').related('reactions').related('assignees');
```

### Refining Relationships

By default all matching relationship rows are returned, but this can be refined. The `related` method accepts an optional second function which is itself a query.

```tsx
z.query.issue.related(
  'comments',
  // It is common to use the 'q' shorthand variable for this parameter,
  // but it is a _comment_ query in particular here, exactly as if you
  // had done z.query.comment.
  q => q.orderBy('modified', 'desc').limit(100).start(lastSeenComment),
);
```

This _relationship query_ can have all the same clauses that top-level queries can have.

<Note type="warning" heading="Order and limit not supported in junction relationships">
Using `orderBy` or `limit` in a relationship that goes through a junction table (i.e., a many-to-many relationship) is not currently supported and will throw a runtime error. See [bug 3527](https://bugs.rocicorp.dev/issue/3527).

You can sometimes work around this by making the junction relationship explicit, depending on your schema and usage.
</Note>

### Nested Relationships

You can nest relationships arbitrarily:

```tsx
// Get all issues, first 100 comments for each (ordered by modified,desc),
// and for each comment all of its reactions.
z.query.issue.related(
	'comments', q => q.orderBy('modified', 'desc').limit(100).related(
		'reactions')
	)
);
```

## Where

You can filter a query with `where()`:

```tsx
z.query.issue.where('priority', '=', 'high');
```

The first parameter is always a column name from the table being queried. Intellisense will offer available options (sourced from your [Zero Schema](/docs/zero-schema)).

### Comparison Operators

Where supports the following comparison operators:

| Operator                                 | Allowed Operand Types         | Description                                                              |
| ---------------------------------------- | ----------------------------- | ------------------------------------------------------------------------ |
| `=` , `!=`                               | boolean, number, string       | JS strict equal (===) semantics                                          |
| `<` , `<=`, `>`, `>=`                    | number                        | JS number compare semantics                                              |
| `LIKE`, `NOT LIKE`, `ILIKE`, `NOT ILIKE` | string                        | SQL-compatible `LIKE` / `ILIKE`                                          |
| `IN` , `NOT IN`                          | boolean, number, string       | RHS must be array. Returns true if rhs contains lhs by JS strict equals. |
| `IS` , `IS NOT`                          | boolean, number, string, null | Same as `=` but also works for `null`                                    |

TypeScript will restrict you from using operators with types that don’t make sense – you can’t use `>` with `boolean` for example.

<Note type="note" slug="adding-operators">
  If you don’t see the comparison operator you need, let us know, many are easy
  to add.
</Note>

### Equals is the Default Comparison Operator

Because comparing by `=` is so common, you can leave it out and `where` defaults to `=`.

```tsx
z.query.issue.where('priority', 'high');
```

### Comparing to `null`

As in SQL, ZQL’s `null` is not equal to itself (`null ≠ null`).

This is required to make join semantics work: if you’re joining `employee.orgID` on `org.id` you do **not** want an employee in no organization to match an org that hasn’t yet been assigned an ID.

When you purposely want to compare to `null` ZQL supports `IS` and `IS NOT` operators that work just like in SQL:

```tsx
// Find employees not in any org.
z.query.employee.where('orgID', 'IS', null);
```

TypeScript will prevent you from comparing to `null` with other operators.

### Compound Filters

The argument to `where` can also be a callback that returns a complex expression:

```tsx
// Get all issues that have priority 'critical' or else have both
// priority 'medium' and not more than 100 votes.
z.query.issue.where(({cmp, and, or, not}) =>
  or(
    cmp('priority', 'critical'),
    and(cmp('priority', 'medium'), not(cmp('numVotes', '>', 100))),
  ),
);
```

`cmp` is short for _compare_ and works the same as `where` at the top-level except that it can’t be chained and it only accepts comparison operators (no relationship filters – see below).

Note that chaining `where()` is also a one-level `and`:

```tsx
// Find issues with priority 3 or higher, owned by aa
z.query.issue.where('priority', '>=', 3).where('owner', 'aa');
```

### Relationship Filters

Your filter can also test properties of relationships. Currently the only supported test is existence:

```tsx
// Find all orgs that have at least one employee
z.query.organization.whereExists('employees');
```

The argument to `whereExists` is a relationship, so just like other relationships it can be refined with a query:

```tsx
// Find all orgs that have at least one cool employee
z.query.organization.whereExists('employees', q =>
  q.where('location', 'Hawaii'),
);
```

As with querying relationships, relationship filters can be arbitrarily nested:

```tsx
// Get all issues that have comments that have reactions
z.query.issue.whereExists('comments',
	q => q.whereExists('reactions'));
);
```

The `exists` helper is also provided which can be used with `and`, `or`, `cmp`, and `not` to build compound filters that check relationship existence:

```tsx
// Find issues that have at least one comment or are high priority
z.query.issue.where({cmp, or, exists} =>
  or(
    cmp('priority', 'high'),
    exists('comments'),
  ),
);
```

## Data Lifetime and Reuse

Zero reuses data synced from prior queries to answer new queries when possible. This is what enables instant UI transitions.

But what controls the lifetime of this client-side data? How can you know whether any partiular query will return instant results? How can you know whether those results will be up to date or stale?

The answer is that the data on the client is simply the union of rows returned from queries which are currently syncing. Once a row is no longer returned by any syncing query, it is removed from the client. Thus, there is never any stale data in Zero.

So when you are thinking about whether a query is going to return results instantly, you should think about _what other queries are syncing_, not about what data is local. Data exists locally if and only if there is a query syncing that returns that data.

<Note type="note" heading="Caches vs Replicas">
  This is why we often say that despite the name `zero-cache`, Zero is not technically a cache. It's a *replica*.

A cache has a random set of rows with a random set of versions. There is no expectation that the cache any particular rows, or that the rows' have matching versions. Rows are simply updated as they are fetched.

A replica by contrast is eagerly updated, whether or not any client has requested a row. A replica is always very close to up-to-date, and always self-consistent.

Zero is a _partial_ replica because it only replicates rows that are returned by syncing queries.

</Note>

## Query Lifecycle

<ImageLightbox
  src="/images/reading-data/query-lifecycle.png"
  caption="The lifecycle of a ZQL query."
/>

Queries can be either _active_ or _backgrounded_. An active query is one that is currently being used by the application. Backgrounded queries are not currently in use, but continue syncing in case they are needed again soon.

Active queries are created one of three ways:

1. The app calls `q.materialize()` to get a `View`.
2. The app uses a platform binding like React's `useQuery(q)`.
3. The app calls [`preload()`](#preloading) to sync larger queries without a view.

Active queries sync until they are _deactivated_. The way this happens depends on how the query was created:

1. For `materialize()` queries, the UI calls `destroy()` on the view.
2. For `useQuery()`, the UI unmounts the component (which calls `destroy()` under the covers).
3. For `preload()`, the UI calls `cleanup()` on the return value of `preload()`.

### Background Queries

By default a deactivated query stops syncing immediately.

But it's often useful to keep queries syncing beyond deactivation in case the UI needs the same or a similar query in the near future. This is accomplished with the `ttl` parameter:

```ts
const [user] = useQuery(z.query.user.where('id', userId), {ttl: '1d'});
```

The `ttl` parameter specifies how long the app developer wishes the query to run in the background. The following formats are allowed (where `%d` is a positive integer):

| Format    | Meaning                                                                              |
| --------- | ------------------------------------------------------------------------------------ |
| `none`    | No backgrounding. Query will immediately stop when deactivated. This is the default. |
| `%ds`     | Number of seconds.                                                                   |
| `%dm`     | Number of minutes.                                                                   |
| `%dh`     | Number of hours.                                                                     |
| `%dd`     | Number of days.                                                                      |
| `%dy`     | Number of years.                                                                     |
| `forever` | Query will never be stopped.                                                         |

If the UI re-requests a background query, it becomes an active query again. Since the query was syncing in the background, the very first synchronous result that the UI receives after reactivation will be up-to-date with the server (i.e., it will have `resultType` of `complete`).

Just like other types of queries, the data from background queries is available for use by new queries. A common pattern in to [preload](#preloading) a subset of most commonly needed data with `{ttl: 'forever'}` and then do more specific queries from the UI with, e.g., `{ttl: '1d'}`. Most often the preloaded data will be able to answer user queries, but if not, the new query will be answered by the server and backgrounded for a day in case the user revisits it.

### Client Capacity Management

Zero has a default soft limit of 20,000 rows on the client-side, or about 20MB of data assuming 1KB rows.

This limit can be increased with the [`--target-client-row-count`](./zero-cache-config#target-client-row-count) flag, but we do not recommend setting it higher than 100,000.

<Note slug="client-capacity" heading="Why does Zero store so little data client-side?">
Contrary to the design of other sync engines, we believe that storing tons of data client-side doesn't make sense. Here are some reasons why:

- Initial sync will be slow, slowing down initial app load.
- Because storage in browser tabs is unreliable, initial sync can occur surprisingly often.
- We want to answer queries _instantly_ as often as possible. This requires client-side data in memory on the main thread. If we have to page to disk, we may as well go to the network and reduce complexity.
- Even though Zero's queries are very efficient, they do still have some cost, expecially hydration. Massive client-side storage would result in hydrating tons of queries that are unlikely to be used every time the app starts.

Most importantly, no matter how much data you store on the client, there will be cases where you have to fallback to the server:

- Some users might have huge amounts of data.
- Some users might have tiny amounts of available client storage.
- You will likely want the app to start fast and sync in the background.

Because you have to be able to fallback to server the question becomes _what is the **right** amount of data to store on the client?_, not _how can I store the absolute max possible data on the client?_

The goal with Zero is to answer 99% of queries on the client from memory. The remaining 1% of queries can fallback gracefully to the server. 20,000 rows was chosen somewhat arbitrarily as a number of rows that was likely to be able to do this for many applications.

There is no hard limit at 20,000 or 100,000. Nothing terrible happens if you go above. The thing to keep in mind is that:

1. All those queries will revalidate everytime your app boots.
2. All data synced to the client is in memory in JS.

</Note>

Here is how this limit is managed:

1. Active queries are never destroyed, even if the limit is exceeded. Developers are expected to keep active queries well under the limit.
2. The `ttl` value counts from the moment a query deactivates. Backgrounded queries are destroyed immediately when the `ttl` is reached, even if the limit hasn't been reached.
3. If the client exceeds its limit, Zero will destroy backgrounded queries, least-recently-used first, until the store is under the limit again.

### Thinking in Queries

Although IVM is a very efficient way to keep queries up to date relative to re-running them, it isn't free. You still need to think about how many queries you are creating, how long they are kept alive, and how expensive they are.

This is why Zero defaults to _not_ backgrounding queries and doesn't try to aggressively fill its client datastore to capacity. You should put some thought into what queries you want to run in the background, and for how long.

Zero currently provides a few basic tools to understand the cost of your queries:

- The client logs a warning for slow query materializations. Look for `Slow query materialization` in your logs. The default threshold is `5s` (including network) but this is configurable with the `slowMaterializeThreshold` parameter.
- The client logs the materialization time of all queries at the `debug` level. Look for `Materialized query` in your logs.
- The server logs a warning for slow query materializations. Look for `Slow query materialization` in your logs. The default threshold is `5s` but this is configurable with the `log-slow-materialize-threshold` configuration parameter.

We will be adding more tools over time.

## Completeness

Zero returns whatever data it has on the client immediately for a query, then falls back to the server for any missing data. Sometimes it's useful to know the difference between these two types of results. To do so, use the `result` from `useQuery`:

```tsx
const [issues, issuesResult] = useQuery(z.query.issue);
if (issuesResult.type === 'complete') {
  console.log('All data is present');
} else {
  console.log('Some data is missing');
}
```

The possible values of `result.type` are currently `complete` and `unknown`.

The `complete` value is currently only returned when Zero has received the server result. But in the future, Zero will be able to return this result type when it _knows_ that all possible data for this query is already available locally. Additionally, we plan to add a `prefix` result for when the data is known to be a prefix of the complete result. See [Consistency](#consistency) for more information.

## Handling Missing Data

It is inevetable that there will be cases where the requested data cannot be found. Because Zero returns local results immediately, and server results asynchronously, displaying "not found" / 404 UI can be slightly tricky. If you just use a simple existence check, you will often see the 404 UI flicker while the server result loads:

```tsx
const [issue, issuesResult] = useQuery(
  z.query.issue.where('id', 'some-id').one(),
);

// ❌ This causes flickering of the UI
if (!issue) {
  return <div>404 Not Found</div>;
} else {
  return <div>{issue}</div>;
}
```

The way to do this correctly is to only display the "not found" UI when the result type is `complete`. This way the 404 page is slow but pages with data are still just as fast.

```tsx
const [issue, issuesResult] = useQuery(
  z.query.issue.where('id', 'some-id').one(),
);

if (!issue && issueResult.type === 'complete') {
  return <div>404 Not Found</div>;
}

if (!issue) {
  return null;
}

return <div>{issue}</div>;
```

## Listening to Changes

Currently, the way to listen for changes in query results is not ideal. You can add a listener to a materialized view which has the new data and result as parameters:

```ts
z.query.issue.materialize().addListener((issues, issuesResult) => {
  // do stuff...
});
```

However, using this method will maintain its own materialized view in memory which is wasteful. It also doesn't allow for granular listening to events like `add` and `remove` of rows.

A better way would be to create your own view without actually storing the data which will also allow you to listen to specific events. Again, the API is not good and will be improved in the future.

```ts
// Inside the View class
// Instead of storing the change, we invoke some callback
push(change: Change): void {
  switch (change.type) {
    case 'add':
      this.#onAdd?.(change)
      break
    case 'remove':
      this.#onRemove?.(change)
      break
    case 'edit':
      this.#onEdit?.(change)
      break
    case 'child':
      this.#onChild?.(change)
      break
    default:
      throw new Error(`Unknown change type: ${change['type']}`)
  }
}
```

(see View implementations in [`zero-vue`](https://github.com/danielroe/zero-vue/blob/f25808d4b7d1ef0b8e01a5670d7e3050d6a64bbf/src/view.ts#L77-L89) or [`zero-solid`](https://github.com/rocicorp/mono/blob/51995101d0657519207f1c4695a8765b9016e07c/packages/zero-solid/src/solid-view.ts#L119-L131))

## Preloading

Almost all Zero apps will want to preload some data in order to maximize the feel of instantaneous UI transitions.

In Zero, preloading is done via queries – the same queries you use in the UI and for auth.

However, because preload queries are usually much larger than a screenful of UI, Zero provides a special `preload()` helper to avoid the overhead of materializing the result into JS objects:

```tsx
// Preload the first 1k issues + their creator, assignee, labels, and
// the view state for the active user.
//
// There's no need to render this data, so we don't use `useQuery()`:
// this avoids the overhead of pulling all this data into JS objects.
z.query.issue
  .related('creator')
  .related('assignee')
  .related('labels')
  .related('viewState', q => q.where('userID', z.userID).one())
  .orderBy('created', 'desc')
  .limit(1000)
  .preload();
```

## Running Queries Once

Usually subscribing to a query is what you want in a reactive UI, but every so often you'll need to run a query just once. To do this, use the `run()` method:

```tsx
const results = await z.query.issue.where('foo', 'bar').run();
```

By default, `run()` only returns results that are currently available on the client. That is, it returns the data that would be given for [`result.type === 'unknown'`](#completeness).

If you want to wait for the server to return results, pass `{type: 'complete'}` to `run`:

```tsx
const results = await z.query.issue.where('foo', 'bar').run({type: 'complete'});
```

<Note slug="default-run">
As a convenience you can also directly await queries:

```ts
await z.query.issue.where('foo', 'bar');
```

This is the same as saying `run()` or `run({type: 'unknown'})`.

</Note>

## Consistency

Zero always syncs a consistent partial replica of the backend database to the client. This avoids many common consistency issues that come up in classic web applications. But there are still some consistency issues to be aware of when using Zero.

For example, imagine that you have a bug database w/ 10k issues. You preload the first 1k issues sorted by created.

The user then does a query of issues assigned to themselves, sorted by created. Among the 1k issues that were preloaded imagine 100 are found that match the query. Since the data we preloaded is in the same order as this query, we are guaranteed that any local results found will be a _prefix_ of the server results.

The UX that result is nice: the user will see initial results to the query instantly. If more results are found server-side, those results are guaranteed to sort below the local results. There's no shuffling of results when the server response comes in.

Now imagine that the user switches the sort to ‘sort by modified’. This new query will run locally, and will again find some local matches. But it is now unlikely that the local results found are a prefix of the server results. When the server result comes in, the user will probably see the results shuffle around.

To avoid this annoying effect, what you should do in this example is also preload the first 1k issues sorted by modified desc. In general for any query shape you intend to do, you should preload the first `n` results for that query shape with no filters, in each sort you intend to use.

<Note slug="no-duplicate-rows" heading="Zero does not sync duplicate rows">
Zero syncs the *union* of all active queries' results. You don't have to worry about syncing many sorts of the same query when it's likely the results will overlap heavily.

</Note>

In the future, we will be implementing a consistency model that fixes these issues automatically. We will prevent Zero from returning local data when that data is not known to be a prefix of the server result. Once the consistency model is implemented, preloading can be thought of as purely a performance thing, and not required to avoid unsightly flickering.


--- permissions.mdx ---

Permissions are expressed using [ZQL](reading-data) and run automatically with every read and write.

## Define Permissions

Permissions are defined in [`schema.ts`](/docs/zero-schema) using the `definePermissions` function.

Here's an example of limiting reads to members of an organization and deletes to only the creator of an issue:

```ts
// The decoded value of your JWT.
type AuthData = {
  // The logged-in user.
  sub: string;
};

export const permissions = definePermissions<AuthData, Schema>(schema, () => {
  // Checks if the user exists in a related organization
  const allowIfInOrganization = (
    authData: AuthData,
    eb: ExpressionBuilder<Schema, 'issue'>,
  ) =>
    eb.exists('organization', q =>
      q.whereExists('user', q => q.where('id', authData.sub)),
    );

  // Checks if the user is the creator
  const allowIfIssueCreator = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('creatorID', authData.sub);

  return {
    issue: {
      row: {
        select: [allowIfInOrganization],
        delete: [allowIfIssueCreator],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

`definePermission` returns a _policy_ object for each table in the schema. Each policy defines a _ruleset_ for the _operations_ that are possible on a table: `select`, `insert`, `update`, and `delete`.

## Access is Denied by Default

If you don't specify any rules for an operation, it is denied by default. This is an important safety feature that helps ensure data isn't accidentally exposed.

To enable full access to an action (i.e., during development) use the `ANYONE_CAN` helper:

```ts
import {ANYONE_CAN} from '@rocicorp/zero';

const permissions = definePermissions<AuthData, Schema>(schema, () => {
  return {
    issue: {
      row: {
        select: ANYONE_CAN,
        // Other operations are denied by default.
      },
    },
    // Other tables are denied by default.
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

To do this for all actions, use `ANYONE_CAN_DO_ANYTHING`:

```ts
import {ANYONE_CAN_DO_ANYTHING} from '@rocicorp/zero';

const permissions = definePermissions<AuthData, Schema>(schema, () => {
  return {
    // All operations on issue are allowed to all users.
    issue: ANYONE_CAN_DO_ANYTHING,

    // Other tables are denied by default.
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Permission Evaluation

Zero permissions are "compiled" into a JSON-based format at build-time. This file is stored in the `{ZERO_APP_ID}.permissions` table of your upstream database. Like other tables, it replicates live down to `zero-cache`. `zero-cache` then parses this file, and applies the encoded rules to every read and write operation.

<Note type="warning" slug="rule-compilation-is-dumb">
The compilation process is very simple-minded (read: dumb). Despite looking like normal TypeScript functions that receive an `AuthData` parameter, rule functions are not actually invoked at runtime. Instead, they are invoked with a "placeholder" `AuthData` at build time. We track which fields of this placeholder are accessed and construct a ZQL expression that accesses the right field of `AuthData` at runtime.

The end result is that you can't really use most features of JS in these rules. Specifically you cannot:

- Iterate over properties or array elements in the auth token
- Use any JS features beyond property access of `AuthData`
- Use any conditional or global state

Basically only property access is allowed. This is really confusing and we're working on a better solution.

</Note>

## Permission Deployment

During development, permissions are compiled and uploaded to your database completely automatically as part of the `zero-cache-dev` script.

For production, you need to call `npx zero-deploy-permissions` within your app to update the permissions in the production database whenever they change. You would typically do this as part of your normal schema migration or CI process. For example, the SST deployment script for [zbugs](/docs/samples#zbugs) looks like this:

```ts
new command.local.Command(
  'zero-deploy-permissions',
  {
    create: `npx zero-deploy-permissions -p ../../src/schema.ts`,
    // Run the Command on every deploy ...
    triggers: [Date.now()],
    environment: {
      ZERO_UPSTREAM_DB: commonEnv.ZERO_UPSTREAM_DB,
      // If the application has a non-default App ID ...
      ZERO_APP_ID: commonEnv.ZERO_APP_ID,
    },
  },
  // after the view-syncer is deployed.
  {dependsOn: viewSyncer},
);
```

See the [SST Deployment Guide](deployment#guide-multi-node-on-sstaws) for more details.

## Rules

Each operation on a policy has a _ruleset_ containing zero or more _rules_.

A rule is just a TypeScript function that receives the logged in user's `AuthData` and generates a ZQL [where expression](reading-data#compound-filters). At least one rule in a ruleset must return a row for the operation to be allowed.

## Select Permissions

You can limit the data a user can read by specifying a `select` ruleset.

Select permissions act like filters. If a user does not have permission to read a row, it will be filtered out of the result set. It will not generate an error.

For example, imagine a select permission that restricts reads to only issues created by the user:

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueCreator = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('creatorID', authData.sub);

  return {
    issue: {
      row: {
        select: [allowIfIssueCreator],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

If the issue table has two rows, one created by the user and one by someone else, the user will only see the row they created in any queries.

## Insert Permissions

You can limit what rows can be inserted and by whom by specifying an `insert` ruleset.

Insert rules are evaluated after the entity is inserted. So if they query the database, they will see the inserted row present. If any rule in the insert ruleset returns a row, the insert is allowed.

Here's an example of an insert rule that disallows inserting users that have the role 'admin'.

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfNonAdmin = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'user'>,
  ) => cmp('role', '!=', 'admin');

  return {
    user: {
      row: {
        insert: [allowIfNonAdmin],
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Update Permissions

There are two types of update rulesets: `preMutation` and `postMutation`. Both rulesets must pass for an update to be allowed.

`preMutation` rules see the version of a row _before_ the mutation is applied. This is useful for things like checking whether a user owns an entity before editing it.

`postMutation` rules see the version of a row _after_ the mutation is applied. This is useful for things like ensuring a user can only mark themselves as the creator of an entity and not other users.

Like other rulesets, `preMutation` and `postMutation` default to `NOBODY_CAN`. This means that every table must define both these rulesets in order for any updates to be allowed.

For example, the following ruleset allows an issue's owner to edit, but **not** re-assign the issue. The `postMutation` rule enforces that the current user still own the issue after edit.

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: [allowIfIssueOwner],
          postMutation: [allowIfIssueOwner],
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

This ruleset allows an issue's owner to edit and re-assign the issue:

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: [allowIfIssueOwner],
          postMutation: ANYONE_CAN,
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

And this allows anyone to edit an issue, but only if they also assign it to themselves. Useful for enforcing _"patches welcome"_? 🙃

```ts
definePermissions<AuthData, Schema>(schema, () => {
  const allowIfIssueOwner = (
    authData: AuthData,
    {cmp}: ExpressionBuilder<Schema, 'issue'>,
  ) => cmp('ownerID', authData.sub);

  return {
    issue: {
      row: {
        update: {
          preMutation: ANYONE_CAN,
          postMutation: [allowIfIssueOwner],
        },
      },
    },
  } satisfies PermissionsConfig<AuthData, Schema>;
});
```

## Delete Permissions

Delete permissions work in the same way as `insert` permissions except they run _before_ the delete is applied. So if a delete rule queries the database, it will see that the deleted row is present. If any rule in the ruleset returns a row, the delete is allowed.

## Debugging

See [Debugging Permissions](./debug/permissions).

## Examples

See [hello-zero](https://github.com/rocicorp/hello-zero/blob/main/src/schema.ts) for a simple example of write auth and [zbugs](https://github.com/rocicorp/mono/blob/main/apps/zbugs/shared/schema.ts#L217) for a much more involved one.


--- debug/permissions.mdx ---

Given that permissions are defined in their own file and internally applied to queries, it might be hard to figure out if or why a permission check is failing.

## Read Permissions

You can use the `analyze-query` utility with the `--apply-permissions` flag to see the complete query Zero runs, including read permissions.

```bash
npx analyze-query
  --schema-path='./shared/schema.ts'
  --query='issue.related("comments")'
  --apply-permissions
  --auth-data='{"userId":"user-123"}'
```

If the result looks right, the problem may be that Zero is not receiving the `AuthData` that you think it is. You can retrieve a query hash from websocket or server logs, then ask Zero for the details on that specific query.

Run this command with the same environment you run `zero-cache` with. It will use your `upstream` or `cvr` configuration to look up the query hash in the cvr database.

```bash
npx analyze-query
  --schema-path='./shared/schema.ts'
  --hash='3rhuw19xt9vry'
  --apply-permissions
  --auth-data='{"userId":"user-123"}'
```

<Note type="note" slug="decompilation">
  The printed query can be different than the source ZQL string, because it is rebuilt from the query AST. But it should be logically equivalent to the query you wrote.
</Note>

## Write Permissions

Look for a `WARN` level log in the output from `zero-cache` like this:

```
Permission check failed for {"op":"update","tableName":"message",...}, action update, phase preMutation, authData: {...}, rowPolicies: [...], cellPolicies: []
```

Zero prints the row, auth data, and permission policies that was applied to any failed writes.

<Note slug="ast-format">
  The ZQL query is printed in AST format. See [Query ASTs](./query-asts) to
  convert it to a more readable format.
</Note>


--- zero-cache-config.mdx ---

`zero-cache` is configured either via CLI flag or environment variable. There is no separate `zero.config` file.

You can also see all available flags by running `zero-cache --help`.

## Required Flags

### Auth

One of [Auth JWK](#auth-jwk), [Auth JWK URL](#auth-jwk-url), or [Auth Secret](#auth-secret) must be specified. See [Authentication](/docs/auth/) for more details.

### Replica File

File path to the SQLite replica that zero-cache maintains. This can be lost, but if it is, zero-cache will have to re-replicate next time it starts up.

flag: `--replica-file`<br/>
env: `ZERO_REPLICA_FILE`<br/>
required: `true`

### Upstream DB

The "upstream" authoritative postgres database. In the future we will support other types of upstream besides PG.

flag: `--upstream-db`<br/>
env: `ZERO_UPSTREAM_DB`<br/>
required: `true`

## Optional Flags

### Admin Password

A password used to administer zero-cache server, for example to access the `/statz` endpoint.

flag: `--admin-password`<br/>
env: `ZERO_ADMIN_PASSWORD`<br/>
required: `false`

### App ID

Unique identifier for the app.

Multiple zero-cache apps can run on a single upstream database, each of which is isolated from the others, with its own permissions, sharding (future feature), and change/cvr databases.

The metadata of an app is stored in an upstream schema with the same name, e.g. `zero`, and the metadata for each app shard, e.g. client and mutation ids, is stored in the `{app-id}_{#}` schema. (Currently there is only a single "0" shard, but this will change with sharding).

The CVR and Change data are managed in schemas named `{app-id}_{shard-num}/cvr` and `{app-id}_{shard-num}/cdc`, respectively, allowing multiple apps and shards to share the same database instance (e.g. a Postgres "cluster") for CVR and Change management.

Due to constraints on replication slot names, an App ID may only consist of lower-case letters, numbers, and the underscore character.

Note that this option is used by both `zero-cache` and `zero-deploy-permissions`.

flag: `--app-id`<br/>
env: `ZERO_APP_ID`<br/>
default: `zero`

### App Publications

Postgres PUBLICATIONs that define the tables and columns to replicate. Publication names may not begin with an underscore, as zero reserves that prefix for internal use.

If unspecified, zero-cache will create and use an internal publication that publishes all tables in the public schema, i.e.:

```
CREATE PUBLICATION _{app-id}_public_0 FOR TABLES IN SCHEMA public;
```

Note that once an app has begun syncing data, this list of publications cannot be changed, and zero-cache will refuse to start if a specified value differs from what was originally synced. To use a different set of publications, a new app should be created.

flag: `--app-publications`<br/>
env: `ZERO_APP_PUBLICATIONS`<br/>
default: `[]`

### Auth JWK

A public key in JWK format used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-jwk`<br/>
env: `ZERO_AUTH_JWK`<br/>
required: `false`

### Auth JWK URL

A URL that returns a JWK set used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-jwks-url`<br/>
env: `ZERO_AUTH_JWKS_URL`<br/>
required: `false`

### Auto Reset

Automatically wipe and resync the replica when replication is halted. This situation can occur for configurations in which the upstream database provider prohibits event trigger creation, preventing the zero-cache from being able to correctly replicate schema changes. For such configurations, an upstream schema change will instead result in halting replication with an error indicating that the replica needs to be reset. When auto-reset is enabled, zero-cache will respond to such situations by shutting down, and when restarted, resetting the replica and all synced clients. This is a heavy-weight operation and can result in user-visible slowness or downtime if compute resources are scarce.

flag: `--auto-reset`<br/>
env: `ZERO_AUTO_RESET`<br/>
default: `true`

### Auth Secret

A symmetric key used to verify JWTs. Only one of jwk, jwksUrl and secret may be set.

flag: `--auth-secret`<br/>
env: `ZERO_AUTH_SECRET`<br/>
required: `false`

### Change DB

The Postgres database used to store recent replication log entries, in order to sync multiple view-syncers without requiring multiple replication slots on the upstream database. If unspecified, the upstream-db will be used.

flag: `--change-db`<br/>
env: `ZERO_CHANGE_DB`<br/>
required: `false`

### Change Max Connections

The maximum number of connections to open to the change database. This is used by the change-streamer for catching up zero-cache replication subscriptions.

flag: `--change-max-conns`<br/>
env: `ZERO_CHANGE_MAX_CONNS`<br/>
default: `5`

### Change Streamer Mode
The mode for running or connecting to the change-streamer:

* `dedicated`: runs the change-streamer and shuts down when another
  change-streamer takes over the replication slot. This is appropriate in a
  single-node configuration, or for the replication-manager in a
  multi-node configuration.
* `discover`: connects to the change-streamer as internally advertised in the
  change-db. This is appropriate for the view-syncers in a multi-node

flag: `--change-streamer-mode`<br/>
env: `ZERO_CHANGE_STREAMER_MODE`<br/>
default: `dedicated`

### Change Streamer Port

The port on which the change-streamer runs. This is an internal protocol between the replication-manager and zero-cache, which runs in the same process in local development. If unspecified, defaults to --port + 1.

flag: `--change-streamer-port`<br/>
env: `ZERO_CHANGE_STREAMER_PORT`<br/>
required: `false`

### CVR DB

The Postgres database used to store CVRs. CVRs (client view records) keep track of the data synced to clients in order to determine the diff to send on reconnect. If unspecified, the upstream-db will be used.

flag: `--cvr-db`<br/>
env: `ZERO_CVR_DB`<br/>
required: `false`

### CVR Max Connections

The maximum number of connections to open to the CVR database. This is divided evenly amongst sync workers.

Note that this number must allow for at least one connection per sync worker, or zero-cache will fail to start. See num-sync-workers.

flag: `--cvr-max-conns`<br/>
env: `ZERO_CVR_MAX_CONNS`<br/>
default: `30`

### Initial Sync Table Copy Workers

The number of parallel workers used to copy tables during initial sync. Each worker copies a single table at a time, fetching rows in batches of `initial-sync-row-batch-size`.

flag: `--initial-sync-table-copy-workers`<br/>
env: `ZERO_INITIAL_SYNC_TABLE_COPY_WORKERS`<br/>
default: `5`

### Lazy Startup

Delay starting the majority of zero-cache until first request.

This is mainly intended to avoid connecting to Postgres replication stream until the first request is received, which can be useful i.e., for preview instances.

Currently only supported in single-node mode.

flag: `--lazy-startup`<br/>
env: `ZERO_LAZY_STARTUP`<br/>
default: `false`

### Litestream Executable

Path to the litestream executable. This option has no effect if litestream-backup-url is unspecified.

flag: `--litestream-executable`<br/>
env: `ZERO_LITESTREAM_EXECUTABLE`<br/>
required: `false`

### Litestream Config Path

Path to the litestream yaml config file. zero-cache will run this with its environment variables, which can be referenced in the file via `${ENV}` substitution, for example:

- ZERO_REPLICA_FILE for the db Path
- ZERO_LITESTREAM_BACKUP_LOCATION for the db replica url
- ZERO_LITESTREAM_LOG_LEVEL for the log Level
- ZERO_LOG_FORMAT for the log type

flag: `--litestream-config-path`<br/>
env: `ZERO_LITESTREAM_CONFIG_PATH`<br/>
default: `./src/services/litestream/config.yml`

### Litestream Log Level

flag: `--litestream-log-level`<br/>
env: `ZERO_LITESTREAM_LOG_LEVEL`<br/>
default: `warn`
values: `debug`, `info`, `warn`, `error`

### Litestream Backup URL

The location of the litestream backup, usually an s3:// URL. This is only consulted by the replication-manager. view-syncers receive this information from the replication-manager. 

flag: `--litestream-backup-url`<br/>
env: `ZERO_LITESTREAM_BACKUP_URL`<br/>
required: `false`

### Litestream Checkpoint Threshold MB

The size of the WAL file at which to perform an SQlite checkpoint to apply the writes in the WAL to the main database file. Each checkpoint creates a new WAL segment file that will be backed up by litestream. Smaller thresholds may improve read performance, at the expense of creating more files to download when restoring the replica from the backup.

flag: `--litestream-checkpoint-threshold-mb`<br/>
env: `ZERO_LITESTREAM_CHECKPOINT_THRESHOLD_MB`<br/>
default: `40`

### Litestream Incremental Backup Interval Minutes

The interval between incremental backups of the replica. Shorter intervals reduce the amount of change history that needs to be replayed when catching up a new view-syncer, at the expense of increasing the number of files needed to download for the initial litestream restore.

flag: `--litestream-incremental-backup-interval-minutes`<br/>
env: `ZERO_LITESTREAM_INCREMENTAL_BACKUP_INTERVAL_MINUTES`<br/>
default: `15`

### Litestream Snapshot Backup Interval Hours

The interval between snapshot backups of the replica. Snapshot backups make a full copy of the database to a new litestream generation. This improves restore time at the expense of bandwidth. Applications with a large database and low write rate can increase this interval to reduce network usage for backups (litestream defaults to 24 hours).

flag: `--litestream-snapshot-backup-interval-hours`<br/>
env: `ZERO_LITESTREAM_SNAPSHOT_BACKUP_INTERVAL_HOURS`<br/>
default: `12`

### Litestream Restore Parallelism

The number of WAL files to download in parallel when performing the initial restore of the replica from the backup.

flag: `--litestream-restore-parallelism`<br/>
env: `ZERO_LITESTREAM_RESTORE_PARALLELISM`<br/>
default: `48`

### Log Format

Use text for developer-friendly console logging and json for consumption by structured-logging services.

flag: `--log-format`<br/>
env: `ZERO_LOG_FORMAT`<br/>
default: `"text"`<br/>
values: `text`, `json`

### Log IVM Sampling

How often to collect IVM metrics. 1 out of N requests will be sampled where N is this value.

flag: `--log-ivm-sampling`<br/>
env: `ZERO_LOG_IVM_SAMPLING`<br/>
default: `5000`

### Log Level

Sets the logging level for the application.

flag: `--log-level`<br/>
env: `ZERO_LOG_LEVEL`<br/>
default: `"info"`<br/>
values: `debug`, `info`, `warn`, `error`

### Log Slow Hydrate Threshold

The number of milliseconds a query hydration must take to print a slow warning.

flag: `--log-slow-hydrate-threshold`<br/>
env: `ZERO_LOG_SLOW_HYDRATE_THRESHOLD`<br/>
default: `100`

### Log Slow Row Threshold

The number of ms a row must take to fetch from table-source before it is considered slow.

flag: `--log-slow-row-threshold`<br/>
env: `ZERO_LOG_SLOW_ROW_THRESHOLD`<br/>
default: `2`

### Log Trace Collector

The URL of the trace collector to which to send trace data. Traces are sent over http. Port defaults to 4318 for most collectors.

flag: `--log-trace-collector`<br/>
env: `ZERO_LOG_TRACE_COLLECTOR`<br/>
required: `false`

### Number of Sync Workers

The number of processes to use for view syncing. Leave this unset to use the maximum available parallelism. If set to 0, the server runs without sync workers, which is the configuration for running the replication-manager.

flag: `--num-sync-workers`<br/>
env: `ZERO_NUM_SYNC_WORKERS`<br/>
required: `false`

### Per User Mutation Limit Max

The maximum mutations per user within the specified windowMs.

flag: `--per-user-mutation-limit-max`<br/>
env: `ZERO_PER_USER_MUTATION_LIMIT_MAX`<br/>
required: `false`

### Per User Mutation Limit Window (ms)

The sliding window over which the perUserMutationLimitMax is enforced.

flag: `--per-user-mutation-limit-window-ms`<br/>
env: `ZERO_PER_USER_MUTATION_LIMIT_WINDOW_MS`<br/>
default: `60000`

### Port

The port for sync connections.

flag: `--port`<br/>
env: `ZERO_PORT`<br/>
default: `4848`

### Push URL

The URL of the API server to which zero-cache will push mutations. Required if you use [custom mutators](/docs/custom-mutators).

flag: `--push-url`<br/>
env: `ZERO_PUSH_URL`<br/>
required: `false`

### Query Hydration Stats

Track and log the number of rows considered by query hydrations which take longer than **log-slow-hydrate-threshold** milliseconds.

This is useful for debugging and performance tuning.

flag: `--query-hydration-stats`<br/>
env: `ZERO_QUERY_HYDRATION_STATS`<br/>
required: `false`

### Replica Vacuum Interval Hours

Performs a VACUUM at server startup if the specified number of hours has elapsed since the last VACUUM (or initial-sync). The VACUUM operation is heavyweight and requires double the size of the db in disk space. If unspecified, VACUUM operations are not performed.

flag: `--replica-vacuum-interval-hours`<br/>
env: `ZERO_REPLICA_VACUUM_INTERVAL_HOURS`<br/>
required: `false`

### Server Version

The version string outputted to logs when the server starts up.

flag: `--server-version`<br/>
env: `ZERO_SERVER_VERSION`<br/>
required: `false`

### Storage DB Temp Dir

Temporary directory for IVM operator storage. Leave unset to use `os.tmpdir()`.

flag: `--storage-db-tmp-dir`<br/>
env: `ZERO_STORAGE_DB_TMP_DIR`<br/>
required: `false`

### Target Client Row Count

A soft limit on the number of rows Zero will keep on the client. 20k is a good default value for most applications, and we do not recommend exceeding 100k. See [Client Capacity Management](/docs/reading-data#client-capacity-management) for more details.

flag: `--target-client-row-count`<br/>
env: `ZERO_TARGET_CLIENT_ROW_COUNT`<br/>
default: `20000`

### Task ID

Globally unique identifier for the zero-cache instance. Setting this to a platform specific task identifier can be useful for debugging. If unspecified, zero-cache will attempt to extract the TaskARN if run from within an AWS ECS container, and otherwise use a random string.

flag: `--task-id`<br/>
env: `ZERO_TASK_ID`<br/>
required: `false`

### Upstream Max Connections

The maximum number of connections to open to the upstream database for committing mutations. This is divided evenly amongst sync workers. In addition to this number, zero-cache uses one connection for the replication stream.

Note that this number must allow for at least one connection per sync worker, or zero-cache will fail to start. See num-sync-workers.

flag: `--upstream-max-conns`<br/>
env: `ZERO_UPSTREAM_MAX_CONNS`<br/>
default: `20`