Defining Schemas

ReJot allows you to define Public and Consumer Schemas using TypeScript code, which offers several benefits:

• Type safety and validation
• Better code organization and reusability
• Integration with your existing development workflow

Setting Up

First, add the required dependencies to your project:

npm install --save @rejot-dev/contract @rejot-dev/adapter-postgres

Creating Public and Consumer Schema Definitions

Create a new file called schemas.ts that will contain your public and consumer schemas:

import { z } from "zod";

import {
  createPostgresPublicSchemaTransformation,
  createPostgresConsumerSchemaConfig,
} from "@rejot-dev/adapter-postgres";
import { createConsumerSchema } from "@rejot-dev/contract/consumer-schema";
import { createPublicSchema } from "@rejot-dev/contract/public-schema";

const myPublicSchema = createPublicSchema("my-public-schema", {
  source: { dataStoreSlug: "my-source-datastore", tables: ["my_table"] },
  outputSchema: z.object({
    id: z.string(),
    name: z.string(),
  }),
  transformations: [], // See next step
  version: {
    major: 1,
    minor: 0,
  },
});

const myConsumerSchema = createConsumerSchema({
  source: {
    manifestSlug: "my-manifest",
    publicSchema: {
      name: "my-public-schema",
      majorVersion: 1,
    },
  },
  destinationDataStoreSlug: "my-destination-datastore",
  transformations: [], // See next step
});

export default {
  myPublicSchema,
  myConsumerSchema,
};

Transformations

Transformations are a crucial part of schema definitions that specify how data is transformed between different stages of the sync process. Transformation are used by both the publishing and consuming parties, for publishers, the transformation defines how data from the internal representation should be mapped to the public schema. Conversely on the consuming side the transformation maps the public schema back into an internal representation for the consumer.

Public Schema Transformations

For public schemas, you need to create a SQL transformation for each table referenced in the schema. These transformations:

• Take a primary key as input parameter(s) ($1, $2, etc)
• Return exactly one row that matches the public schema’s output structure

Here’s an example of a public schema transformation:

transformations: [
  createPostgresPublicSchemaTransformation(
    "my_table",
    `SELECT id, name FROM my_table WHERE id = $1`,
  ),
];

Multi-table transformations

A common case is to (partially) de-normalize your data for the public schema, to ensure correct results ReJot should listen to updates to all of these tables. As a data producer you will have to define schema transformations for each table.

transformations: [
    createPostgresPublicSchemaTransformation(
      "accounts",
      `SELECT
          accounts.id,
          accounts.name,
          addresses.country
        FROM
          accounts
          JOIN addresses ON accounts.id = addresses.account_id
        WHERE
          accounts.id = $1`,
    ),
    createPostgresPublicSchemaTransformation(
      "addresses",
      `SELECT
          accounts.id,
          accounts.name,
          addresses.country
        FROM
          accounts
          JOIN addresses ON accounts.id = addresses.account_id
        WHERE
          addresses.id = $1`,
    ),
  ],

Consumer Schema Transformations

For consumer schemas, transformations handle the insertion of public schema data into the destination data store. These transformations:

• Use named parameters (e.g., :name, :country) to reference fields from the public schema
• Must handle conflicts appropriately since ReJot doesn’t guarantee exactly-once delivery

Here’s an example of a consumer schema transformation:

transformations: [
  createPostgresConsumerSchemaConfig(
    "INSERT INTO destination_table (id, name) VALUES (:id, :name) ON CONFLICT (id) DO UPDATE SET name = :name",
  ),
];

Materializing Schemas

Code-based schemas need to be materialized into your manifest file to be usable by sync services. This is done using the collect command:

rejot-cli collect schemas.ts --write

This command will:

1. Read your schema definitions from schemas.ts
2. Generate the corresponding manifest entries
3. Update your manifest file (if --write is specified)

The manifest includes a definitionFile key for each schema, which points to the actual source file and is re-used later on to update collected schemas.

If you want to change a schema, edit the file referenced by definitionFile and run the collect command again.

Next Steps

Now that you know how to define schemas, you can run sync services to start and manage sync services using manifests.