GDPR & Data Privacy

Event sourcing architectures are built on an immutable, append-only log: once written, events record historical facts that should never be modified or deleted. However, Article 17 of the General Data Protection Regulation (GDPR) grants individuals the "Right to Erasure" (the right to be forgotten).

Naively deleting rows from an event store breaks stream versioning, destroys causal integrity, and invalidates deterministic state replay. To satisfy regulatory erasure requirements without compromising the integrity of your event store, two production-proven patterns are available:

1. Pattern A: Reference-Based PII (Keep PII Outside the Event Store) — Store personal data in standard mutable PostgreSQL tables and reference only opaque, non-semantic identifiers inside event payloads.
2. Pattern B: In-Stream Envelope Encryption (Crypto-Shredding) — Encrypt personal fields inside event payloads using per-entity encryption keys. Revoking a key renders the ciphertext permanently unreadable (tombstoned).

Architectural Decision Matrix

Pattern A: Reference-Based PII (Keep PII Outside the Event Store)

The simplest and most resilient way to handle GDPR Article 17 is to keep personal data entirely out of the event log.

Instead of embedding names, email addresses, or physical addresses in event payloads, store PII in a standard mutable PostgreSQL table (user_profiles). The immutable events contain only an opaque, non-semantic identifier (such as userId: "usr_123").

1. PostgreSQL Schema Design

Define a standard relational table for personal data. Because this table is separate from the event log, it supports standard relational features including uniqueness constraints, foreign keys, and indexes:

CREATE TABLE user_profiles (
  user_id TEXT PRIMARY KEY,
  email TEXT NOT NULL,
  full_name TEXT NOT NULL,
  phone TEXT,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Plaintext indexes for fast lookup and uniqueness
CREATE UNIQUE INDEX idx_user_profiles_email ON user_profiles (lower(email));

The event payload contains only non-personal business facts alongside the opaque identifier:

{
  "userId": "usr_123",
  "orderId": "ord_987",
  "totalAmount": 14900,
  "currency": "EUR"
}

2. Atomic Writes with eventStore.withTransaction

To prevent inconsistencies where a profile is updated but the corresponding event fails to append (or vice-versa), execute both operations inside a single PostgreSQL transaction using eventStore.withTransaction:

import { eventStore } from "./event-store";

await eventStore.withTransaction(async (client) => {
  // 1. Upsert mutable profile with PII
  await client.query(
    `INSERT INTO user_profiles (user_id, email, full_name, updated_at)
     VALUES ($1, $2, $3, NOW())
     ON CONFLICT (user_id) DO UPDATE
     SET email = EXCLUDED.email, full_name = EXCLUDED.full_name, updated_at = NOW()`,
    [userId, email, fullName],
  );

  // 2. Append event referencing only the opaque userId
  await eventStore.append(
    {
      streamId: `User-${userId}`,
      expectedVersion: 0,
      events: [
        {
          type: "UserRegistered",
          data: { registeredAt: new Date().toISOString() },
        },
      ],
    },
    { client }, // Atomic: commits or rolls back together with the profile upsert
  );
});

Because both writes share the same PostgreSQL client and transaction, your system guarantees atomic consistency with zero distributed transaction overhead.

3. Deletion Flow (Physical Deletion)

When a user exercises their GDPR Article 17 Right to Erasure, delete their row from user_profiles using standard SQL:

await pool.query("DELETE FROM user_profiles WHERE user_id = $1", [userId]);

The event stream (User-usr_123 or orders linked to usr_123) remains completely untouched in the append-only log. The event history continues to record the business fact that a registration or purchase occurred, but the identifier usr_123 is now an orphaned pseudonym that can no longer be resolved to an identifiable natural person.

4. Handling Orphaned References During Replay

When read models or downstream projections join events with user_profiles, they must handle missing profile records gracefully:

async function getUserSummary(userId: string) {
  // Replay immutable business events
  const events = await eventStore.load(`User-${userId}`);

  // Resolve current profile (may be null if deleted)
  const { rows } = await pool.query(
    "SELECT email, full_name FROM user_profiles WHERE user_id = $1",
    [userId],
  );
  const profile = rows[0] ?? null;

  return {
    userId,
    isDeleted: !profile,
    email: profile?.email ?? "[DELETED]",
    fullName: profile?.full_name ?? "Anonymized User",
    registrationEvent: events.find((e) => e.type === "UserRegistered"),
  };
}

5. Architectural Trade-offs

Advantages:

• Zero Cryptographic Overhead: No root keys, keyrings, encryption envelopes, or performance penalty for encryption/decryption.
• Full SQL Indexability: Search, filter, and sort profiles by email or name using standard B-Tree or full-text indexes (pg_trgm).
• Unambiguous Legal Erasure: Physical deletion via DELETE is universally accepted by data protection authorities as true erasure under GDPR Article 17.
• Simplicity (KISS): Regular PostgreSQL CRUD logic without special evolve-handler fallbacks or tombstone types.

Limitations:

• Loss of Historical Snapshot: The profile table holds only the current state. You cannot see what a user's address was at the exact moment a past order occurred (unless you maintain a separate audited version history).
• Reduced Event Autonomy: Events are not self-contained. Any consumer or projection that requires personal details must query the relational profile table.
• Identifier Hygiene: An ID is not anonymous simply because it is called id. It must be an opaque, non-semantic UUID or random token that cannot be reversed or correlated with the person through external systems.

Pattern B: In-Stream Envelope Encryption (Crypto-Shredding)

When downstream consumers and projections require self-contained event payloads, or when personal data is distributed across multiple distinct streams, per-entity envelope encryption (crypto-shredding) enables instant, surgical data revocation.

Events store personal data encrypted under a unique, entity-specific AES-256-GCM key. Revoking that single key renders all historical ciphertext for that entity unreadable in $O(1)$ time, without modifying any append-only event rows.

Two-Tier Envelope Encryption Architecture

Alvyn implements a two-tier key hierarchy:

1. Versioned Root Secrets (Key Encryption Key): A configured server secret (currentVersion) wraps and unwraps per-entity keys using scrypt and AES-256-GCM. Rotating root secrets requires zero database re-encryption.
2. Per-Entity AES Keys (Data Encryption Key): Each entity (e.g. user, tenant) gets a unique 256-bit AES key stored encrypted in the crypto_keys table.
3. Field-Level Authenticated Encryption: Individual PII fields (e.g. email, address.street) are encrypted with the entity key, each with a unique random IV and GCM authentication tag.

AES-GCM also authenticates the meaning of each encrypted value. Entity-key envelopes are bound to their key ID, magic marker, and secret version. PII fields are bound to their event ID, entity-key ID, field path, and stored version as Additional Authenticated Data (AAD). Moving a valid ciphertext to another row, event, or field causes decryption to fail.

How It Works (Write & Read Paths)

Write Path (append)

1. For each event with encryptedFields + cryptoKeyId, the entity's AES key is retrieved and decrypted from the database using the configured secret version.
2. Each PII field (specified as dot-paths, e.g. "address.street") is:
   • Extracted from the event data
   • JSON-stringified and encrypted with AES-256-GCM (random IV per field)
   • Stored in the encrypted_data column as { version, ciphertext, iv, authTag } (base64-encoded)
3. The data column stores the event payload with PII fields removed.

Example of what gets stored in the database:

data column:            { "loginCount": 5 }
encrypted_data column:  { "name": { "version": 2, "ciphertext": "...", "iv": "...", "authTag": "..." },
                          "email": { "version": 2, "ciphertext": "...", "iv": "...", "authTag": "..." } }
crypto_key_id column:   "user:abc123"

Read Path (load)

1. If the entity's crypto key is active: fields are decrypted and merged back into the event data. The caller sees the complete event as if encryption was transparent.
2. If the key is revoked and fields were actually encrypted: the event is returned as a TombstonedEvent with non-encrypted fields preserved and encrypted field paths set to null. The payload is in its original schema and is not upcasted. A configured encrypted field that was absent from an event is not encrypted and does not cause a tombstone later. Missing key rows or missing crypto configuration are errors, not intentional erasure.

Setting Up Encryption

1. Configure Versioned Secrets

Generate secret values for the versions you will use:

openssl rand -base64 32

Configure the complete keyring and explicitly select the version used for new encryption. The order of entries is not significant:

const eventStore = new EventStore({
  pool,
  secrets: {
    currentVersion: 2,
    secrets: [
      { version: 1, value: process.env.GDPR_CRYPTO_SECRET_V1! },
      { version: 2, value: process.env.GDPR_CRYPTO_SECRET_V2! },
    ],
  },
});

The same configuration can be supplied through the environment:

GDPR_CRYPTO_SECRETS=2:new-secret-base64,1:old-secret-base64
GDPR_CRYPTO_CURRENT_VERSION=2

Secret values are strings. All values are strengthened with scrypt to derive the 32-byte key used to protect entity keys. Use a high-entropy generated value; scrypt makes guessing more expensive but cannot add entropy to a weak secret. Keep every old secret needed to read existing versioned envelopes until that data has been intentionally retired; removing a configured version makes those envelopes unreadable.

Secret versions are unsigned 32-bit integers from 0 to 4294967295. Gaps are allowed, and currentVersion must identify one of the configured secrets. Use a higher version for each new rotation, but do not encode the active version by reordering the keyring. Rotating between secret versions does not require a database migration.

2. Create Entity Keys

Create a per-entity key before appending encrypted events:

await eventStore.createCryptoKey("user:abc123");

This is idempotent — if the key already exists, it is a no-op.

3. Configure Aggregate Encryption

In the aggregate definition, specify which fields contain PII per event type:

type UserState = {
  name: string;
  email: string;
  address: { street: string; city: string };
};

const User = defineAggregate<UserState, UserEvents>()({
  streamPrefix: "User",
  // ... evolve ...

  encryption: {
    cryptoKeyId: (entityId) => `user:${entityId}`,
    encryptedFields: {
      UserRegistered: ["name", "email", "address.street"],
      UserRenamed: ["name"],
    },
  },
});

4. Or Use Low-Level API

If not using aggregates, specify encryption per-event in the append call:

await eventStore.append({
  streamId: "User-abc123",
  expectedVersion: 0,
  events: [
    {
      type: "UserRegistered",
      data: { name: "Alice", email: "alice@example.com", loginCount: 0 },
      encryptedFields: ["name", "email"],
      cryptoKeyId: "user:abc123",
    },
  ],
});

Dot-Path Field Notation

Encrypted fields are specified using dot-path notation for nested objects:

encryptedFields: ["name", "address.street", "address.city"];

Given event data:

{
  "name": "Alice",
  "address": { "street": "123 Main", "city": "Berlin" },
  "active": true
}

After encryption, data column contains:

{ "address": {}, "active": true }

And encrypted_data column contains:

{
  "name": { "version": 2, "ciphertext": "...", "iv": "...", "authTag": "..." },
  "address.street": {
    "version": 2,
    "ciphertext": "...",
    "iv": "...",
    "authTag": "..."
  },
  "address.city": {
    "version": 2,
    "ciphertext": "...",
    "iv": "...",
    "authTag": "..."
  }
}

Deletion Flow (Revoking Keys)

The complete flow for handling a user deletion request via crypto-shredding:

// 1. When user registers — create their crypto key
await eventStore.createCryptoKey("user:abc123");

// 2. Normal operation — append events with encrypted PII
await User.append(eventStore, {
  entityId: "abc123",
  expectedVersion: -1,
  events: [
    {
      type: "UserRegistered",
      data: {
        name: "Alice",
        email: "alice@example.com",
        address: { street: "123 Main", city: "Berlin" },
      },
    },
  ],
});

// 3. User requests deletion — revoke the key
await eventStore.revokeKey("user:abc123");

// 4. All future reads return tombstones for encrypted events
const user = await User.load(eventStore, "abc123");
// user.state contains the result of your null-safe evolve handlers

What revokeKey Does

Within a single transaction:

1. Sets revoked_at = now() on the crypto key.
2. Sets encrypted_key = NULL, permanently removing the database copy of the wrapped entity key.

The crypto-key row remains only to retain non-sensitive audit metadata such as key_id, revoked_at, algorithm, and created_at. Revocation is idempotent: repeating it keeps the row tombstoned and the key material absent.

Appends lock the entity-key row until their transaction commits. Revocation uses the same PostgreSQL row lock, so the ordering is safe across multiple application replicas: an append that acquired the lock first commits before revocation, while an append that starts after revocation sees the revoked key and fails with CryptoKeyRevokedError.

Handling Tombstones in Evolve Handlers

After key revocation, encrypted events are returned as TombstonedEvent:

interface TombstonedEvent {
  globalPosition: bigint;
  streamId: string;
  streamVersion: number;
  type: string; // Event type name — you know what happened
  data: unknown; // Original-schema public data; encrypted field paths are null
  redactedPaths: readonly string[];
  schemaVersion: number;
  extensions: object; // Extensions are NOT encrypted
  createdAt: Date;
  tombstoned: true;
}

Important characteristics:

• Only encrypted field paths within data become null — the PII is cryptographically irrecoverable; other fields remain available in the original schema.
• extensions is still available (extensions are never encrypted).
• type is still available — you know what business event happened, just not the personal details.
• Non-encrypted events in the same stream are not affected.

Evolve handlers must distinguish complete events from redacted, original-schema payloads. Validate public fields before using them and do not retain erased PII from previous state:

evolve: {
  UserRegistered: (state, event) => {
    if ("tombstoned" in event) {
      const data = event.data;
      const active = typeof data === "object" && data !== null &&
        "active" in data && typeof data.active === "boolean"
          ? data.active : false;
      return { ...state, name: "", email: "", active };
    }
    return { ...state, name: event.data.name,
      email: event.data.email, active: event.data.active };
  },
}

The creation event can itself be redacted; initialize any encrypted fields from safe defaults while retaining public fields. Avoid copying old PII from an earlier state when a field has been explicitly redacted.

Snapshot warning: Aggregate encryption is not inherited by defineSnapshot. A snapshot can otherwise persist decrypted PII in plaintext, surviving source-key revocation. Configure snapshot encryption explicitly with the same entity key and all sensitive snapshot-state paths; see the snapshot encryption example. Also erase or invalidate downstream read models, caches, and exported copies.

Rotating Secrets Without Downtime

For a single application instance, add the new version to the keyring and set it as currentVersion. For an HA or rolling deployment, use two phases so every replica can read version 2 before any replica writes it.

Phase 1: Distribute the New Secret

Deploy this configuration to every replica while version 1 remains current:

secrets: {
  currentVersion: 1,
  secrets: [
    { version: 1, value: "old-secret" },
    { version: 2, value: "new-secret" },
  ],
};

Wait until all replicas have this configuration. They continue writing version 1, but all of them can decrypt version 2.

Phase 2: Activate the New Secret

After phase 1 is complete, keep the same keyring and change only the active version:

secrets: {
  currentVersion: 2,
  secrets: [
    { version: 1, value: "old-secret" },
    { version: 2, value: "new-secret" },
  ],
};

New entity-key envelopes and encrypted fields use version 2. A replica still on phase 1 can read a version 2 envelope and will never downgrade it back to version 1. Existing version 1 envelopes are decrypted by direct version lookup and are not rewritten during reads. The next append for an entity lazily re-wraps only its entity key with version 2; existing event ciphertext is not rewritten. No full database migration or downtime is required.

Removing the Old Secret

Do not remove version 1 merely because phase 2 finished. Lazy rotation means an inactive entity may keep a version 1 envelope indefinitely. Before removal, verify that no active entity-key envelope still uses it:

SELECT
  get_byte(encrypted_key, 4)::bigint * 16777216
    + get_byte(encrypted_key, 5)::bigint * 65536
    + get_byte(encrypted_key, 6)::bigint * 256
    + get_byte(encrypted_key, 7)::bigint AS secret_version,
  count(*)
FROM event_store.crypto_keys
WHERE revoked_at IS NULL
GROUP BY secret_version
ORDER BY secret_version;

Use your configured schema instead of event_store. Keep the old secret until the old version count is zero, retain it indefinitely, or intentionally retire the remaining data. An inactive key is not automatically rewrapped by reads.

Error Handling

CryptoKeyRevokedError is only thrown on write operations. On read, revoked keys simply produce tombstoned events without throwing.

Cryptographic Security Details

• Algorithm: AES-256-GCM (authenticated encryption) for both secret-to-entity and entity-to-field encryption.
• IV: 12 bytes, randomly generated per encryption operation.
• Auth tag: 16 bytes.
• Entity key storage format: Versioned envelope [magic][secret version][iv][authTag][ciphertext] stored as BYTEA; the header and key ID are authenticated as AAD.
• Field storage format: { version, ciphertext, iv, authTag } as base64 strings in JSONB; event ID, key ID, field path, and version are authenticated as AAD.
• Key cache: Decrypted entity keys are cached in a per-read-call Map (not across requests) to avoid redundant DB lookups within a single stream load.

Operational & Legal Realities (Backups, WAL, and Retention)

Regardless of whether you choose Pattern A (Reference Tables) or Pattern B (Crypto-Shredding), neither pattern physically alters historical database backups or Write-Ahead Logs (WAL). Understanding these operational boundaries is essential for an airtight compliance posture.

1. Database Backups & Point-in-Time Recovery (PITR)

• Pattern A (Reference Tables): Executing DELETE FROM user_profiles WHERE user_id = $1 deletes the row in the active database. However, a database dump or cloud snapshot taken yesterday still contains that row. If that backup is ever restored, deleted profiles will reappear unless your disaster recovery procedures re-apply deletion logs.
• Pattern B (Crypto-Shredding): Calling eventStore.revokeKey(keyId) zeroes the encrypted_key in the live database. Past backups still contain the encrypted key and root secret. If an old backup is restored, the key becomes active again unless the key revocation is replayed or root secrets have rotated.
• Actionable Guidance: Establish a clearly documented backup retention schedule (e.g., 14 to 30 days). Under GDPR guidance, data in static disaster-recovery archives is generally considered compliant if it is put beyond practical use and naturally purged as backup retention windows expire.

2. Write-Ahead Logs (WAL) & Streaming Replicas

PostgreSQL WAL segments record low-level disk page modifications. In Pattern A, WAL contains the pre-image of the deleted row; in Pattern B, WAL contains the pre-image of the revoked key.

• WAL files are transient and automatically recycled based on PostgreSQL configuration (wal_keep_size, max_wal_size, and archive cleanup scripts).
• Standby replicas replicate deletion transactions near-instantaneously: a SQL DELETE or revokeKey query executed on the primary propagates to all streaming replicas within milliseconds.

3. Legal Posture & Data Protection Authorities

• Physical Deletion (Pattern A): Universally recognized by European Data Protection Authorities (including the CNIL, ICO, and BfDI) as definitive erasure under Article 17.
• Crypto-Shredding (Pattern B): Cryptographic erasure is widely accepted as an effective technical measure when key destruction is permanent, the algorithm is state-of-the-art (AES-256-GCM), and root keys are controlled. Because encrypted ciphertext remains in the immutable log, supervisory authorities classify the residual data as pseudonymized until the ciphertext itself is purged or becomes impossible to decrypt even against future computational advances.