Connections and connectors

A connector is a type. A connection is one configured instance of that type, owned by one org. Connectors are code; connections are rows.

A connection is a row

Creating a connection inserts a row in the connection table. It holds:

• org_id — the tenant that owns the feed. Every event it produces is stamped with this org, and every read is fenced by it before any visibility rule runs.
• connector — the type key (slack, http, …).
• name — your label for this feed, e.g. "Sales workspace".
• settings — the non-secret answers you gave (paths, workspace labels, flags).
• credentials — the secret answers, sealed. AES-256-GCM under LOUVAIN_CREDENTIALS_KEY, stored as an envelope that records which sealer wrote it. List endpoints never select this column, so a connection listing is structurally incapable of leaking a token.
• ingest_token_hash — the SHA-256 of the connection's own ingest token. The plaintext token is returned exactly once, at creation.
• status — disabled, enabled, or error. New connections start disabled. error is not final: the manager retries it on a schedule (below) until it stays up or someone pauses it. A disabled connection's ingest token stops working: disabling stops louvain's own runner, and it also refuses the credential, so anything else still holding it — a customer's script, an inbound http source that pushes rather than being polled, a runner on a host nobody remembered — is refused too. Disabling that only stopped the runner meant the page said "Disabled" while events kept arriving. An error connection still authenticates: that is one louvain is actively retrying, not one an operator turned off.
• last_event_at, last_error — liveness and the last failure.
• attempts, next_attempt_at — how many runs in a row have failed, and when an errored connection is tried again. The page renders these as "Error · retrying in 12 min".

Credentials are decrypted at exactly one site: spawning a runner for that connection. Nothing on a read path can decrypt them.

Two ways in, and the difference is coverage

A definition declares its authMethods — the ways a customer can prove they may read the source. Most platforms offer two, and choosing between them is not a detail of the login screen. It decides how much of the company the feed can see.

Signing in personally (kind: "oauth") is one click and needs no administrator. The connector has no secret fields at all: authorize() returns a URL, the customer approves at the provider, and the credential is held there rather than by louvain. The feed then reads exactly what that person reads — their mail, their files, the channels they are in. For one team that is often the whole point.

A service credential (kind: "credentials") is a bot token, an app installation or a service account, issued by an administrator at the source and pasted into the form, where it is sealed as above. The feed reads the workspace.

Every method declares reads, and it is declared rather than inferred from the protocol, because the protocol gets it wrong: Slack's OAuth install issues a bot token and reads the workspace, while Google's OAuth reads one person's Drive. The Connections page shows the two as tabs with that sentence under them, and the connection row afterwards carries a Reads as you or Reads the workspace chip — because two connections to the same platform can differ by an order of magnitude in what they cover, and nothing else on the row would say so.

Where one platform is read by several connector types, the tabs span the types too. Slack is the case: the Socket Mode app (slack), the corporate export (slack-export) and the brokered OAuth install (slack-oauth) declare one family, so the page shows one Slack card with a tab per way in — each tab carrying its method's reads and its type's membership, because two of them sync channel membership and one reads the workspace as a single container. The keys stay separate: a key is in every container id and tuple subject already written, so the family groups the card and nothing else.

A connection that has authorised is ready, not running, and still has to be enabled. If the authorisation call itself fails — a broker that is down, a key nobody set — nothing is created: the connection is rolled back, because a row whose authorisation never left the building is not a state worth keeping. Once the customer has been sent to the provider the row survives, since they may finish over there minutes later and come back.

A definition declares three more things that exist only so the picker can be read at a glance, none of which touch ingest or visibility: a category (messaging, files, records, custom) for grouping, keywords for the words people search that a product's own name does not contain, and a brand naming the logo to draw.

It also declares membership, which is not cosmetic at all — see what a source can tell louvain about permissions. synced sources publish per-object membership; container sources do not, and their content is read at container visibility. The picker shows this as a chip on every entry, before you connect it.

The ingest token

Every connection gets its own bearer token, prefixed louvainc_. It is the only thing that identifies a feed to the API:

authorization: Bearer louvainc_deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef

The API derives the org and the connection from the token, never from anything in the payload. A forged or unknown token is a 401 — there is no downgrade path. Rotating the token re-seals a new one and the old one stops authenticating immediately; deleting the connection detaches its events (ingest history is not rewritten) and removes the credential entirely.

Two shapes of connector type

Every connector definition declares a mode, and the mode decides whether anything runs.

Runner-backed (streaming, one-shot)

The definition carries a run() that turns the connection's answers into a process to spawn. The connection manager reconciles rows to processes on a 3-second loop:

• status: enabled and not running → spawn, with LOUVAIN_API_URL and LOUVAIN_INGEST_TOKEN injected into the child's environment.
• status changed away from enabled → SIGTERM.
• Non-zero exit → restart on the next reconcile for the first 3 failures, then the connection flips to error with the exit code in last_error and a retry scheduled in next_attempt_at: 1 minute, then 2, 4, 8 … capped at an hour, forever, until an operator pauses it. A run that stays up 5 minutes resets the schedule. The schedule is on the row, so an API restart resumes it rather than starting over. It does not crashloop, and it does not give up: a restart cap once left a feed dead for 23 hours over a failure that cleared itself in one.
• Every failure is retried the same way, including a bad credential. Runners exit 1 for a revoked token, a broker 5xx and an unreachable API alike, so the manager does not pretend to tell them apart; retrying a dead credential costs one process spawn an hour, and not retrying a live one costs the feed.
• Exit code 0 → the connection flips to disabled. A one-shot import that finished is shown as not running, because it is not running.

A runner that follows a change feed keeps its cursor in louvain, not in its own process, through GET/PUT /v1/ingest/state — scoped by the ingest token, so a connection can read and overwrite its own cursor and no other. The runner is the thing that restarts; the cursor has to outlive it. Advance it only after ingest has accepted the events it covers.

Runners are out-of-process and language-neutral by design. A built deployment runs the compiled runner on node; a dev checkout runs the TypeScript source through tsx. The connector definition is the same either way.

Inbound (inbound)

An inbound type has no run() and no runner. It is a credential and a permission boundary, nothing more: you get an ingest token, and your system pushes events to the ingest endpoint on its own schedule. The manager skips inbound connections entirely when reconciling — without that, an enabled inbound connection would fail to spawn every few seconds and mark itself errored, which would be a lie about a healthy feed.

What an inbound source pushes crosses exactly the same authenticated ingest, the same org fence, and the same visibility rules as a runner-backed feed. The escape hatch is not a bypass.

The connector types that ship

These are the types registered in core. The Connections UI renders whatever is registered, so this list is the catalog you will see.

slack and slack-export declare the slack family, so the page shows them as one Slack card with a tab each — Slack app (Socket Mode) and Export archive — rather than two entries a customer has to tell apart.

A further twenty-one types — Slack, Teams, Gmail, Google Drive, SharePoint, Notion, GitHub, Salesforce and the rest — are registered by the enterprise edition and authorise through a broker rather than taking a pasted token. They are ordinary definitions in the same registry: see OAuth sources. The brokered Slack (slack-oauth) joins the same Slack card as two more tabs.

One more type, demo, is registered only when LOUVAIN_DEV_INSECURE_HEADER=true. It manufactures synthetic traffic for local development and the playground. In any deployment without that flag it is absent, not hidden — it never appears in a customer's picker.

Seeded connections, for a demo workspace

LOUVAIN_DEMO_FIXTURES=true adds one extra way in — Seeded demo data — to a short list of connector types, for building a demonstration workspace that holds a synthetic corpus instead of a live feed. A seeded connection is connectable with no broker behind it and stays enabled without starting a runner; everything downstream is ordinary, so its containers, permission tuples and provenance are the real ones and the visibility algebras never learn it was seeded. The data is synthetic; the plumbing is not.

Like the dev header, it is refused outside local development: preflight fails boot rather than warning, because a connection nobody has to authenticate is not something a real deployment should be able to create. The seeded method is appended to a type's existing ways in, never made the default.

Details per type: Slack and HTTP inbound.

How a connection maps to permission containers

A connection does not grant access to anything. Access comes from the events it sends.

Each event names a source.connector and a source.container — the ACL-bearing unit at the source, such as a Slack channel. louvain stores the container as <connector>/<container>, so a #sales channel from the slack connector becomes the container slack/sales.

Membership is a tuple:

{
  "resource": "container:slack/sales",
  "relation": "member",
  "subject": "user:slack/U042ABCDEF"
}

Three rules follow from this, and violating any of them produces a feed that looks perfectly healthy and that nobody can read anything out of:

1. The resource must be container:<connector>/<container> — matching the container the events themselves carry. A tuple pointing at a container no event uses grants access to nothing.
2. The subject must be connector-namespaced — user:slack/U042ABCDEF, not user:U042ABCDEF. The readable-set projection resolves a person's linked source identities as connector/sourceUserId. An unprefixed subject syncs happily and is then never looked up.
3. References must be type:id. Malformed references are rejected at ingest with a 400 rather than at delivery, because the relay that carries every permission change to the authorization store retries forever on a tuple that will never become valid.

Zero attestations means invisible. louvain never widens access: an event whose container nobody is a member of is visible to nobody, which is the correct outcome and not an error.

Adding a connection in the UI

Connections at /app/connections is the catalogue. Every registered type is on the page under what it is — conversations, documents, records — with its logo, what it treats as one place (Channels, Files, Mailboxes), and a mark on the ones that sync their own membership. Search matches names, descriptions and keywords, so "email" finds Gmail and Outlook.

1. Sign in and open Connections. What you already run is the strip of marks at the top; hover one for its name, status and volume, or click it to pause, rotate its token or remove it.
2. Click any source to set it up. Each ships its own setup steps — what you must do at the source first — so the form tells you what to go and get, and which way in you choose is a tab at the top of it. A platform read by several connector types is one card too, with a tab per way in; the tab you pick decides which type the connection is created under.
3. Fill in the fields. Fields declared kind: "secret" are split off and sealed; everything else is stored as plain settings. Missing required fields are rejected with the field keys named.
4. The ingest token is shown once. Copy it if the source needs it (inbound types); for runner-backed types the manager hands it to the runner automatically and you never need to touch it.
5. Press Enable when you want the feed running. Nothing reads anything until you do.

The same thing over the API

Create body:

{
  "connector": "http",
  "name": "CRM notes",
  "settings": { "workspace": "acme" }
}

credentials is a flat { key: value } map of strings; the API splits it against the connector's own field descriptors, so you send the field keys the catalog told you about and nothing has to know which ones are secret.

Roles gate administration only. Being an owner lets you create and delete connections; it does not let you see one more claim than your principal's containers allow.

Registering a new connector type

Adding a platform is a definition plus a runner script — never a change to the web app, because the UI renders whatever is registered. An extension loaded with LOUVAIN_EXTENSIONS=<path> calls registerConnector at boot:

api.registerConnector("sample-firehose", {
  label: "Sample firehose",
  description: "Reference connector type registered by an extension — appears in the UI.",
  mode: "streaming",
  fields: [{ key: "workspace", label: "Workspace label", kind: "text", required: false }],
  run: (_credentials, settings) => ({
    script: "apps/api/scripts/demo-stream.ts",
    args: [],
    env: { LOUVAIN_DEMO_SETTINGS: JSON.stringify(settings) },
  }),
});

The registry is a deliberate extension point. The mechanism is pluggable; enforcement is not. Whatever a registered runner emits still crosses the same authenticated ingest, the same org fence, and the same visibility rules.