Contents
Start here
TERSL — the language
TERSL/DB — the system
Portal guide
HTTP API
Reference
TERSL/DB — the system

How it works

The namespace and its change sequence number, what happens to a request, how writes are ordered and retried, and how whole files go in and out.

Overview#

This page describes what TERSL/DB does with a namespace and with each request sent to it. It covers the change sequence number, the shape of a request and its answer, what happens when a write is refused, how concurrent writers and retries are handled, and how whole files are exported and loaded. After reading it you can send a request, read the answer correctly, and retry safely.

The namespace and its CSN#

A namespace is one TERSL document. Its history is a numbered sequence of writes: the first committed write is CSN 1, and every later committed write takes the next number. The change sequence number (CSN) orders everything that happens to a namespace. A request that commits a write is answered with the CSN it was committed at; every other answer carries the CSN of the state it was computed from. Both are in the Terse-Csn response header.

Every write is kept as an entry in the namespace's log, with the declaration text exactly as it was sent. The current state is the result of applying those entries in CSN order, and so is the state at any earlier CSN. Each entry also records the version of the language it was written in and is always re-applied under that version, so an upgrade of the service does not change what an earlier entry meant. The record page describes the log in detail.

What a request does#

All reads and writes go through one route, POST /v1/ns/{ns}/issue, with a TERSL body. When a request arrives, the service does the following, in order:

  1. It authenticates the key and checks that the namespace's grants admit the key to something. If either check fails, the answer is an empty 404, the same as for a namespace that does not exist.
  2. It checks the account's status and the plan's rate limits. A request over a rate waits on its connection for its turn, or is refused with 429 (see Plans, rates and limits).
  3. It reads the body and splits it into declarations and queries.
  4. It applies the declarations as one unit, checking each effect against the key's grants. If every check passes, the write is committed as one log entry with one new CSN.
  5. It answers the queries against the state after the write, showing only what the key's grants make visible.

The body and the answer are text/plain; charset=utf-8. A body that is not valid UTF-8 is refused with 400.

The Issue#

One request is called an Issue. Its body carries declarations, then a line consisting exactly of the seam token %%%SEAM%%%, then queries. Either half may be empty: a body with no seam line is all declarations, and a body that starts with the seam line is queries only. If your data could contain a line equal to %%%SEAM%%%, choose another token with the Terse-Seam header; the answer is framed with the same token.

The queries half accepts query lines only. A mutation directive written there is ignored with a warning, so a queries half can never change state.

The answer carries the query results, then the seam line, then the Messages channel: one line per message, starting error:, warn: or info:, with the channel and line number they refer to (error: declarations line 7: …, warn: queries line 2: …). The Terse-Diagnostics header counts them as errors=N; warnings=M.

This Issue adds a container and an object, and reads the container back. Expected-Csn asks the service to apply the write only if the namespace is still at CSN 41.

http
POST /v1/ns/7QK4M2C/issue HTTP/1.1
Authorization: Bearer <key>
Content-Type: text/plain; charset=utf-8
Expected-Csn: 41
Terse-Agent: harbour-planner

# Places
## Harbour(berths: 12)
fishing boat
%%%SEAM%%%
? Places.Harbour
http
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Terse-Csn: 42
Terse-Chain: 5e6b0f7c2d9a41e38b7f6c0a9d2e14b5c7a8f3e6d1b0c9a2e4f7d8c6b5a3e1f0
Terse-Diagnostics: errors=0; warnings=0

# Places
## Harbour(berths: 12)
fishing boat
%%%SEAM%%%

Terse-Chain is the hash-chain value of the newest entry; keep it if you want to check later that the service still holds the write (see The record). An Issue with an empty declarations half commits nothing and reports the current CSN.

Refused writes#

HTTP status describes whether the request could be processed. Once an Issue has been processed, the answer is 200, even if the write in it was refused, and the reason is in the Messages channel. A write is refused, and nothing in it is applied, when:

  • one of its effects is not permitted by the key's grants (one TERSE-SEC-001 line names the right and the path);
  • the engine refuses the declarations as written;
  • Expected-Csn does not match the namespace's current CSN.

A refused write spends no CSN and leaves no entry. Terse-Csn then reports the CSN the namespace was already at, so a client that compares it with the CSN it last saw can tell whether its write committed. Some warnings and diagnostics can accompany a write that did commit, so read the Messages channel as well as the status.

Some refusals carry their own status instead of 200: 413 for a body over the plan's request-size limit or a line over a content limit, 422 for a structural directive the service cannot carry out as written, 507 when the namespace is at the plan's namespace-size limit, and 409 for a write to a read-only namespace. Each of these bodies is one or more error: lines ending nothing was processed or nothing was committed. The status codes page lists them all.

Concurrent writers#

A namespace applies its writes one at a time, each against the state the previous one left. Two clients that write at the same time are both served, in turn. If you need a write to apply only to the state you last read, send Expected-Csn with that state's CSN. If another write landed in between, yours is refused in the Messages channel with status 200, and you can read again and decide.

If the namespace is briefly unavailable, the answer is 503 with a Retry-After header and nothing has been processed; retry the same request after that many seconds. A 503 with the code TERSE-STO-001 means other activity on the namespace prevented the commit; nothing was committed, and Terse-Csn reports the standing CSN.

Retries and idempotency keys#

A client that loses a response cannot tell from the network alone whether its write landed. Send a Terse-Idempotency-Key header (1 to 128 printable ASCII characters) with a fresh value for each logical write, and the same value on every retry of that write.

  • If a retry with the same key and the same body arrives while the original is among the namespace's most recent 1,000 log entries, nothing is applied again. The answer is 200 with the original write's CSN in Terse-Csn and the line info: duplicate of csn N; not re-applied.
  • The same key with a different body is refused with 422 and the code TERSE-IDM-001.
  • A key is scoped to the namespace and to the API key that sent it.

Two answers mean the write may or may not have landed. A 500 with TERSE-STO-003 means the commit failed at a point where the outcome is unknown; a 504 with TERSE-REQ-001 means the request did not answer in time, and the write runs to its outcome anyway. In both cases, resend with the same idempotency key, which is safe. Without a key, send a queries-only Issue, check whether the write is present, and if it is not, resend with Expected-Csn set to the CSN that Issue reported; if the first attempt lands in between, the resend is refused in the Messages channel.

Read-only namespaces#

A namespace can be marked read-only in its companion document. Queries are served as usual. A write is refused with 409 and the body error: namespace X is read-only (# Meta(readOnly) in its companion); its admin lifts it. A copy of a namespace starts read-only. See Keys, grants and visibility for who sees the 409.

Export#

GET /v1/ns/{ns}/export returns the namespace's state as one .terse attachment named <ns>-<csn>.terse. The content is the answer to a bare ? query, so a key receives in the file exactly what its grants let it read. The file begins with a comment line // TERSL export naming the namespace and CSN and ends with a comment line // TERSL export complete with counts; a file without the closing line was cut short. Both lines are comments, so the open-source tools read the file as plain state. To export an earlier moment, use the address form (7QK4M2C@4096) or the Terse-As-Of-Csn and Terse-As-Of headers.

Bulk load#

POST /v1/ns/{ns}/load accepts a TERSL file far larger than one Issue may carry. The service plans the file into batches and applies each batch as an ordinary write with its own CSN. Each batch commits as a unit, and if the load stops part-way, the batches before the stop stay committed. The answer is a 200 that starts at once and streams one line per phase and per batch, then a summary that ends with committed through line N.

A load can be a dry run that reports the plan and applies nothing, can close the namespace to other writes or to every other request while it runs, can stop or continue after a refused batch, and can resume after a given line of the file. With a Terse-Idempotency-Key, a resent load commits no batch twice. Only one load runs on a namespace at a time. The Terse-Load-* headers that control these choices are listed on Headers, and the portal's Bulk export and load page runs loads in a browser.