circagraph
Sign inSign up

openchronology/timeline-app on GitHub

The OpenChronology repository on GitHub, mirrored by och-import-github: issues and pull requests as durations (ongoing while open), what happened to them as moments, plus releases, milestones, commits and branches. The Branches view shows the most recently active branches each in a track of its own.

Source: https://github.com/openchronology/timeline-app

3,131 moments · 2026-10-05 CE 20:21:49 UTC – 2026-11-16 CE 12:10:45 UTC · by @athan · @athan

#295 Whole-timeline downloads are cached per revision

–

  • github
  • github/openchronology/timeline-app
  • github/openchronology/timeline-app#295
  • pull request
  • state:merged

Closes #204. The server now keeps each `.ochx`/`.och` file it makes for a timeline's revision and sends that file to the next download of the same revision, instead of building it from the database again. ## What's cached The editor's large-timeline export (`/api/timelines/{id}/export/{name}.ochx`), `.ochx` downloads with `?download=1` (a timeline's, or a saved revision's), and `.och` files (`/api/timelines/{id}/file`). Comparisons aren't cached. A save makes a new revision, so its next download is made fresh and kept in turn. Keys are ` :r : `, or `:s :` for a saved revision. ## How - **`server/export-cache.mjs`**, with entries in a new `oc_export_cache` table (idempotent `schema.sql`). - **First download:** sent as it's made, exactly as before, while it is written to a staged file beside it. The file is kept only if it is complete, under the limit, and of the revision asked for. A save that lands mid-build makes it newer, so it's sent but not kept. - **One build per key:** a build claims its row, and the claim works across servers. A download that arrives while this process is building waits for it, then gets the cached file. One arriving while another server is building sends its own copy without storing it. A claim abandoned for 15 minutes is taken over. - **Access:** checked on every download, cached or not, through `store.access`. A saved revision since forgotten (history turned off) is refused, via `store.revisionReadable`. - **Cached responses:** - `ETag` / `If-None-Match` answers 304, and single `Range` requests get 206 or 416. - `Content-Length` is set, with `Cache-Control: private, no-cache`. - This goes through both the Node server and the Next.js adapter, which now passes byte chunks through. - **Eviction:** least recently downloaded first, keeping the total within `EXPORT_CACHE_MAX_BYTES`. A file larger than the whole limit is sent but not kept. - **Removal:** - Deleting a timeline forgets its files at once. - A sweep (at the first download, then every 10 minutes on an unref'd timer) removes files of timelines deleted elsewhere (administration, account deletion), abandoned builds, and orphan folders on disk. - It also removes files handed out at the public address whose timeline is no longer public: made private, or restricted by moderators. Making a timeline private through its settings removes those files immediately. - **Public address:** with `EXPORT_CACHE_PUBLIC_URL` (a web server or CDN in front of the directory or bucket), public timelines' cached files redirect there (302). This applies to the export address and saved-revision downloads only, not `/file` or `document?download=1`, which scripts `fetch` and which would need CORS. Private timelines' files are never redirected. File names are 32 random hex digits plus the file name. - **S3:** a single PUT, a ranged GET and a DELETE, signed with Signature Version 4 in `node:crypto`, so there is no new dependency. The signer matches the AWS test suite's `get-vanilla` vector, and the backend works against a real S3 server (SeaweedFS). Path style is the default, with host style as an option. - **Also fixed:** `streamResponse` added a `close` listener on every backpressure wait and never removed it. Long downloads hit Node's MaxListeners warning. ## Configuration `.env.example`, the packaged env template, `compose.yml`, `docs/administration.md` ("Cached downloads") and `docs/docker.md`: | Setting | Meaning | | --- | --- | | `EXPORT_CACHE` | `disk`, `s3` or `off`. Unset, `disk` when a directory is known (`EXPORT_CACHE_DIR`, or systemd's `CACHE_DIRECTORY`), else off | | `EXPORT_CACHE_DIR` | the disk cache's directory (staging for S3) | | `EXPORT_CACHE_MAX_BYTES` | e.g. `10GB`; 2 GiB unset | | `EXPORT_CACHE_PUBLIC_URL` | a file server or CDN serving the cache's files | | `EXPORT_CACHE_S3_ENDPOINT`, `_REGION`, `_BUCKET`, `_PREFIX`, `_ACCESS_KEY_ID`, `_SECRET_ACCESS_KEY`, `_PATH_STYLE` | an S3-compatible bu… https://github.com/openchronology/timeline-app/pull/295

Related

Open #295 Whole-timeline downloads are cached per revision in the timeline

#295 Whole-timeline downloads are cached per revision