circagraph
Sign inSign up

openchronology/timeline-app on GitHub

The OpenChronology repository on GitHub, mirrored by och-import-github: issues and pull requests as durations (ongoing while open), what happened to them as moments, plus releases, milestones, commits and branches. The Branches view shows the most recently active branches each in a track of its own.

Source: https://github.com/openchronology/timeline-app

3,131 moments · 2026-10-05 CE 20:21:49 UTC – 2026-11-16 CE 12:10:45 UTC · by @athan · @athan

#154 Make saves follow the nodes they touch, and timeline creation linear

–

  • github
  • github/openchronology/timeline-app
  • github/openchronology/timeline-app#154
  • pull request
  • state:merged

Step 1 of #135: saves now scale with the nodes they touch, and a timeline is created in linear time. ## What changed **Tree path query** (`MOMENT_TREE.path` and `INTERVAL_TREE.path` in `server/avl.mjs`). Before a save changes anything, it reads the nodes it needs in one query. - **Before:** the query walked from the root once per key, deduplicated `(key, id)` pairs on every step, and parsed each key from text at every step. - **Now:** the keys are cast once and descend together. Each node splits the keys that reach it between its children (`ARRAY(... WHERE k n.time`), so each node on the paths is read once. - **Why it matters:** appended log moments share nearly their whole path. A 5,000-moment append now reads a few dozen nodes, instead of 5,000 × depth lookups. - **Comparisons:** tree comparisons parse each node's and key's time once. Parsing rationals was the largest CPU cost of a save. **Creating a timeline was quadratic.** Starting saved history (`copyKeys` in `history.mjs`, run when a timeline is created) copied each page by looking every entity up again by key. - Durations are looked up by an expression (`band->>'id'`), and the planner doesn't use its index before a new timeline's statistics count its rows. Every lookup read every duration. - At 200,000 moments that one statement took up to 144 s. At a million, creation took about 96 minutes. - Each page is now copied by the statement that reads it (`copyAll`), in key order through an index. **Adaptive batches in the log writers** (`TimelineWriter.send`, used by `och-import-logs` and the syslog receiver). - Batches are split to a size that adapts per timeline: - a save taking 8 s or more halves the next ones, down to 64 entries; - so does one the server stopped as too slow, which is sent again, halved; - quick full saves let them grow back, up to 5,000. - Moments go before durations, which may name them. Resending a part is harmless: saves are upserts by ID. - The server now answers a statement timeout (`57014`) with **503** and a clear message, instead of a generic 500. The writers already treat 503 as temporary. **Benchmark:** `bench/save-size.mjs [sizes] [runs]` times creation, then appends of 10, 1,000 and 5,000 moments and a 1,000-moment change spread through the timeline, at each size. ## Measurements 8-core VM, PostgreSQL 17, benchmark timelines (`bench/save-size.mjs`, median of 3): | | `main` | This PR | |---|--:|--:| | Create 200,000 moments | 283 s | 126 s | | Append 10 / 1,000 / 5,000 at 200,000 | 0.08 / 1.73 / 4.04 s | 0.11 / 0.91 / 2.49 s | | Change 1,000 spread at 200,000 | 0.79 s | 0.56 s | | Create 1,000,000 moments | ~96 min (earlier measurement) | 636 s | | Append 10 / 1,000 / 5,000 at 1,000,000 | 5,000: 21–25 s (in #135) | 0.24 / 1.35 / 5.22 s | | Change 1,000 spread at 1,000,000 | | 0.75 s | The largest batch at a million moments is now well under the 30 s statement timeout. Creation queries went from 45 s at 200k to 111 s at 400k, close to linear. ## Tests - **New `test/tree-paths-postgres.mjs`** (in CI): after one prefetch, walking to every key (and every child on the way) needs no further query. It covers moments and durations, on a timeline and on a fork reading through it, with: - existing and new times, shared times, and keys before and after everything; - appended keys, which must read under 60 nodes. Prefetching only fills a cache, so the other tests can't see a path query that misses nodes. Swapping the split in either tree's query fails this test; both fault checks were run. - **`test/syslog.test.mjs`:** the writer splits over-limit batches with durations last, halves after slow saves down to the minimum, grows after fast ones, keeps limits per timeline, and resends a 503'd save halved, failing at the minimum. - **Other checks:** - all 34 database suites pass locally, apart from the two browser suites this VM can't run; - 258 unit tests pass; - the build passes. ## Left for later in #135 - **C……

Related

Open #154 Make saves follow the nodes they touch, and timeline creation linear in the timeline

#154 Make saves follow the nodes they touch, and timeline…