circagraph
Sign inSign up

openchronology/timeline-app on GitHub

The OpenChronology repository on GitHub, mirrored by och-import-github: issues and pull requests as durations (ongoing while open), what happened to them as moments, plus releases, milestones, commits and branches. The Branches view shows the most recently active branches each in a track of its own.

Source: https://github.com/openchronology/timeline-app

3,131 moments · 2026-10-05 CE 20:21:49 UTC – 2026-11-16 CE 12:10:45 UTC · by @athan · @athan

#202 och-convert's memory and file-size limits stop large timelines with real metadata

–

  • github
  • github/openchronology/timeline-app
  • github/openchronology/timeline-app#202
  • issue
  • state:closed

Found while fixing #199 (PR #201). With the overview index loaded in bulk, `och-convert write` now writes 200,000 moments in about 9 s, well within its 20 s CPU limit. The other two limits it runs under (`resource_limits` in `native-store/src/bin/och-convert.rs`, Linux) are now the ones a large timeline reaches first: - **Address space, 512 MiB (`RLIMIT_AS`):** at 200,000 moments with a one-word title each, the process peaks at about 400 MB, roughly 1.8 KB per moment. Most of it is the parsed JSON document: `save()` holds the whole `Document`, every moment's metadata as a `serde_json::Value`, before writing anything. - **Output file, 32 MiB (`RLIMIT_FSIZE`):** the same timeline makes a 27.9 MB `.och` (from 13 MB of JSON). Larger metadata pushes it over. Measured on a release build: | Moments | JSON in | .och out | Peak memory | | --- | --- | --- | --- | | 30,000 | ~2 MB | — | 62 MB | | 200,000 | 13 MB | 27.9 MB | ~400 MB | So timelines below the format's 200,000-moment maximum fail once their moments carry real metadata: descriptions, tags, sources. Options: - Parse and insert moments as a stream (e.g. `serde_json`'s `StreamDeserializer` over `events`), so memory no longer grows with the document. - Size the file limit from the input, or from the format's own maximums, instead of a fixed 32 MiB. - Say which limit was hit. (Not checked yet: what message a conversion stopped by `RLIMIT_AS` or `RLIMIT_FSIZE` gives today.) https://github.com/openchronology/timeline-app/issues/202

Related

Open #202 och-convert's memory and file-size limits stop large timelines with real metadata in the timeline

#202 och-convert's memory and file-size limits stop large…