circagraph
Sign inSign up

openchronology/timeline-app on GitHub

The OpenChronology repository on GitHub, mirrored by och-import-github: issues and pull requests as durations (ongoing while open), what happened to them as moments, plus releases, milestones, commits and branches. The Branches view shows the most recently active branches each in a track of its own.

Source: https://github.com/openchronology/timeline-app

3,131 moments · 2026-10-05 CE 20:21:49 UTC – 2026-11-16 CE 12:10:45 UTC · by @athan · @athan

#173 Follow the systemd journal into timelines; name journald imports by their cursor

–

  • github
  • github/openchronology/timeline-app
  • github/openchronology/timeline-app#173
  • pull request
  • state:merged

Closes #151. ## The question in the issue: would syslog give the same data as journalctl? Mostly not. Forwarding the journal to syslog (`ForwardToSyslog=`, or rsyslog's `imjournal`) carries the message text, its time (to the second for RFC 3164), severity, facility, host, program and PID. It loses: - the systemd unit (`forgejo.service`); - microsecond times; - everything else the journal keeps: user, command line, executable, boot ID, source file and line, and the program's own structured fields. Its moments also aren't the ones `och-import-logs --format journald` makes, so a range covered by both would appear twice. So this adds a live journald follower. It gives the same data as the historic `journalctl -o json` import, as the same moments. `docs/journald.md` has the full comparison table. ## The journald follower `server/journald-receiver.mjs` (`npm run journald`; the `openchronology-journald` service in the Debian and RPM packages): - **Reads** `journalctl --follow --output=json` on this machine, or on another over SSH (`JOURNALD_HOST`, `JOURNALD_SSH_KEY`, with a `known_hosts` kept beside its cursor). Arguments are quoted for the remote shell, so cursors containing `;` survive. - **Narrows** what it reads with `JOURNALD_UNITS` and `JOURNALD_MATCHES`. - **Routes** entries with `JOURNALD_TIMELINE`, or with `JOURNALD_ROUTES` rules like the importer's, matching on `unit`, `app`, `host`, `severity` or any journal field. - **Makes the same moments as the importer**, using the journald format and mapping from `log-formats.mjs` and `log-mapping.mjs`: - title: the message; - tags: `journald`, `severity:`, `host:`, `app:`, `unit:`; - metadata: every field. - **Resumes after restarts.** Once everything read has been sent, it keeps the last entry's cursor; on restart it runs `journalctl --after-cursor`. Nothing written while it was down is missed and nothing is sent twice; a crash at worst re-reads entries with the same IDs, which only overwrite. - **Survives journalctl stopping** (rotation, an SSH drop): it restarts journalctl after the last entry read, backing off up to a minute on repeated quick failures. - **Shares delivery with the syslog receiver.** The queues, batching, retries and draining move from `SyslogIngest` into `server/moment-ingest.mjs`, and both receivers now extend it. Syslog behaviour is unchanged (its tests and the live syslog test pass). ## A fix to the journald importer While testing this I found that **journalctl prints an entry's fields in a different order on every run**: the same entry gave three different lines in three runs. The importer named moments by a hash of the line, so re-importing an overlapping journald range (for example Forgejo history) **added duplicates** instead of overwriting. Journald entries are now named by their `__CURSOR`, through a new `identity` on formats (`compileFormat(...).identity`); every other format keeps hashing the line. One limit: moments imported before this fix used the old names, so re-importing a range imported before this release would add those entries once more. ## Packaging **Debian and RPM** install: - `openchronology-journald.service`: runs as `openchronology` with `SupplementaryGroups=systemd-journal` and `StateDirectory=openchronology-journald` for its cursor; hardened like the syslog unit; - `journald.env`, from `/usr/share/openchronology/journald.env.example`; - prerm and spec stop/disable entries. **Not packaged elsewhere:** FreeBSD, macOS and Windows have no journald, so they don't get it. **Package smoke tests:** `server-deb` and `server-rpm` check that the unit and configuration example are installed, and that the follower loads with the bundled Node.js. ## Tests - **New:** `test/journald.test.mjs`, with a fake API and a stand-in journalctl, covers: - the command, including SSH quoting and key options; - moments and routing by unit; - the same ID for an entry whose fields are reordered; - the cursor kept and resumed from; - ……

Related

Open #173 Follow the systemd journal into timelines; name journald imports by their cursor in the timeline

#173 Follow the systemd journal into timelines; name…