#477 Search engines get the pages of events with something of their own
–
- github
- github/openchronology/timeline-app
- github/openchronology/timeline-app#477
- pull request
- state:merged
Closes #467. Every event of a public timeline used to go in the sitemap and to IndexNow, so the synthetic incident timeline's 2,299 log lines were 24% of timescale.info's sitemap. Each timeline now chooses which of its events' pages search engines get, under **Sharing & settings** → **Search engines**: | Setting | Lists | |---|---| | **Automatically** (default) | Events with at least 150 characters of notes and a title no other event of the timeline shares. None when the timeline's notes are templated. | | **All of them** | Every event, as before | | **None** | Only the timeline's own page | **Templated notes** (`server/event-listing.mjs`): - **Sample:** up to 200 of a timeline's notes long enough to list, chosen by a hash of their IDs. - **Reduction:** each note becomes its first 12 words, without numbers, addresses, paths, `code` and quoted text. - **Verdict:** with at least 20 sampled notes, if half reduce to the same words, the timeline lists none automatically. - **Caching:** the verdict is kept per timeline revision, so nothing extra is stored or rebuilt on save. Checked against live timescale.info pages: 30 of 30 sampled incident notes reduce to the same words. Hand-written timelines top out at 2 of 30 (`paleontology-history`) and 1 of 30 (`rome`). So the incident timeline drops out automatically, without setting it to **None**. **What an unlisted event's page does:** - **Kept:** it keeps its page, and its timeline's page still links to it. - **Left out:** of the sitemap and of IndexNow. - **Robots:** its page says `noindex, follow`, so crawlers still follow its links to related events and its timeline. **Changes:** - **Database:** `oc_timelines.index_events` (`auto`, `all` or `none`). - **Setting it:** `indexEvents` in `PATCH /api/timelines/{id}/settings`, documented in OpenAPI. - **Store:** `listedEntityKeys` replaces `entityKeys` for the sitemap and IndexNow. `listedOf` filters a save's changed events for IndexNow, keeping removed ones so search engines hear they're gone. A timeline set to `none` is also left out by `entitiesListed`. - **Event pages:** `eventListed` decides their robots tag. - **Settings page:** a **Search engines** section with the three choices. - **Docs:** `docs/sharing.md#which-events-search-engines-list`. A model to judge this more finely is planned in #476. **Tests:** - **Unit:** - `event-listing.test.mjs`: note shapes and templated notes; - `entity-pages.test.mjs`: `noindex, follow` for an unlisted event. - **PostgreSQL:** - `sharing-postgres`: in the sitemap and in each page's robots tag, it checks **Automatically** (a rich event listed; a thin one and two sharing a title not), a templated timeline, **None**, **All of them**, and a refused value. Its HTTP checks pass here; the browser steps run in CI. - `indexnow-postgres`: on publishing and saving, only listed events are sent; with **None**, only the timeline's page. - The existing sample timelines in both tests have short notes, so they're set to **All of them**. - **Run locally, passing:** `indexnow-postgres`, `labels-postgres`, `timeline-links-postgres`, `relationships-postgres`, `next-postgres`, `administration-postgres`, `summaries-postgres`, and all unit tests except `contents.test.mjs`, which fails on `main` in this checkout as well. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_011fFY8eruEi1xAkkMBni13B https://github.com/openchronology/timeline-app/pull/477
Related
Open #477 Search engines get the pages of events with something of their own in the timeline