Commit Graph
12 Commits
Author SHA1 Message Date
Lauri Ojansivu bc79de8c57 Database health: a collection scan row names the command, the operators per field and the queryHash.
A row said only which fields a slow query filtered on, which is not enough
to find the query in the code or to run explain(). It now shows the command
(for a slow getMore, the query it continues, instead of 'filter on nothing'),
the operators used on each field - never values - and MongoDB's queryHash.

Thanks to Raymond-Voogt and xet7 !

Fixes #6856,
2026-11-02 11:31:47 +02:00
Lauri Ojansivu 0b5c855820 Snap backup and restore instructions work on AlmaLinux, Rocky Linux and Fedora.
snapd keeps the snaps in /var/lib/snapd/snap on Fedora, RHEL, AlmaLinux,
Rocky Linux, CentOS Stream and Arch Linux, which have no /snap, so the
instructions that exported /snap/wekan/current/bin and its lib directory
did not find mongodump and mongorestore there. They now pick the directory
with one S= line and the CPU's library directory with $(uname -m), and the
scripts that can be run from a host shell find the snap in either place.

Thanks to Group-Deed and xet7 !
2026-10-30 23:13:50 +02:00
Lauri Ojansivu b87858baaf Problems / Database: report queries without an index and a high read rate.
The database health probe (server/lib/databaseHealth.js) saw restarts, slow
storage and a full disk, but not what made one instance slow: a 10 GB
collection scanned three times per search, and an hourly job reading all
1.1 million cards (#6824, #6825). Both were visible only in MongoDB's own
log, as "Slow query" lines with COLLSCAN, and in serverStatus: 1.6 billion
documents read in two hours.

Two more checks, every 5 minutes with the others:

- db.collection-scan: the "Slow query" lines MongoDB keeps (getLog
  'global'), newer than the last probe, where a query read a whole
  collection of at least 10,000 documents. Grouped by collection and the
  field names filtered on - never values - with count, documents read and
  time. Small tables read whole by design (settings, roles) and internal
  namespaces are left out. Needs the getLog privilege; without it, and on
  FerretDB, the check finds nothing and is skipped.
- db.scan-rate: serverStatus.metrics.queryExecutor.scannedObjects between
  two probes above 50,000 documents per second. A database restart, which
  resets the counter, is not counted.

Both are rows in Admin Panel / Problems / Database with what to do;
docs/Databases/MongoDB/Storage-Requirements.md lists them.

tests/databaseHealthScans.test.cjs drives the log parsing and the rate,
including that a filter's values never reach the row. From the reporter's
patch.

Thanks to Raymond-Voogt and xet7 !

Fixes #6826.
2026-10-27 21:19:51 +02:00
Lauri Ojansivu 11babe27f2 Docs: snap keys, setting guide and Docker examples that match this WeKan.
- Three snap set lines named keys the snap does not have, which snapd
  stores without a word: MONGO_OPLOG_URL (the key is mongo-oplog-url),
  oauth-enabled (oauth2-enabled) and "snap set wekan set disable-mongodb".
  tests/docsSnapKeysExist.test.cjs checks every snap set in the docs
  against the KEY_ names in snap-src/bin/config.
- Adding-new-Snap-settings-to-code.md used setting names that do not
  exist (LDAP_ENABLED, LDAP_SEARCH_FILTER, a wrong KEY_ line), the edge
  branch and Travis CI. It now follows the real config and wekan-help
  entries, says that a login setting goes through the Admin Panel catalog
  and authEnv(), and lists the other platform files a setting belongs in.
- The LDAP and Traefik Docker examples ran MongoDB 3.2/4.0 with
  --smallfiles, which MongoDB 4.2 removed, and WeKan images from 2018. They
  use mongo:7 and ghcr.io/wekan/wekan:latest; Traefik stays at v1.7, since
  that guide's labels and traefik.toml are v1 syntax.
- The Windows page no longer sets RICHER_CARD_EDITOR, which WeKan does not
  read, and points to build.bat for building from source.

Thanks to xet7 !
2026-10-16 23:52:46 +03:00
Lauri Ojansivu b7589737f5 File docs/Features the way the WeKan menus are.
A feature's guide now sits under the menu it is used from, read from the
templates rather than from where somebody once put it: Header-Bar,
All-Boards, Pages, Board-Header (search and every Board View), Board-Sidebar
(Board Settings, Filter, Members, Labels, Archive), Rules, Swimlanes, Lists,
Cards, Member-Settings, Admin-Panel and Login-Page. Topics used from several
menus (editor, history, theme, translations, email, troubleshooting) are in
General.

- Each area has a README mapping its menu entries to their guides and to the
  template or server module that implements them. Board-Header/README.md
  lists all 40 Board View entries with the template boardBody.jade renders;
  five views (Table, Calendar Mode, Time, Timeline, Map) have no guide of
  their own yet and say so.
- Board-View.md listed 25 views while BOARD_VIEWS has 40; it now lists all
  of them with their English labels.
- Pages keep their file names. Where a released changelog links an old path,
  a "# Moved" page stays there and points to the new one; test-menu.sh skips
  folders holding only those.
- Undo was filed under Login; it is General/Undo now.
- Every link in docs, code comments, scripts and tests follows the move;
  docsLinksResolve checks them all.

Tests that read a doc follow it to its new place. boardViewSettings now fails
when Board-View.md or Board-Header/README.md differs from BOARD_VIEWS, or
names a template boardBody.jade does not render.

Thanks to xet7 !
2026-10-16 23:45:35 +03:00
Lauri Ojansivu fea58e1312 Document which databases WeKan supports, and how each is tested.
docs/Databases/Supported.md is the one list: FerretDB v1 with SQLite (the
default everywhere), PostgreSQL (works), MySQL, MariaDB and SAP HANA
(experimental), FerretDB 2 (not tested), and MongoDB 6, 7, 8 and 9 - with
what checks each: the 110-query conformance catalogue answers the same on
all of them, and WeKan's own tests run on Meteor's bundled MongoDB 8 and on
FerretDB. It also says what to know about MongoDB versions: the Linux
kernel 6.19-7.0.13 refusal and its fixes, AVX, the driver, and upgrades.

The README and the Status section of CHANGELOG.md gain a Databases list,
docs/Databases/README.md links the page, the Meteor 2.14 compatibility guide
is marked historical, and the conformance page counts 110 cases and the
MongoDB backends. CLAUDE.md and AGENTS.md describe the new Status block.

Thanks to xet7 !
2026-10-10 19:07:14 +03:00
Lauri Ojansivu faaa83ed09 Explain and remediate the MongoDB 8.2 crash loop that follows a full disk.
A reported test environment (mongo:8.2.2 in Kubernetes) filled its data
volume: the checkpoint's fdatasync returned ENOSPC, WiredTiger panicked and
mongod aborted - and then it kept failing on every start, with the disk long
since freed. MongoDB 8.2 keeps a throwaway WiredTiger instance under
<dbPath>/_tmp/spilldb for queries that spill to disk and empties it itself on
each start; after the abort that emptying failed ("Failed to clear dbpath of
the internal WiredTiger instance: Directory not empty"), mongod opened the
half-emptied directory, found no version file, reported "Failed to open the
spill WiredTiger instance ... database corruption detected" and fasserted.
Nothing retries, so the pod restarts forever. The data itself is intact;
deleting that one directory is the whole fix.

- snap-src/bin/mongodb-control deletes _tmp/spilldb before every mongod
  start (no mongod is running at that point, and mongod recreates it), and
  its start-failure handler recognises the log line and names the directory.
  The snap ships MongoDB 7 today, so this is a no-op until it does not.
- Admin Panel / Problems: the db.restart row now says what a "No space left
  on device" abort means for a full disk as well as for a network
  filesystem, and both it and db.disk-space name the exact spill directory
  to delete when MongoDB 8.2 will not start afterwards. The probe looks the
  data directory up once and passes it to both rows.
- docs/Databases/MongoDB/Storage-Requirements.md gets the log signature of
  both stages, why the second one never ends, the one command, what must
  not be touched, and that Docker/Kubernetes users run it themselves since
  the database container is MongoDB's own image.
- tests/databaseHealth.test.cjs pins the spill path, both rows' advice, the
  probe wiring, the snap's pre-start deletion (before the first start, after
  the running-instance check, and the only recursive delete apart from the
  existing FerretDB reset) and the documentation.

Thanks to xet7 !
2026-09-11 21:31:10 +03:00
Lauri Ojansivu 03ffb56a5e Index card_comment_reactions, and document why MongoDB's data directory must be local.
From a reported MongoDB crash series (.tools/crash: WeKan and MongoDB in one
Kubernetes pod, the database volume on an SMB/DFS share):

- The crash itself is WiredTiger's checkpoint fsync() returning ENOSPC
  ("No space left on device" with 519 MB used and 10 GiB free) on the
  network share, followed by a fatal assertion and abort - every checkpoint,
  so a restart loop. That is the storage, not WeKan: MongoDB requires a
  local filesystem with real fsync semantics under its data directory.
  docs/Databases/MongoDB/Storage-Requirements.md now says so, shows what
  the log looks like when it is not, and what to do (local block volume;
  attachments may still live on a share; MONGO_URL over TCP is fine).
- The WeKan-side finding in the same logs: card_comment_reactions had no
  index at all, and every board open queried it by `cardId: { $in: [...] }`
  and by `boardId` as a full collection scan - 3,397 COLLSCANs of that one
  collection in the last crash log, up to 1.6 s each, the bulk of its
  "Slow query" lines. server/models/cardCommentReactions.js now ensures
  indexes on cardId, boardId and cardCommentId at startup, like every other
  per-card collection.

tests/cardCommentReactionsIndexes.test.cjs pins both.

Thanks to xet7 !
2026-09-11 11:23:53 +03:00
Lauri Ojansivu c4cb87345b Reorganize documentation and repair links.
Move feature and webserver documentation into the reorganized hierarchy, update references to their new relative paths, and extend link validation across the complete documentation tree.\n\nThanks to xet7 !
2026-08-17 00:24:38 +03:00
Lauri Ojansivu fa289e4a4d Admin Panel Problems: the route and the template say what the menu says.
The pane is called Problems in the menu, its address is /admin/problems, and
docs/Features/Admin-Panel/Problems is where it is documented. Two things still
called it Reports, from before it was renamed:

  * the route NAME was 'admin-reports', where its three siblings are 'setting',
    'people' and 'attachments' - the bare page key. So the one route whose name
    did not match its own address;
  * the template, and its three files, were adminReports.

Both are 'problems' / adminProblems now, across 41 files.

What did NOT change is `legacyBase: '/admin-reports'`. That is an address people
have bookmarked and it still redirects here - the point of the rename is that
the code agrees with the menu, not that an old link should break. The CHANGELOG
entries describing the old addresses are left alone too: a released section is a
record of what shipped.

Thanks to xet7 !
2026-08-16 15:36:04 +03:00
Lauri Ojansivu f628a69889 Reorganized docs Platforms etc.
Thanks to xet7 !
2026-08-03 13:07:12 +03:00
Lauri Ojansivu fea3c432e8 Organize docs/Databases: one directory per database.
It was a flat list where the database was a filename prefix, in three spellings
- MongoDB-Driver-System.md, mongodb-avx-qemu.md, MongoDB_OpLog_Enablement.md -
plus FerretDB2-PostgreSQL.md and a ToroDB-PostgreSQL directory. Somebody asking
"how do I run WeKan on PostgreSQL" had to read the prefixes to find out which
files were even about their database.

Now: Migrations, MongoDB, FerretDB (with 1 and 2, because v1 and v2 are
different products with different backends) and ToroDB (with PostgreSQL). Each
file keeps its name minus the redundant prefix - MongoDB/Driver-System.md,
MongoDB/avx-qemu.md - and the old PostgreSQL.md, which was an index of FerretDB
and ToroDB rather than a document about PostgreSQL, becomes the directory's
README.md.

Every directory has a README.md saying what is in it and where to go next,
including FerretDB/1, which had no documentation at all until now although it
is WeKan's default database and just gained four more backends. Links to a
directory point at the DIRECTORY, not at its README.md, because README.md is
what is opened by default.

Every link that pointed into the old paths is updated - in docs, in
docker-compose-ferretdb-v2-postgresql.yml, and in the wekan.fi website - and
the relative links inside the moved files, which gained a directory level and
so needed one more "../". tests/docsDatabases.test.cjs pins the layout, the
READMEs, the directory links, that every relative link resolves, and that
nothing outside CHANGELOG.md still names an old path.

Thanks to xet7 !
2026-07-28 01:05:27 +03:00