app/board-open.cjs seeds a board of 200, 2000 or 5000 cards, opens it in Chromium and
records the wall clock to the first card and until the DDP WebSocket is quiet, what
crossed it, the Meteor server's CPU by who spends it (Node inspector) and FerretDB's
CPU (pprof). Two runs on each database: the Meteor server's CPU is the same on both
(0.1-0.5 s) and the DDP server is 1% of it; a 5000-card board takes FerretDB (SQLite)
400-600 ms longer to open, the data sent to the browser being the same ~210 KB, a
small board 70-100 ms. FerretDB uses about 4 s of CPU for the 5000-card open: SQLite
39%, filtering in Go 34%, find/getMore/aggregate 0.7-0.8 s each. The README says what
was measured and what was not: the development build, one machine, the browser's own
time not separated, and the aggregate's source not identified.
Thanks to Raymond-Voogt and xet7 !
minimatch before 11.0.1 (GHSA-g3hv-75wg-m249): npm-packages/exceljs still had
10.2.6 through readdir-glob 3, and the vendored jade lockfile under
meteor-jade-loader had three copies through glob 13 and test-exclude; both get
an override to minimatch 11, as the root lockfile has. A test now reads every
package-lock.json of the repository, not only the reported ones.
The benchmark module of embedded engines pulled klauspost/compress 1.18.0,
golang.org/x/net 0.23.0 and golang/glog 1.0.0 (GHSA-259r-337f-4rfw,
GHSA-5cv4-jp36-h3mw, GHSA-qxp5-gwg8-xv66, GHSA-vvgc-356p-c3xw,
GHSA-6wxm-mpqj-6jpf). They are raised to 1.18.7, 0.62.0 and 1.2.4, and
govulncheck then found more in x/net, x/text and OpenTelemetry, which are raised
too (x/net 0.62.0, x/text 0.43.0, otel 1.42.0): it now reports none. A test
holds every go.mod of the repository at or above these floors.
Thanks to dependabot and xet7 !
A benchmark of its own (docs/Databases/FerretDB/1/SpeedTest/embedded, a Go module,
no cgo) loads the speed test's cards into SQLite set up as FerretDB sets it and
into Pebble, Badger, bbolt, BuntDB, Nutsdb and Clover, and asks each the six
questions WeKan asks of cards, at 15,377 and 153,768 cards. A board's cards take
the same time on every engine that is not slower, because decoding the JSON
dominates; the key-value stores win single reads and writes by microseconds that
a FerretDB call hides; Pebble is 12x slower on fetching a board, Clover 90-190x
slower in total, and bbolt with its default fsync 100-160x slower on writes. The
README accounts for every entry of the awesome-go section.
Thanks to Raymond-Voogt and xet7 !
Dolt runs under the existing MySQL handler with no new handler and gives the same
answers, but is by far the slowest: 20,470 ms in total, every card query a
2.8 s whole-table scan. The VersionHistory scripts drive WeKan-shaped changes
through FerretDB onto Dolt (a commit per write, undo and redo with DOLT_REVERT,
AS OF, +0.4 ms a write, a conflict when the same document is restored again) and
through the REST API of TerminusDB (author and message per commit, field-level
restore). ClickHouse and StarRocks were tried with the MySQL handler and need a
handler of their own; the README says where each fails.
Thanks to Raymond-Voogt and xet7 !
TiDB (MySQL protocol, Apache-2.0) runs FerretDB after its metadata table's
generated columns became VIRTUAL, and gives the same answers as MongoDB; it is
a speed test row and opt-in for the conformance run (WEKAN_CONFORMANCE_TIDB=1).
Noria has no JSON type, triggers or generated columns, which the MySQL handler
needs; YugabyteDB refuses a CREATE INDEX on a jsonb expression, which every
FerretDB v1 PostgreSQL index is; CockroachDB is not open source. The README
also records that analyzing a PostgreSQL table after its indexes are created
took the PostgreSQL row from 454 to 145 ms, and that the same on MySQL and
MariaDB did nothing.
Thanks to Raymond-Voogt and xet7 !
Results.md now opens with an At a glance table (rank, total, how many times
slower than the fastest, the same for the slowest user), every table is ordered
by its own numbers, the header says the real data size and the real operating
system, and the load table is ordered by the whole load time. pgrust
(PostgreSQL in Rust, malisper/pgrust:v0.4-beta) is one more row through the
same PostgreSQL handler; in the conformance run it is opt-in with
WEKAN_CONFORMANCE_PGRUST=1. The run is 1/20 of a production WeKan so it takes
minutes.
Thanks to Raymond-Voogt and xet7 !
The ten newest of 30,116 users took 52 ms with an ORDER BY in the database and
38-49 ms decoded and sorted in Go, so the change was not kept; the README says
what was measured so it is not repeated.
Thanks to Raymond-Voogt and xet7 !
SQLite sorts and stores JSONB, PostgreSQL uses a GIN index for membership and
pushes down $or, $elemMatch and $and, MySQL and MariaDB read through their
indexes with multikey side tables, and WeKan stopped scanning whole collections
every hour or at every start. Results.md still shows v1.99.0, which the README
now says, and the open items are updated to match.
Thanks to Raymond-Voogt and xet7 !
A row said only which fields a slow query filtered on, which is not enough
to find the query in the code or to run explain(). It now shows the command
(for a slow getMore, the query it continues, instead of 'filter on nothing'),
the operators used on each field - never values - and MongoDB's queryHash.
Thanks to Raymond-Voogt and xet7 !
Fixes#6856,
snapd keeps the snaps in /var/lib/snapd/snap on Fedora, RHEL, AlmaLinux,
Rocky Linux, CentOS Stream and Arch Linux, which have no /snap, so the
instructions that exported /snap/wekan/current/bin and its lib directory
did not find mongodump and mongorestore there. They now pick the directory
with one S= line and the CPU's library directory with $(uname -m), and the
scripts that can be run from a host shell find the snap in either place.
Thanks to Group-Deed and xet7 !
The database health probe (server/lib/databaseHealth.js) saw restarts, slow
storage and a full disk, but not what made one instance slow: a 10 GB
collection scanned three times per search, and an hourly job reading all
1.1 million cards (#6824, #6825). Both were visible only in MongoDB's own
log, as "Slow query" lines with COLLSCAN, and in serverStatus: 1.6 billion
documents read in two hours.
Two more checks, every 5 minutes with the others:
- db.collection-scan: the "Slow query" lines MongoDB keeps (getLog
'global'), newer than the last probe, where a query read a whole
collection of at least 10,000 documents. Grouped by collection and the
field names filtered on - never values - with count, documents read and
time. Small tables read whole by design (settings, roles) and internal
namespaces are left out. Needs the getLog privilege; without it, and on
FerretDB, the check finds nothing and is skipped.
- db.scan-rate: serverStatus.metrics.queryExecutor.scannedObjects between
two probes above 50,000 documents per second. A database restart, which
resets the counter, is not counted.
Both are rows in Admin Panel / Problems / Database with what to do;
docs/Databases/MongoDB/Storage-Requirements.md lists them.
tests/databaseHealthScans.test.cjs drives the log parsing and the rate,
including that a filter's values never reach the row. From the reporter's
patch.
Thanks to Raymond-Voogt and xet7 !
Fixes#6826.
CodeQL js/incomplete-sanitization, alerts #593, #594 and #595:
docs/Databases/FerretDB/1/SpeedTest/report.cjs escaped `|` as `\|` in
Markdown table cells but left a backslash alone, so a value ending in a
backslash turned the added one into a literal backslash and its pipe
split the cell. One mdCell() helper escapes the backslash first, then the
pipe, and turns a line break into a space; it is also used for the
database's name and the query's title, which were not escaped at all.
tests/speedTestReportCells.test.cjs checks the escaping, that no value
can add a column to its row, and that nothing in the tree escapes a pipe
without the backslash. The report is generated from local speed test
results at build time, so there is no attempt to record in Admin Panel /
Problems.
Thanks to GitHub CodeQL and xet7 !
Rerun of the four FerretDB v1 backends at twice the production data,
with the MySQL/MariaDB index reads and VIRTUAL index columns, the
multikey side tables for MySQL/MariaDB, the PostgreSQL GIN membership
index and SQLite sorting in the database. MongoDB 6-9 and FerretDB 2
keep their rows from the same directory. Every database returned the
same data as MongoDB.
All Boards count went from 8-9 s to 159 ms on PostgreSQL and from
9.4 s to 1.1 s on MySQL; adding the 44 indexes took 172 s on MySQL and
901 s on MariaDB instead of 20 and 40 minutes. This build is before
the fixes for SQLite's People page (the database sort read its key
three times per row), MySQL/MariaDB All Boards with a starred board, and
Due Cards' $and on PostgreSQL, MySQL and MariaDB, which a rerun now in
progress measures.
Thanks to xet7 !
docs/Databases/FerretDB/1/SpeedTest/speedtest.sh (build.sh and build.bat ->
Tests -> Database speed test) is releases/db-conformance.sh with
WEKAN_DB_SUITE=speedtest: the same FerretDB built from source and the same
databases started the same way - SQLite, PostgreSQL, MySQL and MariaDB through
FerretDB v1, MongoDB 6, 7, 8 and 9, and now FerretDB 2 with PostgreSQL and
DocumentDB.
data.cjs generates, deterministically, twice the users, boards, lists,
swimlanes and cards of a production instance (615,070 cards), skewed the way
real ones are; indexes.cjs creates the indexes WeKan creates, read from its
ensureIndex calls; queries.cjs asks 18 common WeKan queries as WeKan's own code
builds them; report.cjs writes Results.md - a row per database, fastest at the
top, a column per query - and fails when a database answered or stored
anything differently from MongoDB. README.md describes it, what it found, what
was improved and what could still be.
db-conformance.sh: WEKAN_DB_BACKENDS runs only the named databases (to repeat
one into an earlier run's WEKAN_LOGDIR), containers are removed with their
volumes, SQLite's data is under .tools/tmp, and FerretDB 2 is a backend of the
speed test, and of the conformance run with WEKAN_CONFORMANCE_FERRETDB2=1. The
conformance catalogue gains a case of writes by string _id and _ids differing
only by case, which found that FerretDB on MariaDB 12.3 lost them.
tests/dbSpeedTest.test.cjs checks the data, the index list, the queries, the
report's order and that a difference fails it, the wiring and the Due Cards
board scope.
Thanks to xet7 !
wekan/FerretDB now releases ferretdb-sqlite-<arch> beside ferretdb-<arch>: the
same FerretDB with only the SQLite handler, about 5 MB smaller, and SQLite is
all a WeKan bundle, image, snap or .spk runs. releases/ferretdb-asset.sh is the
one place that chooses: the newest release that has ferretdb-sqlite-<arch>,
and, until one does, the full ferretdb-<arch> of the newest release, which
has the SQLite handler too. It says in the log which one it took and why,
downloads it, verifies it against its .sha256sum, and returns FERRET_ASSET,
FERRET_TAG, FERRET_URL and FERRET_SHA for the provenance table (also through
$GITHUB_ENV, cached per job so the check and embed steps agree).
Every FerretDB download goes through it: the amd64, arm64, win64, win-arm64,
win32, mac-arm64, mac-x64 and extra-architecture jobs of release-all.yml (the
Docker images, snap, AppImage, Flatpak, single-file and macOS/Windows
packages are made from those bundles), repack-bundle-for-arch.sh for
release-all-missing.yml, bsd.yml, check-arch-binaries.sh,
build-release-bundle.sh and test-download-urls.sh. require-binaries.sh checks
the resume-login floor against the release the helper chose. The provenance
rows record that release and URL instead of /latest/.
No release workflow builds FerretDB: sandstorm-src/build-deps.sh takes the
prebuilt ferretdb-sqlite-amd64 by default (FERRETDB_SOURCE=download), and
sandstorm.yml and release-all.yml's build-sandstorm no longer set
FERRETDB_SOURCE=build; building from source stays a local option. The default
docker-compose.yml downloads ferretdb-sqlite-<arch> (FERRETDB_BINARY, falling
back to ferretdb-<arch> for a release without it); the PostgreSQL, MySQL,
MariaDB and SAP HANA compose files keep the full binary.
tests/ferretdbSqliteAsset.test.cjs drives the helper against a fake GitHub
(newest, older and no release with the SQLite-only binary, .exe names, a
pinned tag, FERRETDB_VARIANT=full, checksum match and mismatch, a missing
platform, bad names) and checks that every download site uses it, that no
workflow builds FerretDB and that only the SQLite compose file takes the
SQLite-only binary. The tests that pinned the old URLs follow the helper.
Thanks to xet7 !
- Three snap set lines named keys the snap does not have, which snapd
stores without a word: MONGO_OPLOG_URL (the key is mongo-oplog-url),
oauth-enabled (oauth2-enabled) and "snap set wekan set disable-mongodb".
tests/docsSnapKeysExist.test.cjs checks every snap set in the docs
against the KEY_ names in snap-src/bin/config.
- Adding-new-Snap-settings-to-code.md used setting names that do not
exist (LDAP_ENABLED, LDAP_SEARCH_FILTER, a wrong KEY_ line), the edge
branch and Travis CI. It now follows the real config and wekan-help
entries, says that a login setting goes through the Admin Panel catalog
and authEnv(), and lists the other platform files a setting belongs in.
- The LDAP and Traefik Docker examples ran MongoDB 3.2/4.0 with
--smallfiles, which MongoDB 4.2 removed, and WeKan images from 2018. They
use mongo:7 and ghcr.io/wekan/wekan:latest; Traefik stays at v1.7, since
that guide's labels and traefik.toml are v1 syntax.
- The Windows page no longer sets RICHER_CARD_EDITOR, which WeKan does not
read, and points to build.bat for building from source.
Thanks to xet7 !
A feature's guide now sits under the menu it is used from, read from the
templates rather than from where somebody once put it: Header-Bar,
All-Boards, Pages, Board-Header (search and every Board View), Board-Sidebar
(Board Settings, Filter, Members, Labels, Archive), Rules, Swimlanes, Lists,
Cards, Member-Settings, Admin-Panel and Login-Page. Topics used from several
menus (editor, history, theme, translations, email, troubleshooting) are in
General.
- Each area has a README mapping its menu entries to their guides and to the
template or server module that implements them. Board-Header/README.md
lists all 40 Board View entries with the template boardBody.jade renders;
five views (Table, Calendar Mode, Time, Timeline, Map) have no guide of
their own yet and say so.
- Board-View.md listed 25 views while BOARD_VIEWS has 40; it now lists all
of them with their English labels.
- Pages keep their file names. Where a released changelog links an old path,
a "# Moved" page stays there and points to the new one; test-menu.sh skips
folders holding only those.
- Undo was filed under Login; it is General/Undo now.
- Every link in docs, code comments, scripts and tests follows the move;
docsLinksResolve checks them all.
Tests that read a doc follow it to its new place. boardViewSettings now fails
when Board-View.md or Board-Header/README.md differs from BOARD_VIEWS, or
names a template boardBody.jade does not render.
Thanks to xet7 !
"[uncaughtException] WeKan is stopping: TypeError: string.toLowerCase is
not a function at later.parse.text ... at Object.schedule
(wekan-ldap/server/sync.js)", in a snap restart loop. The docs give
LDAP_BACKGROUND_SYNC_INTERVAL in milliseconds, settings_get() turns a
numeric value into a Number, and the schedule handed it straight to
later's text parser, which threw from a timer. Text later could not read
did not throw, but made a schedule that never ran - as a quoted
'every 1 mins' from a compose file did, the second report in #2354.
packages/wekan-ldap/server/syncSchedule.js reads the setting and never
throws: milliseconds (a Number or digits) become that interval, later.js
text is read as before, one pair of surrounding quotes is removed, unset
keeps the original default (recur().on(0).minute()), and anything
unreadable logs why and uses the default. The docs, the snap description
and the compose files say both forms work, instead of "leave this unset
so it does not crash".
Tests: tests/ldapSyncSchedule.test.cjs reproduces the crash with the real
later.js from the build, schedules 3600000 ms an hour apart and 10000 ms
ten seconds apart, reads quoted text, keeps the default, never throws on
any value a setting can hold, and fails if any server file hands a setting
to parser.text directly.
Thanks to B2RN and xet7 !
Fixes#2354.
docs/Databases/Supported.md is the one list: FerretDB v1 with SQLite (the
default everywhere), PostgreSQL (works), MySQL, MariaDB and SAP HANA
(experimental), FerretDB 2 (not tested), and MongoDB 6, 7, 8 and 9 - with
what checks each: the 110-query conformance catalogue answers the same on
all of them, and WeKan's own tests run on Meteor's bundled MongoDB 8 and on
FerretDB. It also says what to know about MongoDB versions: the Linux
kernel 6.19-7.0.13 refusal and its fixes, AVX, the driver, and upgrades.
The README and the Status section of CHANGELOG.md gain a Databases list,
docs/Databases/README.md links the page, the Meteor 2.14 compatibility guide
is marked historical, and the conformance page counts 110 cases and the
MongoDB backends. CLAUDE.md and AGENTS.md describe the new Status block.
Thanks to xet7 !
Every login that chooses between a popup and a redirect opened a popup by
default. Google's sign-in pages, and other identity providers', can send
Cross-Origin-Opener-Policy, which makes Meteor's popup look closed at
once: the login was tried before the provider answered, and the user
was back on the sign-in page with no error. Popups are also blocked in
iframes and on some phones. The full-page redirect has no popup to lose.
- OAUTH2_LOGIN_STYLE (OAuth2/OIDC): redirect unless it says popup, in the
server, the Admin Panel catalog and the samples.
- OAUTH_PROVIDERS_LOGIN_STYLE (Google, GitHub, Facebook, X, Meteor
Developer, Weibo, Meetup): redirect by default, in the code, Dockerfile,
snap, Sandstorm package and samples.
- SAML: both identity-provider profiles log in by redirect;
SAML_LOGIN_FLOW=popup still chooses the popup, and the client no longer
falls back to a popup while the configuration loads.
- CAS: redirects unless "popup": true. Its redirect could never finish:
nothing called Meteor.initCas, which the package now does at startup.
The sign-in page also passed username and password as loginWithCas's
options and callback, so its callback never ran, and the CAS redirect
page echoed the request address into HTML unescaped.
A redirect login's error now reaches the reloaded sign-in page, from
accounts-oauth's page-load hook and the SAML and CAS events, instead of
the page coming back silently.
Verified against a freshly built and prepared bundle with the local
identity-provider suite and no login style set: OIDC, SAML, CAS and all
seven Meteor providers log in by redirect, and each refusal is shown on
the sign-in page (31 of 31 with Chromium). Its LDAP wrong-password case
expected wording that changed in 679a8b3499 and is corrected.
Thanks to xet7 !
Use the Markdown textarea for every editor. Remove the retired environment and Snap options from server settings, all platform launchers, build scripts and current documentation. Keep released and archived changelog entries intact.
Keep notification HTML escaping independent of editor configuration, preserve attachment metadata and mention behavior, and cover Markdown and emoji round trips in browser tests. Also import the translator used by attachment error handlers.
Thanks to xet7 !
A reported test environment (mongo:8.2.2 in Kubernetes) filled its data
volume: the checkpoint's fdatasync returned ENOSPC, WiredTiger panicked and
mongod aborted - and then it kept failing on every start, with the disk long
since freed. MongoDB 8.2 keeps a throwaway WiredTiger instance under
<dbPath>/_tmp/spilldb for queries that spill to disk and empties it itself on
each start; after the abort that emptying failed ("Failed to clear dbpath of
the internal WiredTiger instance: Directory not empty"), mongod opened the
half-emptied directory, found no version file, reported "Failed to open the
spill WiredTiger instance ... database corruption detected" and fasserted.
Nothing retries, so the pod restarts forever. The data itself is intact;
deleting that one directory is the whole fix.
- snap-src/bin/mongodb-control deletes _tmp/spilldb before every mongod
start (no mongod is running at that point, and mongod recreates it), and
its start-failure handler recognises the log line and names the directory.
The snap ships MongoDB 7 today, so this is a no-op until it does not.
- Admin Panel / Problems: the db.restart row now says what a "No space left
on device" abort means for a full disk as well as for a network
filesystem, and both it and db.disk-space name the exact spill directory
to delete when MongoDB 8.2 will not start afterwards. The probe looks the
data directory up once and passes it to both rows.
- docs/Databases/MongoDB/Storage-Requirements.md gets the log signature of
both stages, why the second one never ends, the one command, what must
not be touched, and that Docker/Kubernetes users run it themselves since
the database container is MongoDB's own image.
- tests/databaseHealth.test.cjs pins the spill path, both rows' advice, the
probe wiring, the snap's pre-start deletion (before the first start, after
the running-instance check, and the only recursive delete apart from the
existing FerretDB reset) and the documentation.
Thanks to xet7 !
OAUTH_GOOGLE_*, OAUTH_GITHUB_*, OAUTH_FACEBOOK_*, OAUTH_TWITTER_*,
OAUTH_METEOR_DEVELOPER_*, OAUTH_WEIBO_*, OAUTH_MEETUP_*,
OAUTH_PROVIDERS_LOGIN_STYLE, OAUTH_PROVIDERS_MERGE_EXISTING_USERS and
PASSWORDLESS_ENABLED are now documented, commented out or defaulted, in
every place the OAUTH2_* and SAML_* settings already are: docker-compose.yml
and its FerretDB v1 / v2 and MongoDB variants, the Dockerfiles' ENV block,
start-wekan.sh, start-wekan.bat, the Snap config (keys list plus
DESCRIPTION_/DEFAULT_/KEY_ triples) and wekan-help, the Sandstorm package
definition, the stacksmith Docker-secrets reader and secrets/README.md.
The comments name the callback URL to register at each provider,
<ROOT_URL>/_oauth/<service>, and say that Admin Panel / People / Login
overrides the environment.
New docs pages docs/Features/Login/OAuth-Providers.md and Passwordless.md,
linked from the docs index. tests/oauthProvidersPlatformEnv.test.cjs pins
that every variable is present on every platform and that the docs exist
and are linked.
Thanks to xet7 !
From a reported MongoDB crash series (.tools/crash: WeKan and MongoDB in one
Kubernetes pod, the database volume on an SMB/DFS share):
- The crash itself is WiredTiger's checkpoint fsync() returning ENOSPC
("No space left on device" with 519 MB used and 10 GiB free) on the
network share, followed by a fatal assertion and abort - every checkpoint,
so a restart loop. That is the storage, not WeKan: MongoDB requires a
local filesystem with real fsync semantics under its data directory.
docs/Databases/MongoDB/Storage-Requirements.md now says so, shows what
the log looks like when it is not, and what to do (local block volume;
attachments may still live on a share; MONGO_URL over TCP is fine).
- The WeKan-side finding in the same logs: card_comment_reactions had no
index at all, and every board open queried it by `cardId: { $in: [...] }`
and by `boardId` as a full collection scan - 3,397 COLLSCANs of that one
collection in the last crash log, up to 1.6 s each, the bulk of its
"Slow query" lines. server/models/cardCommentReactions.js now ensures
indexes on cardId, boardId and cardCommentId at startup, like every other
per-card collection.
tests/cardCommentReactionsIndexes.test.cjs pins both.
Thanks to xet7 !
Move feature and webserver documentation into the reorganized hierarchy, update references to their new relative paths, and extend link validation across the complete documentation tree.\n\nThanks to xet7 !
The pane is called Problems in the menu, its address is /admin/problems, and
docs/Features/Admin-Panel/Problems is where it is documented. Two things still
called it Reports, from before it was renamed:
* the route NAME was 'admin-reports', where its three siblings are 'setting',
'people' and 'attachments' - the bare page key. So the one route whose name
did not match its own address;
* the template, and its three files, were adminReports.
Both are 'problems' / adminProblems now, across 41 files.
What did NOT change is `legacyBase: '/admin-reports'`. That is an address people
have bookmarked and it still redirects here - the point of the rename is that
the code agrees with the menu, not that an old link should break. The CHANGELOG
entries describing the old addresses are left alone too: a released section is a
record of what shipped.
Thanks to xet7 !
A new install with nothing changed but the port and ROOT_URL could not reach the
database at all:
MongoServerSelectionError: connect ECONNREFUSED 0.0.0.0:27017
reason: TopologyDescription { type: 'ReplicaSetNoPrimary',
servers: Map(1) { '0.0.0.0:27017' => [ServerDescription] },
setName: 'rs0', ... }
0.0.0.0 is in no compose file. It is FerretDB's own listen address, and the
driver was handed it by the server.
The ferretdb service runs with --repl-set-name=rs0, added in #6480/#6481 so
Meteor can tail an OpLog instead of poll-and-diff. FerretDB therefore answers the
hello handshake as a one-member replica set, and fills hosts, me and primary with
its --listen-addr, which is the wildcard 0.0.0.0:27017. A driver that is not in
direct-connection mode reads that reply as an invitation to do replica-set
DISCOVERY: it adopts the advertised member list and drops the seed it was given,
because the server reports a name other than the one that was dialled. So
mongodb://ferretdb:27017 became 0.0.0.0:27017, which inside the wekan container is
the wekan container itself, where nothing listens.
Measured against FerretDB v1.49.0 - the version reported - with the mongodb
driver the bundle ships:
mongodb://localhost:27017/wekan
-> ReplicaSetWithPrimary, servers: 0.0.0.0:27017 (the seed discarded)
mongodb://localhost:27017/wekan?directConnection=true
-> Single, servers: localhost:27017
directConnection=true keeps the driver on exactly the host it was given. It costs
nothing else: the handshake still reports setName rs0, which is the only thing
Meteor checks before it will tail an OpLog (accounts for the check in the mongo
package's oplog_replica_set_error.js), so #6480/#6481 are unaffected and the
commented MONGO_OPLOG_URL examples keep replicaSet=rs0 beside it.
All five FerretDB v1 compose files are fixed, being the ones that start FerretDB
with --repl-set-name. The MongoDB compose files are deliberately left alone: those
are real replica sets whose members are reachable under the names they advertise,
and directConnection would pin Meteor to a single node of one.
tests/ferretdbDirectConnection.test.cjs pins both halves - that a compose file
running FerretDB as a replica set carries directConnection=true on every URL
pointing at it, and that the MongoDB ones do not.
Thanks to Dandrass and xet7 !
Fixes#6582,
"Which CPUs is this published for, and why not that one" was answerable for
the snap and nowhere else, so the same page now exists for the image:
docs/Platforms/FOSS/Container/Docker/CPU-platforms.md. It carries the platform
matrix with each platform's TARGETARCH, TARGETVARIANT and bundle, how the set
is decided on every release from what the base image publishes and which
bundles landed, the three names that differ between Docker, Debian and the
WeKan bundles, and one section per platform that is deliberately NOT an image:
loong64 (no base image exists at any tag), armv6 (gated on a base with
arm/v6), armv7 (the one 32-bit ARM slot must carry the armhf baseline) and the
Windows and macOS bundles. It ends with why the FerretDB image covers more
platforms than the WeKan one - FROM scratch around a static Go binary needs no
userland at all.
The snap's page gains the matching armv6 section - the Snap Store has no armv6
architecture, and its only 32-bit ARM is armhf, which an ARMv6 board cannot
run - and a table of all fifteen bundles against which six become snaps, so
"it is missing" and "it cannot be there" stop looking alike. The two pages
link each other, because the answer differs between them.
docs/Databases/FerretDB/1/README.md said the per-architecture FerretDB binary
was embedded in the bundles "for ppc64le, s390x, riscv64". Every bundle
carries one; those three are part of a longer list of platforms where it is
the DEFAULT because MongoDB publishes no server. It now lists all seventeen
built binaries and separates the three 32-bit ARM builds that are not variants
of each other: armhf is GOARM=7, armv6 is GOARM=6, armel is GOARM=5 software
floating point.
Thanks to xet7 !
Three things the first run on arm64 found.
The FerretDB it built could not start: `panic: commit.txt value ... !=
vcs.revision value ...`. Go stamps the VCS revision into the binary and
FerretDB's build/version panics when it disagrees with the committed
commit.txt, which only the generator refreshes - so every commit made after the
last refresh built a binary that panicked. Fixed in the fork (its act_build now
regenerates them, as act_dist already did).
Ports: FerretDB listened on 27017, which is where a dev server's database lives
and where the compose files publish FerretDB - so this could not run beside
anything else and, worse, could have pointed the tests at somebody else's
database and rewritten it. It listens on 37017 now and publishes the database
server on 35432, both moved on if busy, both overridable with
WEKAN_CONFORMANCE_PORT and WEKAN_CONFORMANCE_DB_PORT, and its containers are
named per run so a compose stack is never reused or stopped.
And Ctrl-C only killed whatever was in the foreground - a registry lookup, a
sleep - after which the loop carried on and reported the interrupted lookup as
"NO linux/arm64", which is a lie about the image. An interrupt ends the run now,
and the image check has three outcomes rather than two: has it, does not have
it, could not ask. A FerretDB that will not start also prints the last lines of
its log, because the reason is usually one line.
Thanks to xet7 !
FerretDB v1 translates one MongoDB query into five different SQL dialects, so
"WeKan starts on MariaDB" says very little. The question that decides whether a
backend can be trusted with a board is whether {n: {$gt: 5}} returns the same
documents, in the same order, as it does on SQLite - and nothing was asking it.
./build.sh -> Tests -> "All databases (sequential)" now does, and build.bat has
the same entry (running the same script through bash rather than a second
implementation that would drift).
It builds FerretDB v1 from source first: the FerretDB subdirectory is cloned
from git@github.com:wekan/FerretDB if it is not there, updated if it is, and
built through its own build.sh, which installs the Go toolchain and the module
dependencies when they are missing. The tests run against THAT binary, not a
downloaded release, so they test the newest code.
Then, for each backend whose database image has a build for THIS CPU - asked of
the registry with docker manifest inspect, so no table here can go stale - it
starts that database, runs the freshly built FerretDB against it, runs the whole
catalogue and stops everything. Sequential, because they all use the same
FerretDB port and a database under test should not be competing with three
others; the menu entry says so.
The catalogue is 100 cases in 15 groups, taken from FerretDB v1's own handler
sources rather than from MongoDB's manual: every query, update and bitwise
operator, the aggregation stages and accumulators, projection and sorting and
paging, count and distinct, indexes and uniqueness, capped collections (which is
how the OpLog exists at all), and the commands whose answers may legitimately
differ. The seed data is deliberately awkward - a null, a missing field, a
negative number, a zero, an empty array, an array of documents - because tidy
data lets a broken translation pass.
Answers are normalised and compared byte for byte against SQLite, so document
ORDER counts. Three cases are compared loosely and say so: an explain plan
differs per engine by definition, and collStats/dbStats are sizes a different
storage engine may answer differently. Two backends failing the same way is
agreement about a limitation; one answering where another fails is a difference.
Everything lands in ../log/<datetime>/ with every other test run: a JSON of what
each backend answered, each backend's container output, the build log, a summary
and db-conformance-report.md. SAP HANA is opt-in behind WEKAN_CONFORMANCE_HANA=1
- amd64-only, ~16 GB of RAM, and SAP's licence to accept - so a menu choice
cannot start it by accident.
Thanks to xet7 !
The page listed images and requirements, which left the reader to work out the
pattern themselves. There is a pattern: the databases are not fifteen separate
questions but seven families, and inside a family the answer is the same one.
Each family now says what IT is missing. PostgreSQL-wire and MySQL-wire need no
code at all, only a live run against their dialect - that is where CockroachDB's
s390x image could be won. Enterprise SQL needs a whole backend each, and the
driver is the obstacle only for Db2, whose cgo-only driver cannot be in a
nine-architecture build. Embedded databases need a pure-Go driver before
anything else. Columnar engines pass the driver gate and fail the second one:
WeKan updates single cards constantly, and they are built for the opposite.
Key-value stores would have to have indexes, filters and atomicity implemented
inside FerretDB, which is not a backend but a database. And what already speaks
MongoDB needs no FerretDB at all - the reason for this page is that on ppc64le,
s390x, riscv64 and loong64 that family has no members.
The page opens with all of it in five sentences and one table, because the
conclusion - that the most valuable work is verifying the three backends that
already exist, not writing a fourth - was previously only visible to whoever
read to the end.
Corrections while checking: Oracle DOES have a pure-Go driver (sijms/go-ora
implements the TNS protocol itself), so it is not blocked by cgo the way Db2 is;
and Cassandra belongs in the CPU table, since its image covers ppc64le and
s390x.
Thanks to xet7 !
WeKan runs on every CPU Node.js runs on, and this fork of FerretDB v1 exists
because MongoDB does not publish a server for most of them. Two questions kept
coming back: which databases even have images for those CPUs, and what would be
missing in FerretDB v1 before it could store into one.
docs/Databases/FerretDB/1/Alternatives.md answers both. The architecture table
is read from each registry's own manifest, not from documentation, and it says
plainly what that data shows: PostgreSQL is the only widely-portable database
server - the only one publishing ppc64le, s390x AND riscv64 - MariaDB covers
ppc64le and s390x, MySQL neither, and MongoDB itself and upstream FerretDB 2
are amd64 + arm64 only. This fork's own image covers nine platforms.
What a new backend needs is taken from the code: a pure-Go driver, because the
binaries are built CGO_ENABLED=0 and that is what makes one build serve nine
architectures (which is why Oracle and Db2 are out, despite Db2's ppc64le and
s390x images); the three backends interfaces, none of them stubbable; a
metadata registry; the SQL features the translation actually uses - JSON path
extraction, containment and type inspection, schemas, expression indexes, a
parseable EXPLAIN, and the record-id column that capped collections and
therefore the OpLog are built on; correct MongoDB semantics on top; and a live
integration run, which is the whole difference between the confirmed and
experimental rows.
And the shortcut: a database that speaks the PostgreSQL or MySQL wire protocol
needs no new backend, only an existing one that survives its dialect -
CockroachDB brings s390x that way, for the price of a test run.
Thanks to xet7 !
It was a flat list where the database was a filename prefix, in three spellings
- MongoDB-Driver-System.md, mongodb-avx-qemu.md, MongoDB_OpLog_Enablement.md -
plus FerretDB2-PostgreSQL.md and a ToroDB-PostgreSQL directory. Somebody asking
"how do I run WeKan on PostgreSQL" had to read the prefixes to find out which
files were even about their database.
Now: Migrations, MongoDB, FerretDB (with 1 and 2, because v1 and v2 are
different products with different backends) and ToroDB (with PostgreSQL). Each
file keeps its name minus the redundant prefix - MongoDB/Driver-System.md,
MongoDB/avx-qemu.md - and the old PostgreSQL.md, which was an index of FerretDB
and ToroDB rather than a document about PostgreSQL, becomes the directory's
README.md.
Every directory has a README.md saying what is in it and where to go next,
including FerretDB/1, which had no documentation at all until now although it
is WeKan's default database and just gained four more backends. Links to a
directory point at the DIRECTORY, not at its README.md, because README.md is
what is opened by default.
Every link that pointed into the old paths is updated - in docs, in
docker-compose-ferretdb-v2-postgresql.yml, and in the wekan.fi website - and
the relative links inside the moved files, which gained a directory level and
so needed one more "../". tests/docsDatabases.test.cjs pins the layout, the
READMEs, the directory links, that every relative link resolves, and that
nothing outside CHANGELOG.md still names an old path.
Thanks to xet7 !
Renames rebuild-wekan.sh -> build.sh, rebuild-wekan.bat -> build.bat, and the
guard test tests/rebuildWekanDevServer.test.cjs -> tests/buildDevServer.test.cjs,
and updates every reference across the repo: the docs (Build-from-source, Windows,
Sandstorm, Raspberry Pi, virtual-appliance, Debugging, Developer-Documentation,
FerretDB, OpLog, Rclone, Emoji, Multiverse roadmap, Security remediation, Tests),
README.md, CONTRIBUTING.md, CLAUDE.md, package.json (test:unit:node), the release
scripts (release-all.sh, fix-changelog-hashes.sh, node-update.sh), the disabled
playwright workflow, playwright.config.js, the REST spec comment, tools/forge-mirror.js,
and the CHANGELOG. The menu and every option are unchanged; only the filename and
its references. The website has no references.
Thanks to xet7 !
The per-swimlane-lists model was reverted (its migrations were removed in #6521);
lists are board-wide and the same lists appear in every swimlane. Update the docs
to say so:
* Features/Lists/Lists.md and Features/Board/Swimlanes.md now state plainly that
lists are shared across swimlanes (a card belongs to one list and one
swimlane), and that per-swimlane lists were an experiment that was reverted.
* The docs/Databases/Migrations/ writeups (MIGRATION_SYSTEM_IMPROVEMENTS,
MIGRATION_SYSTEM_REVIEW_COMPLETE, CODE_CHANGES_SUMMARY, SESSION_SUMMARY,
verify-migrations.sh) describe the old cron-driven migration system and the
per-swimlane-lists migration, both removed; each now carries an OBSOLETE banner
pointing at the current board-wide model and the live repair paths
(merge-per-swimlane-lists + repairBoardData).
The dated PerUserDataAudit2025-12-23 snapshot is left as-is: it already observed
that lists are board-wide, and it is a point-in-time record.
Thanks to xet7 !
Fix#6458 completely: the reporter's host runs inside a hypervisor that
masks AVX, and the old per-tool AVX wrappers (amd64-only, PATH-based)
were bypassed by absolute-path mongod invocations.
- New general snap-src/bin/cpu-exec: runs any binary directly when the
CPU has the features it declares (--features x86_64=avx,aarch64=atomics
or WEKAN_REQUIRED_CPU_FEATURES) and through same-arch qemu-user when
not; plain zero-overhead exec with no declared features. Lookup order:
WEKAN_QEMU_USER, qemu-<arch> next to the script (bundle/Docker),
$SNAP/qemu-<arch>, the snap's migratemongo/avx one, then system
qemu-user(-static). Clear error + direct run when no qemu exists, so
the real SIGILL surfaces instead of hiding.
- Snap: mongodb-control and migration-control run every mongod 7 through
cpu-exec, so MongoDB works without AVX (slower) and migration can READ
modern MongoDB data on such CPUs; completed-FerretDB switch and
migration fallbacks unchanged (switch_to_completed_ferretdb extracted).
- release-all.yml: every Linux bundle .zip ships cpu-exec + this-arch's
static qemu-user (amd64/arm64/ppc64le/s390x/riscv64; stripped from
win64/mac bundles), flowing into Docker and Snap automatically;
Sandstorm .spk gets both via build-deps.sh. Bundle launcher and Docker
entrypoint route node/ferretdb through cpu-exec.
- Docs: mongodb-avx-qemu.md and mongodb-raspi4-qemu.md document the
built-in mechanism.
Tests, with negative cases, for the newest features and fixes at
changelog: htmljs array-vs-attributes (#6459), description draft
normalize/reset (#6455), comment draft lifecycle (#5547), attachment
delete guard (#5282), rules move actions + import normalization (#6472),
snap mongod-failure/migration recovery (#6458#6466), FerretDB polling
defaults (#6467#6468), UI density guards for every #6465 fix, and a
behavioral cpu-exec test against fake cpuinfo + fake qemu. All wired
into test:unit:node. Go table tests for the FerretDB SQLite filter
pushdown are in the wekan/FerretDB repo.
Updated ChangeLog.
Thanks to xet7.
Add a "Build from source & start (up -d --build)" action to the Docker
menus in rebuild-wekan.sh and rebuild-wekan.bat. It builds the wekan-app
image from the local Dockerfile, tags it as the image the chosen compose
file references (ghcr.io/wekan/wekan:latest), and starts that freshly
built container instead of a possibly-stale prebuilt image pulled from
the registry - useful when a source change (e.g. the FerretDB
Version-page detection) is not in the pulled image yet.
Remove the obsolete top-level "version:" attribute from all docker
compose files, which Docker Compose v2 warns about and ignores:
docker-compose.yml, docker-compose-ferretdb-v2-postgresql.yml,
docker-compose-multitenancy.yml, .devcontainer/docker-compose.yml, and
docs/Databases/ToroDB-PostgreSQL/docker-compose.yml.
Thanks to xet7 !
Rename docker-compose-ferretdb-v1-sqlite.yml -> docker-compose.yml (the
new default) and the old MongoDB docker-compose.yml ->
docker-compose-mongodb-v7.yml, so "docker compose up -d" uses the light,
self-contained FerretDB v1 + embedded SQLite from
https://github.com/wekan/FerretDB by default.
Update rebuild-wekan.sh and rebuild-wekan.bat Docker menus to list
FerretDB v1 SQLite first (default) and point at the new filenames.
Update the compose files' own header comments (self-identification and
cross-references), note the new default at the top of docker-compose.yml,
and update the Docker docs (main Docker.md gets a compose-file/database
map; FerretDB2-PostgreSQL.md points at the v2 postgresql compose file).
Thanks to xet7 !
Many docs still used flat GitHub-wiki page-name links like [Backup](Backup),
[OAuth2](OAuth2) or [FAQ](FAQ) that only resolved on the old wiki; in the docs/
hierarchy they were broken. Rewrite them to correct relative paths of the
existing pages (e.g. ](Backup) -> ](Backup.md), ](OAuth2) -> ](../Login/OAuth2.md),
](Sandstorm) -> ](../Platforms/FOSS/Sandstorm/Sandstorm.md)).
Mapping: page name -> docs/<...>/<Name>.md by filename, with explicit handling
for renamed pages (Caddy/Nginx-Webserver-Config, Wekan-Markdown, the Sandstorm
grain export page, Uberspace) and the REST-API-<X> family -> docs/API/<X>.md.
Ambiguous names resolved by preferring a same-directory file or the primary page.
Anchors and markdown titles preserved. 280 links across the docs tree.
Out of scope (left as-is): source-code references (models/*.js etc.) and
[Link](url) placeholders, which are not docs pages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GitHub wiki is disabled and its content was migrated into docs/, so every
https://github.com/wekan/wekan/wiki/... link now redirects to the repo home and
is broken. Rewrite all internal links in docs/ to relative paths:
- wiki/<Page> -> relative path to the corresponding docs/ file (renamed pages
mapped explicitly: Caddy-Webserver-Config->Webserver/Caddy.md, Wekan-Markdown
->Features/Markdown/Markdown.md, REST-API-Role->API/Role.md, etc.)
- wiki home -> docs/README.md
- blob|tree|raw/main|master/<path> -> relative path (anchors and #Lxx line
ranges preserved; bare URLs wrapped as [name](path) so they stay clickable)
Kept absolute (intentionally not flattened to the current ref):
- ref-pinned links (edge, devel, version tags, commit hashes)
- links whose target no longer exists in the tree
wiki/issues -> the GitHub issues tracker URL (external). 436 links across 113
files; verified no wiki links remain and no broken relative links introduced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lauri Ojansivu
