upstream/wekan
2
0 Fork 1
mirror of https://github.com/wekan/wekan.git synced 2026-11-05 01:20:07 -05:00
Code Releases 1 Activity

Startup schema upgrade, self-healing attachments, migration speed; fix

Browse Source 
#5695 #4897 #4654 #5174 #4825 #5659 #4730 #4419 #4158 #4560 #4036
#1959 #1971.

- Startup schema upgrade (server/lib/schemaUpgradeSteps.js): on start,
  check that data from EVERY old WeKan version is migrated to the newest
  structure and migrate only what is missing - all platforms, MongoDB and
  FerretDB. Version-gated (marker stores WeKan version + datetime of the
  last re-check; one findOne per boot until a new release), non-blocking,
  live dashboard at /schema-upgrade-status showing the product name, fast
  on big databases (distinct() set-joins, updateMany batches). Steps:
  archived-flag backfill, swimlane structure + visibility rescue
  (#1959 #1971: cards under deleted/archived/foreign swimlanes reassigned
  to a visible swimlane; card-insert hook validates supplied swimlaneIds),
  merge-per-swimlane-lists (v7.98/v8.0x list copies merged back into
  shared lists: no duplicate columns in Lists view, everything visible in
  Swimlanes view, cards keep their swimlane), embedded checklist items,
  customFields boardIds, allows* backfills, members isActive, permission
  lowercase, filesystem path healing.
- Migration speed (5-hour reports): insertMany batches in the text phase,
  preloaded target-record maps in the file phases.
- Snap attachment-repair + FILES_ONLY importer mode (see prior commit
  wiring) extended with CFS FileSystem-store coverage
  (ATTACHMENTS_STORE_PATH era), missing-binary reporting, and
  migrate-gridfs-to-fs.mjs bucket fixes.
- OAuth2/OIDC: OAUTH2_LOGIN_STYLE=redirect honored (#5695); concurrent
  logins no longer contaminate each other's user data (#4897); account
  merge no longer wipes avatars/templates (#4560).
- LDAP: background sync fixed (#4654), password login rejected after LDAP
  migration (#4419), LDAP_ENCRYPTION=true actually encrypts (#4158),
  group filter accepts multiple groups and admits admin-sync groups
  (#4036).
- All Boards tiles: per-list card counts + member avatars restored
  non-reactively (#5174 #4825). Uniform default list widths (#5659).
  Declined invitations no longer resurrect membership (#4730).
- Tests with negative cases for every fix (tests/*.test.cjs; suite green
  except the pre-existing environment-only htmljs failure).

Closes #5600: OAUTH2_CA_CERT already works - the fetch migration
(49a272d6c) applies the CA to all OIDC calls via https.Agent.
Closes #3828: board members seeing all board cards is by design (no
per-card visibility model), and organizations/teams now exist.

Thanks to mueschel, ArturRuta, gerardo-junior, fabianrbz, Miuler,
butteredCat-2021, Griiimm, preciousamorc, farwayer, LeoLu-eng, hnb1,
rangersdo and xet7.
This commit is contained in:
Lauri Ojansivu committed 2026-07-17 03:07:59 +03:00
1 parent 65bf215cbe
commit 9ddbc777dd
57 files changed
+6308 -262
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGELOG.md
+173 -1
View File
@@ -88,7 +88,179 @@ them up next.
# Upcoming WeKan ® release
This release fixes the following bugs:
This release adds the following new features:
- **Snap: SELF-HEALING attachments — the fixes below apply themselves automatically on
upgrade, no commands needed** ([#6473](https://github.com/wekan/wekan/issues/6473),
`snap-src/bin/attachment-repair` (new), `snap-src/bin/wekan-control`,
`snap-src/bin/migration-control`, `snap-src/bin/wekan-force-migrate`,
`releases/migrate-mongodb-to-ferretdb.mjs`, `snap-src/bin/migrate-mongo3-to-ferretdb.mjs`,
`snapcraft.yaml`, `snapcraft-core26.yaml`). The snap auto-refreshes on ~15k servers, so a fix
that needs `snap run wekan.migrate` typed by hand does not reach most of them — and a full
re-migration would be WRONG anyway, because it rebuilds FerretDB from the frozen MongoDB
source and would lose boards/cards users created on FerretDB since migrating. Instead, on
every start on FerretDB, `wekan-control` now launches `attachment-repair` in the
**background**: it starts a temporary source mongod (7.x or the bundled 3.2 reader — the
MongoDB data was never modified, so everything is recoverable) and runs the migration
importer in a new **incremental FILES_ONLY mode** that **checks what is already migrated
and migrates only what is missing**: text collections are never touched, every attachment/
avatar whose target record is on `fs` with the file actually on disk is verified and
skipped, records deleted by users since the migration are never resurrected, and only the
missing binaries/records are extracted — so on healthy servers the repair is a fast no-op
and on #6473-affected servers the attachments simply appear, live, while WeKan runs. It
runs **once** (marker `$SNAP_COMMON/.attachments-files-v2-done`; a fresh successful
migration pre-writes it, `snap run wekan.migrate` clears it, failures retry on the next
start without ever blocking WeKan from starting), and a manual
`snap run wekan.repair-attachments` command is registered too. The importer itself now
stamps the migration marker with `filesVersion` (currently 2) and, when it finds a marker
with an older `filesVersion` — or `FILES_ONLY=true` in the environment — automatically
switches to this incremental repair, so **Docker/source installs get the same self-healing**
by simply re-running the same importer command they migrated with. Verified end-to-end
against two live FerretDB instances with the real importer: a broken-migration target
(missing CollectionFS record, gridFsFileId-only record, marker without filesVersion) was
repaired — record re-created with its card linkage, binary extracted, record repointed —
while a board renamed on FerretDB after the migration stayed untouched, and the second run
exited immediately as already-migrated. Behavioral tests (positive + negative) drive the
real bash script with stubbed snap tooling: `tests/attachmentRepair.test.cjs`. Thanks to
mueschel and xet7.
- **All platforms: startup schema upgrade — WeKan now CHECKS on start that data from EVERY old
WeKan version has been migrated to the newest database structure, and migrates only what is
missing** ([#6473](https://github.com/wekan/wekan/issues/6473) follow-up,
[#1959](https://github.com/wekan/wekan/issues/1959), [#1971](https://github.com/wekan/wekan/issues/1971),
`server/lib/schemaUpgradeSteps.js` (new), `server/startupSchemaUpgrade.js` (new),
`server/migrations/ensureValidSwimlaneIds.js`, `server/imports.js`). WeKan v0.9–v8.00 ran
startup migrations; v8.01 disabled them (large databases meant long downtime) in favour of
read-time compatibility — which covers the swimlane era but not everything, so text data
from old versions could sit invisible in the database. The new startup upgrade reinstates
the safety net without the downtime: **version-gated** (the `_wekan_migration` marker stores
the WeKan version and datetime of the previous successful re-check, so while the version is
unchanged a boot costs ONE `findOne` — a full re-check is mandatory only after a new WeKan
release, or `WEKAN_FORCE_SCHEMA_UPGRADE=true`; opt out with `WEKAN_SKIP_SCHEMA_UPGRADE=true`),
**non-blocking** (runs in the background, WeKan serves immediately), with a **live migration
dashboard at `/schema-upgrade-status`** on every platform that shows the Admin Panel product
name when one is set (not "WeKan") plus per-step progress, and **fast on big databases**
(bounded existence probes, `distinct()` set-joins instead of card scans, and server-side
`updateMany` batches instead of per-document round trips — thousands of cards never mean
thousands of queries). Steps, each verified against `git show v8.00:server/migrations.js`
and each idempotent: `archived-flag-backfill` (docs missing `archived` never match the
`archived: false` view queries — whole boards/lists/cards were invisible in the Swimlanes
and Lists views), `swimlane-structure` (every board gets a visible swimlane, every list/card
a `swimlaneId`, cards with a dangling `listId` are rescued to a visible list — and, fixing
#1959 and #1971, unarchived cards whose swimlane was DELETED, ARCHIVED or belongs to another
board are reassigned to the board's first visible swimlane, so everything is visible in both
the Swimlanes view and the Lists view; the card-insert hook now also validates
client-supplied swimlaneIds so new cards can never land under a deleted/archived swimlane
again), `checklist-items-embedded` (pre-v0.79 embedded `checklist.items[]` extracted to the
`ChecklistItems` collection — the text was in the database but never shown),
`customfields-boardIds` (pre-v2.49 scalar `boardId` → `boardIds` array — old custom field
definitions and their card values were orphaned), `board-allows-defaults` (the ~33
defaultValue-true `allows*` flags backfilled — a missing flag rendered as false and HID
existing descriptions/checklists/comments/attachments), `board-members-isactive` (members
without `isActive` were denied board access), `board-permission-lowercase` ('PUBLIC' boards
had silently become member-only), and `fs-path-heal` (filesystem attachments/avatars whose
recorded path predates the current WRITABLE_PATH layout — v6.10-18 `uploads/<coll>`,
v6.19-v8.4x `WRITABLE_PATH/<coll>`, CFS→ostrio temp files — are located and repointed/copied
into the current layout). A step that fails or leaves unresolved work never blocks WeKan
from starting and keeps the version un-stamped so the next boot re-checks. 36+ unit tests
with negative cases (`tests/schemaUpgradeSteps.test.cjs`), plus verified end-to-end against
a live FerretDB (SQLite) with old-shape seed data: all 8 steps migrate correctly and the
second boot is gated to a no-op. Thanks to mueschel and xet7.
- **Migration speed: batched writes and preloaded lookups instead of per-document round trips**
(5-hour migrations reported; `releases/migrate-mongodb-to-ferretdb.mjs`). The text phase now
copies each batch with ONE unordered `insertMany` round trip (falling back to per-document
`replaceOne` upserts only for batches that hit duplicates on resumed/re-run migrations), and
the attachment/avatar phases preload the target's metadata records once per bucket instead
of one `findOne` per file (bounded: collections over 100k records fall back to per-file
lookups). The startup schema upgrade uses the same philosophy (`distinct()`/`updateMany`).
Thanks to mueschel and xet7.
and fixes the following bugs:
- **OAuth2/OIDC: `OAUTH2_LOGIN_STYLE=redirect` was ignored — a popup always opened**
([#5695](https://github.com/wekan/wekan/issues/5695), `packages/wekan-oidc/oidc_client.js`,
`server/authentication.js`, `server/models/settings.js`, `client/components/main/layouts.js`).
The setting was stored correctly in the OIDC service configuration, but the login button
always passed `loginStyle: 'popup'`, and Meteor's `OAuth._loginStyle` gives the caller's
option precedence — so the admin's redirect setting silently lost on every login (and the
Meteor-internal `loginStyle` option even leaked into the provider's authorization URL). The
client now honors a configured `loginStyle: 'redirect'` over the button's generic popup
default (explicit caller choices still win; Safari-private-mode popup fallback kept) and no
longer leaks the option to the provider. Also repaired the existing
`OIDC_REDIRECTION_ENABLED=true` "go straight to the provider" feature, which was doubly
broken since the Meteor 3 port: `isOidcRedirectionEnabled` inspected a Promise (always
false), and the client handler assigned an undeclared variable (strict-mode ReferenceError).
Behavioral tests drive the real client code in a VM against Meteor's loginStyle precedence:
`tests/oauth2LoginStyle.test.cjs` (12 tests, positive + negative). Thanks to ArturRuta and
xet7.
- **LDAP background sync only worked once per user** ([#4654](https://github.com/wekan/wekan/issues/4654),
`packages/wekan-ldap/server/ldap.js`, `sync.js`, new `userIdFilter.js`): `getUserById` crashed or built the
invalid filter `(|(=user))` when `LDAP_UNIQUE_IDENTIFIER_FIELD` was unset/empty (it never consulted
`LDAP_USER_SEARCH_FIELD`, where the stored id actually comes from), and the username sync passed `$set` as
query OPTIONS to `findOneAsync` — logging "Syncing user username" while writing nothing. New shared
`buildUserIdFilter()` ORs across both configured fields, `idAttribute` is persisted on new LDAP users, and
the username sync actually updates. Tests: `tests/ldapUserIdFilter.test.cjs` (11). Thanks to fabianrbz and xet7.
- **All Boards page: per-list card counts and member avatars never showed** ([#5174](https://github.com/wekan/wekan/issues/5174),
[#4825](https://github.com/wekan/wekan/issues/4825), new `models/lib/boardTileData.js`,
`server/publications/boards.js`, `client/components/boards/boardsList.js`, `.jade`): the helpers were stubbed
to `[]` to stop the #4214 reactive "icons dance", and the gating flags were never published. New non-reactive
`getAllBoardsTileData` method (one boards query, one lists query, one grouped card count) fetched once per
page visit; per-board "Show card count per list"/"Show Board members avatars" settings enforced strictly both
ways. Tests: `tests/boardTileData.test.cjs` (17). Thanks to mueschel, Miuler and xet7.
- **Lists rendered with different widths by default** ([#5659](https://github.com/wekan/wekan/issues/5659),
new `models/lib/listWidth.js`, `client/components/lists/list.js`, `listHeader.js`, `models/users.js`): the
default width was duplicated in four resolution paths that disagreed (270 vs 272), so lists on the same
(public) board could differ with no customization. Single source of truth (272), all paths normalize
out-of-range values the same way; customized widths still win. Tests: `tests/listWidthDefaults.test.cjs` (12).
Thanks to butteredCat-2021 and xet7.
- **Declining a board invitation kept the board active in the member's overview** ([#4730](https://github.com/wekan/wekan/issues/4730),
`server/models/boards.js`, new `models/lib/boardInvites.js`): the decline flow called `quitBoard` (deactivate)
then `acceptInvite`, which unconditionally REACTIVATED membership — it also let any removed member re-add
themselves. `quitBoard` now clears the pending invitation (and works for stale-invite-only users);
`acceptInvite` only activates when an invitation actually exists. Tests: `tests/boardInvites.test.cjs` (11).
Thanks to Griiimm and xet7.
- **After migrating a user from password to LDAP, the old local password still logged in**
([#4419](https://github.com/wekan/wekan/issues/4419), new `server/lib/ldapPasswordLoginGuard.js`,
`server/authentication.js`): a `validateLoginAttempt` hook now rejects password-service logins for
`authenticationMethod: 'ldap'` users while LDAP is enabled — respecting the `LDAP_LOGIN_FALLBACK=true`
feature, never touching other services or session resumes, and opt-out-able with
`LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN=true` so no deployment is hard-locked. Tests:
`tests/ldapPasswordLoginGuard.test.cjs` (12). Thanks to preciousamorc and xet7.
- **`LDAP_ENCRYPTION=true` (the documented value) silently connected WITHOUT encryption**
([#4158](https://github.com/wekan/wekan/issues/4158), new `packages/wekan-ldap/server/encryptionSetting.js`,
`ldap.js`, `docs/Login/LDAP.md`): only the undocumented `ssl`/`tls` values did anything, and any other value
(including `true`, which JSON-parses to a boolean) meant silent plaintext. Now `true`→LDAPS,
`starttls`→STARTTLS, legacy `ssl`/`tls` keep their historical meanings with a deprecation notice, and unknown
values log a clear warning listing the accepted ones. Tests: `tests/ldapEncryptionSetting.test.cjs` (22).
Thanks to farwayer and xet7.
- **OIDC login onto an existing account wiped the profile — avatars and templates disappeared**
([#4560](https://github.com/wekan/wekan/issues/4560), `server/models/users.js`): the
`OAUTH2_MERGE_EXISTING_USERS` merge path replaced the whole `profile` with the OIDC-derived one, losing
`avatarUrl`, `templatesBoardId` (+ template swimlanes), language and preferences. The merge now preserves the
stored profile and only fills gaps/updates the asserted fullname; the fail-closed linking rules
(GHSA-mp7g-hj5q-gxhq) are untouched. Tests: `tests/oidcProfileMerge.test.cjs` (7, fails on pre-fix code).
Thanks to LeoLu-eng and xet7.
- **OAuth2/OIDC: concurrent logins could contaminate each other's user data — users saw stale
or missing emails/username/teams, and the database could disagree with the UI**
([#4897](https://github.com/wekan/wekan/issues/4897), `packages/wekan-oidc/oidc_server.js`,
`packages/wekan-oidc/loginHandler.js`). The OIDC server flow kept `profile`, `serviceData`
and `userinfo` as MODULE-SCOPE variables shared by every login of every user: fields the
current login did not overwrite leaked from the previous user's login (refreshToken,
whitelisted id-token claims, branch-dependent email), and because the handler awaits the
token/userinfo requests, two interleaved logins wrote into the SAME objects — a login could
complete carrying another user's id/email/username, updating the wrong user document. The
`PROPAGATE_OIDC_DATA` group/attribute path additionally ran on implicit GLOBALS
(`teamArray`, `isAdmin`, `user_email`, …) with awaits between assignment and use, so
concurrent logins could write one user's email/teams/admin flag onto another user's document
— real database corruption, matching the "web interface shows different data vs mongodb"
report. All login state is now per-login locals, the login handler compares actual values
(the old username/fullname comparisons compared a string to an object, always true), and a
regression test proves isolation under concurrent logins and fails against the pre-fix code:
`tests/oidcLoginStateIsolation.test.cjs` (11 tests, positive + negative). Thanks to
gerardo-junior and xet7.
- **Snap/Docker: after the MongoDB → FerretDB migration ALL CollectionFS-era attachments were
missing** ([#6473](https://github.com/wekan/wekan/issues/6473),
client/components/boards/boardsList.jade
+7 -2
View File
@@ -197,14 +197,19 @@ template(name="boardList")
span.board-list-item-name(title="{{boardOpenAndMoveHint}}")
+viewer
= title
//- #5174 / #4825: the per-board allows* flags are
//- resolved server-side into the one-shot tile data
//- (the raw fields are not even published to this
//- page), so the show* helpers below gate the whole
//- block consistently for every board.
unless currentSetting.hideBoardMemberList
if allowsBoardMemberList
if showBoardMemberList _id
.minicard-members
each member in boardMembers _id
a.name
+userAvatar(userId=member noRemove=true)
unless currentSetting.hideCardCounterList
if allowsCardCounterList
if showCardCounterList _id
.minicard-lists.flex.flex-wrap
each list in boardLists _id
.item
client/components/boards/boardsList.js
+31 -25
View File
@@ -132,16 +132,6 @@ function saveWorkspace(workspaceId, { name, icon }) {
}
Template.boardList.helpers({
hideCardCounterList() {
/* Bug Board icons random dance https://github.com/wekan/wekan/issues/4214
return Utils.isMiniScreen() && Session.get('currentBoard'); */
return true;
},
hideBoardMemberList() {
/* Bug Board icons random dance https://github.com/wekan/wekan/issues/4214
return Utils.isMiniScreen() && Session.get('currentBoard'); */
return true;
},
BoardMultiSelection() {
return BoardMultiSelection;
},
@@ -193,6 +183,15 @@ Template.boardList.onCreated(function () {
Meteor.call('getMyShareableGroups', (err, res) => {
if (!err && res) this.shareableGroups.set(res);
});
// #5174 / #4825: per-board tile data (per-list card counts + member ids) for
// the All Boards tiles, computed server-side and fetched exactly ONCE here —
// deliberately NOT inside an autorun and NOT from reactive getLists/getCards
// cursors, so the #4214 "icons random dance" (a reactive re-render loop on
// this page, which is why the old helpers were stubbed out) cannot return.
this.boardTileDataVar = new ReactiveVar({});
Meteor.call('getAllBoardsTileData', (err, res) => {
if (!err && res) this.boardTileDataVar.set(res);
});
let currUser = ReactiveCache.getCurrentUser();
let userLanguage;
if (currUser && currUser.profile) {
@@ -598,25 +597,32 @@ Template.boardList.helpers({
.slice()
.sort((a, b) => (a.title || '').localeCompare(b.title || ''));
},
// #5174 / #4825: the board tiles' per-list card-count line and member avatar
// row. Data comes from the one-shot getAllBoardsTileData method fetch in
// onCreated (see there) — NOT from reactive getLists/getCards cursors, which
// caused the #4214 "icons random dance" and got these helpers stubbed to []
// (hiding the counters/avatars for everyone). The show* helpers resolve the
// per-board opt-in flags (allowsCardCounterList / allowsBoardMemberList,
// board sidebar "Show at All Boards page"), strictly, server-side — so a
// board with the checkbox off (or never set) consistently shows nothing.
boardLists(boardId) {
/* Bug Board icons random dance https://github.com/wekan/wekan/issues/4214
const lists = ReactiveCache.getLists({ 'boardId': boardId, 'archived': false },{sort: ['sort','asc']});
const ret = lists.map(list => {
let cardCount = ReactiveCache.getCards({ 'boardId': boardId, 'listId': list._id }).length;
return `${list.title}: ${cardCount}`;
});
return ret;
*/
return [];
const tile = (Template.instance().boardTileDataVar.get() || {})[boardId];
return (tile && tile.showLists && tile.lists) || [];
},
boardMembers(boardId) {
/* Bug Board icons random dance https://github.com/wekan/wekan/issues/4214
const lists = ReactiveCache.getBoard(boardId)
const boardMembers = lists?.members.map(member => member.userId);
return boardMembers;
*/
return [];
const tile = (Template.instance().boardTileDataVar.get() || {})[boardId];
return (tile && tile.showMembers && tile.memberIds) || [];
},
showCardCounterList(boardId) {
const tile = (Template.instance().boardTileDataVar.get() || {})[boardId];
return !!(tile && tile.showLists && tile.lists && tile.lists.length);
},
showBoardMemberList(boardId) {
const tile = (Template.instance().boardTileDataVar.get() || {})[boardId];
return !!(tile && tile.showMembers && tile.memberIds && tile.memberIds.length);
},
isStarred() {
client/components/forms/forms.css
+19
View File
@@ -298,6 +298,25 @@ textarea::-moz-placeholder {
font-size: clamp(20px, 4vw, 26px);
margin: 0.5vh 1.5vw;
}
/* CJK (Japanese/Chinese/Korean) labels contain no spaces, so browsers may
insert a line break between any two characters. In the narrow add-card /
edit footers this made short labels such as "追加" (Add) and "リンク"
(Link) wrap mid-word. `word-break: keep-all` forbids breaks inside CJK
words while leaving Latin wrapping behavior unchanged, and `flex-wrap`
lets the footer wrap between its links instead of inside them.
See https://github.com/wekan/wekan/issues/4023 */
.edit-controls,
.add-controls {
flex-wrap: wrap;
word-break: keep-all;
}
.edit-controls button[type=submit],
.add-controls button[type=submit] {
white-space: nowrap;
}
.add-controls .quiet {
white-space: nowrap;
}
[type="checkbox"]:not(:checked),
[type="checkbox"]:checked {
position: absolute;
client/components/lists/list.js
+31 -20
View File
@@ -19,9 +19,17 @@ const { calculateIndex } = Utils;
//
// The old per-list "min width / max width / automatic" knobs are gone — the
// rendered width is now a single hard value (see list.jade).
//
// #5659: the default/minimum width and the resolution order live in ONE
// Meteor-free module shared with models/users.js, so every path (member,
// anonymous public-board visitor, fixed mode) falls back to the SAME default
// and all lists render the same width when nothing is customized.
// ---------------------------------------------------------------------------
const DEFAULT_LIST_WIDTH = 272;
const MIN_LIST_WIDTH = 270;
import {
DEFAULT_LIST_WIDTH,
MIN_LIST_WIDTH,
resolveListWidth,
} from '/models/lib/listWidth';
function isPersonalListWidth(boardId) {
const board = ReactiveCache.getBoard(boardId);
@@ -122,25 +130,28 @@ function saveAnonFixedListWidth(boardId, width) {
function effectiveListWidth(list) {
if (!list) return DEFAULT_LIST_WIDTH;
// #5729 In fixed width mode every list renders at the single shared value.
if (isFixedListWidth(list.boardId)) {
return fixedListWidthValue(list.boardId);
// #5659: all inputs are gathered here, but the fallback order and the ONE
// default width live in models/lib/listWidth.js (shared + unit tested).
const fixedEnabled = isFixedListWidth(list.boardId);
const personalMode = isPersonalListWidth(list.boardId);
let personalWidth = null;
if (personalMode) {
const user = ReactiveCache.getCurrentUser();
if (user) {
const widths = user.getListWidths();
personalWidth = widths[list.boardId] && widths[list.boardId][list._id];
} else {
personalWidth = readAnonListWidth(list.boardId, list._id);
}
}
const shared =
typeof list.width === 'number' && list.width >= MIN_LIST_WIDTH
? list.width
: DEFAULT_LIST_WIDTH;
if (!isPersonalListWidth(list.boardId)) {
return shared;
}
const user = ReactiveCache.getCurrentUser();
if (user) {
const widths = user.getListWidths();
const w = widths[list.boardId] && widths[list.boardId][list._id];
return typeof w === 'number' && w >= MIN_LIST_WIDTH ? w : shared;
}
const w = readAnonListWidth(list.boardId, list._id);
return w || shared;
return resolveListWidth({
// #5729 In fixed width mode every list renders at the single shared value.
fixedEnabled,
fixedWidth: fixedEnabled ? fixedListWidthValue(list.boardId) : null,
sharedWidth: list.width,
personalMode,
personalWidth,
});
}
// Can the current user change THIS list's width by dragging?
client/components/lists/listHeader.js
+12 -6
View File
@@ -9,6 +9,13 @@ import { isHexColor, toHex } from '/models/lib/contrastColor';
import { MultiSelection } from '/client/lib/multiSelection';
import { Utils } from '/client/lib/utils';
import { lazyListCardCount } from '/client/lib/lazyCards';
// #5659: single source of truth for the default/minimum list width, shared
// with client/components/lists/list.js and models/users.js.
import {
DEFAULT_LIST_WIDTH,
MIN_LIST_WIDTH,
normalizeListWidth,
} from '/models/lib/listWidth';
let listsColors;
Meteor.startup(() => {
@@ -562,12 +569,12 @@ function readAnonFixedListWidth(boardId) {
if (stored) {
const widths = JSON.parse(stored);
const w = widths[boardId];
if (typeof w === 'number' && w >= 270) return w;
if (typeof w === 'number' && w >= MIN_LIST_WIDTH) return w;
}
} catch (e) {
console.warn('Error reading fixed list width from localStorage:', e);
}
return 272;
return DEFAULT_LIST_WIDTH;
}
function isFixedListWidth(boardId) {
@@ -611,8 +618,7 @@ Template.setListWidthPopup.helpers({
if (isFixedListWidth(list.boardId)) {
return fixedListWidthValue(list.boardId);
}
const shared =
typeof list.width === 'number' && list.width >= 270 ? list.width : 272;
const shared = normalizeListWidth(list.width);
if (!isPersonalListWidth(list.boardId)) {
return shared;
}
@@ -620,7 +626,7 @@ Template.setListWidthPopup.helpers({
if (user) {
const widths = user.getListWidths();
const w = widths[list.boardId] && widths[list.boardId][list._id];
return typeof w === 'number' && w >= 270 ? w : shared;
return normalizeListWidth(w, shared);
}
return shared;
},
@@ -702,7 +708,7 @@ Template.setListWidthPopup.events({
const boardId = list.boardId;
const width = parseInt(tpl.$('.list-width-value').val(), 10);
if (!width || width < 270) {
if (!width || width < MIN_LIST_WIDTH) {
tpl.$('.list-width-error').click();
return;
}
client/components/main/layouts.js
+4 -1
View File
@@ -31,7 +31,10 @@ Template.userFormsLayout.onCreated(function () {
Meteor.call('isOidcRedirectionEnabled', (_, result) => {
if (result) {
AccountsTemplates.options.socialLoginStyle = 'redirect';
options = {
// #5695: this is an ES module, so the body runs in strict mode and
// the previous undeclared `options = {...}` assignment threw a
// ReferenceError, breaking the OIDC_REDIRECTION_ENABLED auto-login.
const options = {
loginStyle: AccountsTemplates.options.socialLoginStyle,
};
Meteor.loginWithOidc(options);
client/components/settings/peopleBody.js
+23 -27
View File
@@ -1171,31 +1171,30 @@ Template.modifyTeamsUsers.events({
selectedEltValue = selectedElt.options[selectedElt.selectedIndex].text;
selectedEltValueId = selectedElt.options[selectedElt.selectedIndex].value;
// #4593: `teams` is a forbidden field for direct client-side Users.update
// (see server/permissions/users.js: only the owner may update, and never
// `teams`), so the previous Users.update() calls here were silently denied
// by the server and the bulk team assignment never persisted — a user
// "added" to a team this way never saw the boards that team is assigned
// to. Use the admin-only `editUser` method instead, which persists the
// change and also grants the user membership of the boards the gained
// team is assigned to.
if(document.getElementById('addAction').checked){
for(let i = 0; i < selectedUserChkBoxUserIds.length; i++){
currentUser = ReactiveCache.getUser(selectedUserChkBoxUserIds[i]);
userTms = currentUser.teams;
if(userTms == undefined || userTms.length == 0){
userTms = [];
// Copy, so the cached minimongo document is not mutated in place.
userTms = (currentUser.teams || []).slice();
currUserTeamIndex = userTms.findIndex(function(t){ return t.teamId == selectedEltValueId});
if(currUserTeamIndex == -1){
userTms.push({
"teamId": selectedEltValueId,
"teamDisplayName": selectedEltValue,
})
}
else if(userTms.length > 0)
{
currUserTeamIndex = userTms.findIndex(function(t){ return t.teamId == selectedEltValueId});
if(currUserTeamIndex == -1){
userTms.push({
"teamId": selectedEltValueId,
"teamDisplayName": selectedEltValue,
});
}
});
}
Users.update(selectedUserChkBoxUserIds[i], {
$set:{
teams: userTms
Meteor.call('editUser', selectedUserChkBoxUserIds[i], { teams: userTms }, (error) => {
if (error) {
console.error('Error updating user teams:', error);
}
});
}
@@ -1203,18 +1202,15 @@ Template.modifyTeamsUsers.events({
else{
for(let i = 0; i < selectedUserChkBoxUserIds.length; i++){
currentUser = ReactiveCache.getUser(selectedUserChkBoxUserIds[i]);
userTms = currentUser.teams;
if(userTms !== undefined || userTms.length > 0)
{
currUserTeamIndex = userTms.findIndex(function(t){ return t.teamId == selectedEltValueId});
if(currUserTeamIndex != -1){
userTms.splice(currUserTeamIndex, 1);
}
userTms = (currentUser.teams || []).slice();
currUserTeamIndex = userTms.findIndex(function(t){ return t.teamId == selectedEltValueId});
if(currUserTeamIndex != -1){
userTms.splice(currUserTeamIndex, 1);
}
Users.update(selectedUserChkBoxUserIds[i], {
$set:{
teams: userTms
Meteor.call('editUser', selectedUserChkBoxUserIds[i], { teams: userTms }, (error) => {
if (error) {
console.error('Error updating user teams:', error);
}
});
}
docs/Login/LDAP.md
+6 -1
View File
@@ -385,7 +385,12 @@ services:
# LDAP_BACKGROUND_SYNC_IMPORT_NEW_USERS :
# example : LDAP_BACKGROUND_SYNC_IMPORT_NEW_USERS=true
- LDAP_BACKGROUND_SYNC_IMPORT_NEW_USERS=false
# LDAP_ENCRYPTION : If using LDAPS
# LDAP_ENCRYPTION : Connection encryption. Accepted values:
# 'true' : LDAPS, TLS from the first byte (usually port 636)
# 'starttls' : connect unencrypted (usually port 389), then upgrade with STARTTLS
# 'false' : no encryption (default; strongly advised against)
# Deprecated but still working: 'ssl' (same as 'true'), 'tls' (same as 'starttls').
# Any other value logs a warning and means no encryption.
# example : LDAP_ENCRYPTION=true
- LDAP_ENCRYPTION=false
# LDAP_CA_CERT : The certification for the LDAPS server
models/lib/boardInvites.js
+75
View File
@@ -0,0 +1,75 @@
'use strict';
// Pure decision helpers for board invitation accept / decline (quit) flows.
// Extracted so the logic is unit-testable in plain Node without Meteor
// (mirrors models/lib/boardSortReorder.js).
//
// #4730 ("Bugs in adding and removing users"):
//
// Case 2 of the issue — "User A declines the invitation but still sees board 1
// in the personal overview" — was caused by the decline handler on the All
// Boards page (client/components/boards/boardsList.js), which calls
// `quitBoard` (deactivates the member entry) and then `acceptInvite` to clear
// the invitation. The old server-side `acceptInvite` unconditionally set
// `members.$.isActive = true`, immediately RE-ACTIVATING the membership that
// `quitBoard` had just deactivated — so the declined board stayed in the
// user's personal overview as an active membership.
//
// The same unconditional re-activation also let any user with a deactivated
// member entry (i.e. someone an admin had removed from the board) re-add
// themselves simply by calling `Meteor.call('acceptInvite', boardId)` — the
// "adding and removing user A again doesn't fix it" half of Case 1.
//
// Fix: activating a membership through acceptInvite is only legitimate while
// the user actually holds an invitation (profile.invitedBoards contains the
// board id). Declining is made robust by having quitBoard clear the pending
// invitation itself, so the follow-up acceptInvite call finds no invitation
// and becomes a no-op. quitBoard also succeeds (instead of throwing
// error-board-notAMember) for a user who holds a stale invitation without a
// member entry — e.g. after an admin removed them — so a stuck invited board
// can always be removed from the personal overview.
// Whether the given user profile holds a pending invitation to boardId.
function isInvitedToBoard(profile, boardId) {
if (!boardId) return false;
const invitedBoards = (profile && profile.invitedBoards) || [];
return Array.isArray(invitedBoards) && invitedBoards.includes(boardId);
}
// Decide what `quitBoard` (leave board / decline invitation) must do.
// - A member quits: deactivate the member entry, and clear any pending
// invitation so a later acceptInvite cannot resurrect the membership.
// - A non-member with a pending (stale) invitation: just clear the invitation
// and report success, so the board disappears from the personal overview.
// - Neither member nor invited: nothing to quit — error.
function planQuitBoard({ isMember, isInvited }) {
if (!isMember && !isInvited) {
return {
ok: false,
removeMember: false,
pullInvite: false,
error: 'error-board-notAMember',
};
}
return {
ok: true,
removeMember: !!isMember,
pullInvite: !!isInvited,
};
}
// Decide what `acceptInvite` must do. Only a user actually holding an
// invitation may (re)activate their member entry; for anyone else the call is
// a no-op (#4730: this is what used to re-activate declined/removed members).
function planAcceptInvite({ isInvited }) {
if (!isInvited) {
return { ok: false, pullInvite: false, activateMember: false };
}
return { ok: true, pullInvite: true, activateMember: true };
}
module.exports = {
isInvitedToBoard,
planQuitBoard,
planAcceptInvite,
};
models/lib/boardTileData.js
+119
View File
@@ -0,0 +1,119 @@
'use strict';
// Pure helpers for the All Boards page board tiles: the per-list card-count
// line ("To Do: 3 Doing: 1 ...") and the board-member avatar row shown on
// each board tile. Extracted so the decision logic is unit-testable in plain
// Node without Meteor (mirrors models/lib/boardSortReorder.js).
//
// Background (#5174 screenshots / #4825 / #4214):
// - The original client helpers computed the counts from reactive
// ReactiveCache.getLists()/getCards() cursors inside the tile helpers,
// which made the All Boards icons re-render in a loop ("icons random
// dance", #4214). They were stubbed out to `return []`, so NO tile ever
// showed its card counts or member avatars again, regardless of the
// Admin Panel settings (hideCardCounterList / hideBoardMemberList) or the
// per-board flags.
// - Before the stubbing, tiles rendered inconsistently (#4825: "Show card
// count per list doesn't deactivate", members never shown): the per-board
// flags (allowsCardCounterList / allowsBoardMemberList, opt-in, default
// false) were evaluated loosely and boards created before the feature
// (#4691) have no field at all.
//
// The fix: a Meteor method (getAllBoardsTileData, server/publications/
// boards.js) computes everything ONCE server-side — list titles + card counts
// via a single grouped query, member ids from the board doc — and the client
// stores the result in a ReactiveVar it sets exactly once (no reactive
// cursors), so #4214 cannot return. These helpers are the pure core:
//
// - showsCardCounterList / showsBoardMemberList: the per-board flags are
// honored STRICTLY (=== true). Disabled or missing flag -> hidden, for
// every board, deterministically (fixes both directions of #4825).
// - buildBoardTileData: folds boards + lists + card counts into the map the
// tiles render from, already gated by those flags so a disabled board
// never even carries the data.
// Whether this board opted in to showing the per-list card counter line on
// its All Boards tile. The schema default is false (opt-in via the board
// sidebar "Show card count per list" checkbox); boards created before the
// feature existed have no field, which must mean "off" — not "sometimes on"
// (#4825).
function showsCardCounterList(board) {
return !!(board && board.allowsCardCounterList === true);
}
// Whether this board opted in to showing the member avatar row on its tile
// ("Show Board members avatars" checkbox). Same strict semantics.
function showsBoardMemberList(board) {
return !!(board && board.allowsBoardMemberList === true);
}
// One entry of the tile's card-counter line, e.g. "To Do: 3".
function formatListCount(title, count) {
const n = typeof count === 'number' && count > 0 ? count : 0;
return `${title || ''}: ${n}`;
}
// Fold card docs (only their listId is needed) into { listId: count }. Used
// by the server method as the portable alternative to a $group aggregation.
function countCardsByListId(cards) {
const counts = {};
(cards || []).forEach((card) => {
if (card && card.listId) {
counts[card.listId] = (counts[card.listId] || 0) + 1;
}
});
return counts;
}
// Build the per-board tile data map:
// { [boardId]: { showLists, showMembers, lists: ['Title: N', ...],
// memberIds: ['userId', ...] } }
//
// - boards: board docs with _id, members, allowsCardCounterList,
// allowsBoardMemberList.
// - lists: list docs with _id, boardId, title, sort (non-archived lists of
// the boards that show counters). Rendered in board order (sort asc, title
// as tiebreaker), like the lists on the board itself.
// - cardCountByListId: { listId: count } for non-archived cards.
//
// Boards whose flag is off (or missing) get empty arrays, so the tile shows
// nothing for them even if a caller forgets to check the show* flags.
function buildBoardTileData(boards, lists, cardCountByListId) {
const counts = cardCountByListId || {};
const tiles = {};
(boards || []).forEach((board) => {
if (!board || !board._id) return;
const showMembers = showsBoardMemberList(board);
tiles[board._id] = {
showLists: showsCardCounterList(board),
showMembers,
lists: [],
memberIds: showMembers
? (board.members || [])
.filter((m) => m && m.userId && m.isActive !== false)
.map((m) => m.userId)
: [],
};
});
const orderedLists = (lists || []).slice().sort((a, b) => {
const sa = a && typeof a.sort === 'number' ? a.sort : 0;
const sb = b && typeof b.sort === 'number' ? b.sort : 0;
if (sa !== sb) return sa - sb;
return String((a && a.title) || '').localeCompare(String((b && b.title) || ''));
});
orderedLists.forEach((list) => {
if (!list || !list.boardId || !list._id) return;
const tile = tiles[list.boardId];
if (!tile || !tile.showLists) return;
tile.lists.push(formatListCount(list.title, counts[list._id]));
});
return tiles;
}
module.exports = {
showsCardCounterList,
showsBoardMemberList,
formatListCount,
countCardsByListId,
buildBoardTileData,
};
models/lib/invitationCodeEmail.js
+88
View File
@@ -0,0 +1,88 @@
'use strict';
// Pure helpers for the "invite by email" flow (Meteor method `sendInvitation`
// in server/models/settings.js). Extracted so the rules that guarantee "an
// invitation email always carries a code that will validate at sign-up" are
// unit-testable without Meteor.
//
// #4043 ("Sometimes user get board invitation by mail without invitation
// code"): recipients received an invitation email whose code the sign-up form
// then rejected with "The invitation code doesn't exist". Sign-up only accepts
// a code whose stored document matches { code, email, valid: true }
// (server/models/users.js onCreateUser), so the SENDING side must guarantee:
// 1. the stored email is lowercase — the sign-up form lowercases the typed
// address and MongoDB string matching is case sensitive, so a mixed-case
// invitation document can never match (normalizeInviteEmail);
// 2. a RE-INVITE never re-sends a stale code: when the stored invitation is
// no longer valid (e.g. consumed and left with valid:false) or has no
// usable code, a fresh code must overwrite the old one with `valid`
// restored to true (buildReinviteModifier) — this is the fix proposed in
// the #4043 thread ("send a new invitation code that overwrites the last
// one");
// 3. an email is only sent for an invitation document that exists, has a
// non-empty string code, and is still valid (isInvitationCodeSendable) —
// otherwise fail loudly instead of mailing a dead code;
// 4. a failed SMTP send may only roll back (delete) an invitation code that
// was created by this very invite; deleting a PRE-EXISTING document would
// silently invalidate the code already delivered in an earlier email
// (shouldRemoveInvitationOnEmailFailure).
// Lowercase + trim an invitee address so the stored invitation email always
// matches the (lowercased) address the user signs up with. Non-strings
// normalize to '' (callers skip empty results).
function normalizeInviteEmail(email) {
if (typeof email !== 'string') {
return '';
}
return email.trim().toLowerCase();
}
// True when `code` is a non-empty string — the only shape the sign-up lookup
// { code: <string from the form>, valid: true } can ever match. Legacy or
// hand-inserted documents may carry a Number code, which MongoDB will never
// match against the form's string.
function hasUsableCode(invitation) {
return Boolean(
invitation &&
typeof invitation.code === 'string' &&
invitation.code.length > 0,
);
}
// An invitation email may only be sent for a document that exists, carries a
// usable code, and has not been invalidated (valid === false). A missing
// `valid` field (legacy documents predating the field's defaultValue) counts
// as valid.
function isInvitationCodeSendable(invitation) {
return hasUsableCode(invitation) && invitation.valid !== false;
}
// Build the Mongo modifier for re-inviting an email that already has an
// invitation document. Always refreshes boardsToBeInvited; additionally
// overwrites the code (and restores valid:true) when the stored one could
// never validate at sign-up — stale (valid:false), missing, empty, or not a
// string. `generateCode` supplies the new code (coerced to String so it
// matches the form input type regardless of the generator).
function buildReinviteModifier(invitation, boards, generateCode) {
const $set = { boardsToBeInvited: boards };
if (!invitation || invitation.valid === false || !hasUsableCode(invitation)) {
$set.code = String(generateCode());
$set.valid = true;
}
return { $set };
}
// After an SMTP failure, only an invitation created by this very invite call
// may be deleted (nothing was ever delivered for it). A pre-existing document
// must survive, because its code was already delivered in an earlier email.
function shouldRemoveInvitationOnEmailFailure({ isNewInvitation } = {}) {
return isNewInvitation === true;
}
module.exports = {
normalizeInviteEmail,
hasUsableCode,
isInvitationCodeSendable,
buildReinviteModifier,
shouldRemoveInvitationOnEmailFailure,
};
models/lib/listWidth.js
+84
View File
@@ -0,0 +1,84 @@
'use strict';
// #5659 "List Width settings do not affect Public Board".
//
// The default list width used to be duplicated (and DISAGREE) across the
// width-resolution paths:
// - client/components/lists/list.js -> 272 (DEFAULT_LIST_WIDTH)
// - client/components/lists/listHeader.js -> 272/270 hardcoded literals
// - models/lists.js (schema defaultValue) -> 272
// - models/users.js getListWidth() -> 270
// - models/users.js getListWidthFromStorage() (anonymous/public-board
// storage path) -> 270
// - models/users.js getFixedListWidth() -> 272
//
// Depending on which path resolved a given list's width (logged-in member
// profile, anonymous localStorage on a public board, the shared lists.width,
// or fixed-width mode), the fallback "default" differed, so lists on the SAME
// board could render at different widths even though nobody had customized
// anything — most visibly for logged-out visitors of public boards.
//
// This module is the single source of truth. It is Meteor-free so it can be
// unit tested in plain Node (see tests/listWidthDefaults.test.cjs) and is
// imported by both the client width helpers and the user model helpers.
// The one default width every list renders at when no customization exists.
const DEFAULT_LIST_WIDTH = 272;
// The minimum width a customized list may have (matches the resize handle,
// the Set-Width popup validation and the server-side setFixedListWidth check).
const MIN_LIST_WIDTH = 270;
// A width "counts" (may override the default) only when it is a real, finite
// number within the allowed range.
function isValidListWidth(width) {
return (
typeof width === 'number' &&
Number.isFinite(width) &&
width >= MIN_LIST_WIDTH
);
}
// Normalize a stored width: return it when valid, otherwise the fallback
// (the board-wide default unless the caller supplies another).
function normalizeListWidth(width, fallback = DEFAULT_LIST_WIDTH) {
return isValidListWidth(width) ? width : fallback;
}
// Resolve the width ONE list renders at, given the viewer's context:
// - fixedEnabled / fixedWidth : the viewer's per-board "same width for all
// lists" mode (#5729). When on, every list uses the single fixedWidth.
// - sharedWidth : lists.width — the board-wide width shared by
// all viewers (#6409), including logged-out visitors of public boards.
// - personalMode : board.allowsPersonalListWidth.
// - personalWidth : the viewer's own width for this list
// (profile for logged-in users, localStorage for anonymous ones). Only
// consulted in personal mode.
//
// With NO customization at all this always returns DEFAULT_LIST_WIDTH, for
// every list and every viewer — which is what keeps all lists of a public
// board the same width by default.
function resolveListWidth(options) {
const {
fixedEnabled = false,
fixedWidth = null,
sharedWidth = null,
personalMode = false,
personalWidth = null,
} = options || {};
if (fixedEnabled) {
return normalizeListWidth(fixedWidth);
}
const shared = normalizeListWidth(sharedWidth);
if (!personalMode) {
return shared;
}
return normalizeListWidth(personalWidth, shared);
}
module.exports = {
DEFAULT_LIST_WIDTH,
MIN_LIST_WIDTH,
isValidListWidth,
normalizeListWidth,
resolveListWidth,
};
models/lib/teamBoardMemberSync.js
+108
View File
@@ -0,0 +1,108 @@
'use strict';
// Pure decision logic for keeping board memberships in sync when a user's
// teams change. Extracted so it is unit-testable without Meteor
// (see tests/teamBoardMemberSync.test.cjs).
//
// #4593 ("The new added user that was assigned to existing team cannot be
// applied to kanban with the right authority"): when a team is assigned to a
// board through the board sidebar (addBoardTeamPopup), every user who is in
// the team AT THAT MOMENT is pushed into board.members as a normal member.
// But a user added to the team AFTERWARDS never got a member entry: the
// publications grant them read visibility of the board (boards / board
// publications match `teams.teamId`), yet every authority check that gates
// interaction — board.hasMember() (models/boards.js), allowIsBoardMember*()
// (server/lib/utils.js), Attachments.protected (models/attachments.server.js)
// and board.isVisibleBy() (export) — only looks at board.members. So the new
// team member could at best view the board read-only, could not download its
// attachments, and had strictly less authority than teammates who happened to
// be in the team when it was assigned.
//
// These helpers compute, from a user's old and new team lists, which boards
// must gain the user as a member so a late-joining team member ends up with
// the same authority the team's original members received.
// Return the de-duplicated teamId strings of a user.teams-shaped array
// (`[{ teamId, teamDisplayName }, ...]`). Tolerates a missing/invalid list and
// entries without a teamId.
function teamIdsOf(teams) {
if (!Array.isArray(teams)) return [];
const ids = [];
const seen = new Set();
teams.forEach(team => {
const id = team && team.teamId;
if (!id || typeof id !== 'string' || seen.has(id)) return;
seen.add(id);
ids.push(id);
});
return ids;
}
// Team ids present in newTeams but not in oldTeams — the teams the user just
// gained. Removals are intentionally NOT reported: a user removed from a team
// may also have been invited to a board individually, and membership data
// cannot tell those apart, so team-loss cleanup stays the explicit board-admin
// action it always was (removeBoardTeamPopup).
function gainedTeamIds(oldTeams, newTeams) {
const before = new Set(teamIdsOf(oldTeams));
return teamIdsOf(newTeams).filter(id => !before.has(id));
}
// The member entry a team-derived user receives: a plain active, non-admin
// member, exactly like the entries addBoardTeamPopup creates and with the
// same fully-defaulted flags reconcileBoardTeamMembers assigns to members
// introduced through the team-management path.
function newTeamBoardMemberEntry(userId) {
return {
userId,
isAdmin: false,
isActive: true,
isNoComments: false,
isCommentOnly: false,
isWorker: false,
isNormalAssignedOnly: false,
isCommentAssignedOnly: false,
isReadOnly: false,
isReadAssignedOnly: false,
};
}
// Given the candidate board documents (each with at least _id, type, teams,
// members), the user id and the team ids the user just gained, return the ids
// of the boards the user must be added to as a member:
// - the board must have one of the gained teams assigned AND active,
// - template boards/containers are skipped (#5850: sharing on those is
// group-only; the members tab shows only the creator),
// - a user with ANY existing member entry (even inactive) is left untouched,
// so an explicit per-board decision (e.g. a deactivated member) is never
// silently overridden by a team change.
function boardsToAddMemberTo(boards, userId, gainedIds) {
if (!userId || typeof userId !== 'string') return [];
if (!Array.isArray(boards) || !Array.isArray(gainedIds) || !gainedIds.length) {
return [];
}
const gained = new Set(gainedIds);
const ids = [];
boards.forEach(board => {
if (!board || !board._id) return;
if (board.type === 'template-board' || board.type === 'template-container') {
return;
}
const teams = Array.isArray(board.teams) ? board.teams : [];
const sharedWithGainedTeam = teams.some(
team => team && team.isActive === true && gained.has(team.teamId),
);
if (!sharedWithGainedTeam) return;
const members = Array.isArray(board.members) ? board.members : [];
if (members.some(member => member && member.userId === userId)) return;
ids.push(board._id);
});
return ids;
}
module.exports = {
teamIdsOf,
gainedTeamIds,
newTeamBoardMemberEntry,
boardsToAddMemberTo,
};
models/users.js
+17 -7
View File
@@ -5,6 +5,14 @@ import InviteToBoardRolesSettings, {
INVITE_TO_BOARD_ROLES_ID,
INVITE_TO_BOARD_ROLES_DEFAULT,
} from '/models/inviteToBoardRolesSettings';
// #5659: the default/minimum list width must be the SAME on every resolution
// path (it used to be 270 here but 272 in the client and the lists schema, so
// lists could fall back to different "defaults" on public boards).
import {
DEFAULT_LIST_WIDTH,
MIN_LIST_WIDTH,
normalizeListWidth,
} from '/models/lib/listWidth';
// import { Index, MongoDBEngine } from 'meteor/easy:search'; // Temporarily disabled due to compatibility issues
const { SimpleSchema } = require('/imports/simpleSchema');
const Users = Meteor.users;
@@ -1077,8 +1085,7 @@ Users.helpers({
// #5729 The single width applied to every list when fixed width mode is on.
getFixedListWidth(boardId) {
const { fixedListWidths = {} } = this.profile || {};
const w = fixedListWidths[boardId];
return typeof w === 'number' && w >= 270 ? w : 272;
return normalizeListWidth(fixedListWidths[boardId]);
},
invitedBoards() {
@@ -1168,9 +1175,11 @@ Users.helpers({
getListWidth(boardId, listId) {
const listWidths = this.getListWidths();
if (listWidths[boardId] && listWidths[boardId][listId]) {
return listWidths[boardId][listId];
// #5659: normalize so an out-of-range stored value can not make one
// list's width differ from the shared default everyone else sees.
return normalizeListWidth(listWidths[boardId][listId]);
} else {
return 270; //TODO(mark-i-m): default?
return DEFAULT_LIST_WIDTH;
}
},
getListConstraints() {
@@ -1451,7 +1460,7 @@ Users.helpers({
if (widths[boardId] && widths[boardId][listId]) {
const width = widths[boardId][listId];
// Validate it's a valid number
if (validators.isValidNumber(width, 270, 1000)) {
if (validators.isValidNumber(width, MIN_LIST_WIDTH, 1000)) {
return width;
}
}
@@ -1460,7 +1469,8 @@ Users.helpers({
}
}
return 270; // Return default width
// #5659: same default as every other width-resolution path.
return DEFAULT_LIST_WIDTH;
},
setListWidthToStorage(boardId, listId, width) {
@@ -1470,7 +1480,7 @@ Users.helpers({
}
// Validate width before storing
if (!validators.isValidNumber(width, 270, 1000)) {
if (!validators.isValidNumber(width, MIN_LIST_WIDTH, 1000)) {
console.warn('Invalid list width:', width);
return false;
}
package.json
+1 -1
View File
@@ -25,7 +25,7 @@
},
"scripts": {
"test": "meteor test --once --driver-package meteortesting:mocha",
"test:unit:node": "node tests/filenameSanitizer.test.cjs && node tests/cardBoardConsistency.test.cjs && node tests/cardUrl.test.cjs && node tests/defaultSwimlane.test.cjs && node tests/cardMoveModifier.test.cjs && node tests/authenticationMethod.test.cjs && node tests/linkedCardTarget.test.cjs && node tests/staleWhileRevalidate.test.cjs && node tests/restArrayParam.test.cjs && node tests/minicardLabelHeight.test.cjs && node tests/dateUtils.normalizeDigits.test.cjs && node tests/reconcileBoardTeamMembers.test.cjs && node tests/rulesBoardSelector.test.cjs && node tests/ruleActionResolve.test.cjs && node tests/boardBackground.test.cjs && node tests/weekStart.test.cjs && node tests/ldapUsernameSlug.test.cjs && node tests/ldapGroupAllowlist.test.cjs && node tests/htmljsArrayContent.test.cjs && node tests/cardDescriptionDraft.test.cjs && node tests/commentDraft.test.cjs && node tests/attachmentDeleteGuard.test.cjs && node tests/ruleMoveAction.test.cjs && node tests/snapMigrationRecovery.test.cjs && node tests/migrationAttachmentExtraction.test.cjs && node tests/attachmentStoragePath.test.cjs && node tests/cpuExec.test.cjs && node tests/cpuExecWiring.test.cjs && node tests/ferretdbHasData.test.cjs && node tests/ferretdbPolling.test.cjs && node tests/uiDensity.test.cjs && node tests/editorSubmitKey.test.cjs && node tests/isoWeek.test.cjs && node tests/boardCopyDefaults.test.cjs && node tests/linkedCardTitle.test.cjs && node tests/commentActivity.test.cjs && node tests/swimlaneSort.test.cjs && node tests/subtaskCopy.test.cjs && node tests/subtasksDefaultBoard.test.cjs && node tests/scrollbarCss.test.cjs && node tests/minicardBidiIsolation.test.cjs && node tests/notificationCleanup.test.cjs && node tests/cardSearch.test.cjs && node tests/linkedCardMembers.test.cjs && node tests/verificationEmail.test.cjs && node tests/boardSortReorder.test.cjs && node tests/checklistItemTitles.test.cjs && node tests/swimlaneFilter.test.cjs && node tests/popupOffset.test.cjs && node tests/linkedCardCover.test.cjs && node tests/inviteEmailLanguage.test.cjs && node tests/domainTablePage.test.cjs && node tests/mongoSelectorSafety.test.cjs && node tests/cardsLoading.test.cjs && node tests/base64Chunk.test.cjs && node tests/backupPaths.test.cjs && node tests/backupCron.test.cjs && node tests/avatarUrlSafety.test.cjs && node tests/importedUserReconciliationPlan.test.cjs && node tests/ldapPhoto.test.cjs && node tests/importedBoardMemberPlan.test.cjs && node tests/unit/resetPasswordEmail.test.js",
"test:unit:node": "node tests/filenameSanitizer.test.cjs && node tests/cardBoardConsistency.test.cjs && node tests/cardUrl.test.cjs && node tests/defaultSwimlane.test.cjs && node tests/cardMoveModifier.test.cjs && node tests/authenticationMethod.test.cjs && node tests/linkedCardTarget.test.cjs && node tests/staleWhileRevalidate.test.cjs && node tests/restArrayParam.test.cjs && node tests/minicardLabelHeight.test.cjs && node tests/dateUtils.normalizeDigits.test.cjs && node tests/reconcileBoardTeamMembers.test.cjs && node tests/rulesBoardSelector.test.cjs && node tests/ruleActionResolve.test.cjs && node tests/boardBackground.test.cjs && node tests/weekStart.test.cjs && node tests/ldapUsernameSlug.test.cjs && node tests/ldapGroupAllowlist.test.cjs && node tests/htmljsArrayContent.test.cjs && node tests/cardDescriptionDraft.test.cjs && node tests/commentDraft.test.cjs && node tests/attachmentDeleteGuard.test.cjs && node tests/ruleMoveAction.test.cjs && node tests/snapMigrationRecovery.test.cjs && node tests/migrationAttachmentExtraction.test.cjs && node tests/attachmentRepair.test.cjs && node tests/attachmentStoragePath.test.cjs && node tests/schemaUpgradeSteps.test.cjs && node tests/cpuExec.test.cjs && node tests/cpuExecWiring.test.cjs && node tests/ferretdbHasData.test.cjs && node tests/ferretdbPolling.test.cjs && node tests/uiDensity.test.cjs && node tests/editorSubmitKey.test.cjs && node tests/isoWeek.test.cjs && node tests/boardCopyDefaults.test.cjs && node tests/linkedCardTitle.test.cjs && node tests/commentActivity.test.cjs && node tests/swimlaneSort.test.cjs && node tests/subtaskCopy.test.cjs && node tests/subtasksDefaultBoard.test.cjs && node tests/scrollbarCss.test.cjs && node tests/minicardBidiIsolation.test.cjs && node tests/notificationCleanup.test.cjs && node tests/cardSearch.test.cjs && node tests/linkedCardMembers.test.cjs && node tests/verificationEmail.test.cjs && node tests/boardSortReorder.test.cjs && node tests/checklistItemTitles.test.cjs && node tests/swimlaneFilter.test.cjs && node tests/popupOffset.test.cjs && node tests/linkedCardCover.test.cjs && node tests/inviteEmailLanguage.test.cjs && node tests/domainTablePage.test.cjs && node tests/mongoSelectorSafety.test.cjs && node tests/cardsLoading.test.cjs && node tests/base64Chunk.test.cjs && node tests/backupPaths.test.cjs && node tests/backupCron.test.cjs && node tests/avatarUrlSafety.test.cjs && node tests/importedUserReconciliationPlan.test.cjs && node tests/ldapPhoto.test.cjs && node tests/importedBoardMemberPlan.test.cjs && node tests/unit/resetPasswordEmail.test.js",
"test:unit:all": "npm run test:unit:node && node tests/cards.stickers.test.js && node tests/stickers.test.js && node tests/stickers.packs.test.js && node tests/linkCardPopup.test.js && node tests/trelloCreator.import.test.js && node tests/trelloImport.maplink.test.js && node tests/trelloImport.zip.test.js && node tests/wekanCreator.import.test.js && node tests/wekanCreator.inconsistent.test.js && node tests/unit/oauth2-admin-groups.test.js",
"test:watch": "meteor test --driver-package meteortesting:mocha",
"test:e2e": "node tests/e2e/list-regressions.js",
packages/wekan-ldap/server/encryptionSetting.js
+87
View File
@@ -0,0 +1,87 @@
'use strict';
// #4158 "Documentation/configuration of LDAP_ENCRYPTION". Historically the
// code only understood two magic strings, and both were misleadingly named:
//
// - 'ssl' meant LDAPS (connect to e.g. port 636, TLS from the first byte),
// which was only discoverable by reading the source;
// - 'tls' meant STARTTLS (connect plaintext, then upgrade), even though the
// wiki claimed 'tls' was "more secure than standard LDAPS" (untrue);
// - the documented value 'true' NEVER worked: settings_get() JSON-parses
// the env string 'true' into boolean true, which matched neither 'ssl'
// nor 'tls', so WeKan silently connected WITHOUT encryption;
// - any other value (typos, 'TRUE', 'StartTLS', ...) also silently meant
// "no encryption".
//
// normalizeLdapEncryption() maps every raw LDAP_ENCRYPTION value onto one of
// three explicit connection modes and, where appropriate, a warning for the
// admin. It never throws and never silently guesses:
//
// raw value (case-insensitive) -> mode warning
// -----------------------------------------------------------------------
// 'true' / boolean true -> 'tls' - (LDAPS)
// 'ssl' (legacy) -> 'tls' deprecation: use 'true'
// 'starttls' -> 'starttls' -
// 'tls' (legacy) -> 'starttls' deprecation: use 'starttls'
// 'false' / boolean false / '' /
// undefined / null -> 'off' -
// anything else -> 'off' clear warning listing the
// accepted values
//
// Backwards compatibility: the legacy values keep their historical meaning
// ('ssl' -> LDAPS, 'tls' -> STARTTLS); they only gain a one-line deprecation
// notice. Unknown values still result in an unencrypted connection (as
// before), but now log loudly instead of failing silent.
//
// The function is dependency-free (no Meteor, no Log) so it can be unit
// tested with plain Node; the caller (ldap.js) decides how to emit warnings.
//
// CommonJS module.exports so it can be `import`ed by the ecmascript package
// and `require`d directly by the plain-Node unit test.
const ACCEPTED_VALUES_HELP =
"accepted values are 'true' (LDAPS), 'starttls' (STARTTLS) and 'false' (no encryption); " +
"the legacy values 'ssl' (same as 'true') and 'tls' (same as 'starttls') also still work";
function normalizeLdapEncryption(value) {
// settings_get() JSON-parses 'true'/'false' env strings into booleans, so
// accept booleans as well as strings.
if (value === true) {
return { mode: 'tls' };
}
if (value === undefined || value === null || value === false) {
return { mode: 'off' };
}
const normalized = String(value).trim().toLowerCase();
switch (normalized) {
case '':
case 'false':
return { mode: 'off' };
case 'true':
return { mode: 'tls' };
case 'ssl':
return {
mode: 'tls',
warning:
"LDAP_ENCRYPTION='ssl' is deprecated, please use LDAP_ENCRYPTION='true' instead (both mean LDAPS; 'ssl' keeps working for now)",
};
case 'starttls':
return { mode: 'starttls' };
case 'tls':
return {
mode: 'starttls',
warning:
"LDAP_ENCRYPTION='tls' is deprecated, please use LDAP_ENCRYPTION='starttls' instead (both mean STARTTLS; 'tls' keeps working for now)",
};
default:
return {
mode: 'off',
warning:
`LDAP_ENCRYPTION='${value}' is not a valid value, connecting WITHOUT encryption; ${ACCEPTED_VALUES_HELP}`,
};
}
}
module.exports = { normalizeLdapEncryption, ACCEPTED_VALUES_HELP };
packages/wekan-ldap/server/ldap.js
+57 -13
View File
@@ -1,5 +1,18 @@
import { Client } from 'ldapts';
import { Log } from 'meteor/logging';
import { normalizeLdapEncryption } from './encryptionSetting';
import { buildUserIdFilter } from './userIdFilter';
// #4158: warn about a deprecated/invalid LDAP_ENCRYPTION value only once per
// distinct message, not on every single login attempt (LDAP instantiates a
// new connection per login).
const encryptionWarningsShown = new Set();
function warnOnceAboutEncryption(warning) {
if (warning && !encryptionWarningsShown.has(warning)) {
encryptionWarningsShown.add(warning);
Log.warn(warning);
}
}
// copied from https://github.com/ldapjs/node-ldapjs/blob/a113953e0d91211eb945d2a3952c84b7af6de41c/lib/filters/index.js#L167
function escapedToHex (str) {
@@ -53,7 +66,10 @@ export default class LDAP {
timeout : this.constructor.settings_get('LDAP_TIMEOUT'),
connect_timeout : this.constructor.settings_get('LDAP_CONNECT_TIMEOUT'),
idle_timeout : this.constructor.settings_get('LDAP_IDLE_TIMEOUT'),
encryption : this.constructor.settings_get('LDAP_ENCRYPTION'),
// #4158: normalized to 'tls' (LDAPS), 'starttls' or 'off'. Accepts the
// documented 'true'/'starttls'/'false' plus the legacy 'ssl'/'tls'
// (deprecated but still working); anything else warns and means 'off'.
encryption : normalizeLdapEncryption(this.constructor.settings_get('LDAP_ENCRYPTION')).mode,
ca_cert : this.constructor.settings_get('LDAP_CA_CERT'),
reject_unauthorized : this.constructor.settings_get('LDAP_REJECT_UNAUTHORIZED') !== undefined ? this.constructor.settings_get('LDAP_REJECT_UNAUTHORIZED') : true,
Authentication : this.constructor.settings_get('LDAP_AUTHENTIFICATION'),
@@ -117,8 +133,14 @@ export default class LDAP {
tlsOptions.ca = ca;
}
// #4158: surface a deprecation notice for legacy values ('ssl', 'tls')
// and a clear warning for unknown values instead of failing silently.
warnOnceAboutEncryption(
normalizeLdapEncryption(this.constructor.settings_get('LDAP_ENCRYPTION')).warning,
);
let url;
if (this.options.encryption === 'ssl') {
if (this.options.encryption === 'tls') {
url = `ldaps://${this.options.host}:${this.options.port}`;
} else {
url = `ldap://${this.options.host}:${this.options.port}`;
@@ -133,7 +155,7 @@ export default class LDAP {
strictDN : false,
};
if (this.options.encryption === 'ssl') {
if (this.options.encryption === 'tls') {
clientOptions.tlsOptions = tlsOptions;
}
@@ -141,7 +163,7 @@ export default class LDAP {
this.client = new Client(clientOptions);
if (this.options.encryption === 'tls') {
if (this.options.encryption === 'starttls') {
// Set host parameter for tls.connect which is used by starttls. This shouldn't be needed in newer nodejs versions (e.g v5.6.0).
// https://github.com/RocketChat/Rocket.Chat/issues/2035
tlsOptions.host = this.options.host;
@@ -322,16 +344,24 @@ export default class LDAP {
async getUserById(id, attribute) {
await this.bindIfNecessary();
const Unique_Identifier_Field = this.constructor.settings_get('LDAP_UNIQUE_IDENTIFIER_FIELD').split(',');
const escapedValue = hexToLdapEscaped(id);
let filter;
if (attribute) {
filter = `(${attribute}=${escapedValue})`;
} else {
const filters = Unique_Identifier_Field.map((item) => `(${item}=${escapedValue})`);
filter = `(|${filters.join('')})`;
// #4654: the stored services.ldap.id may come from LDAP_UNIQUE_IDENTIFIER_FIELD
// or, when that is unset/empty, from LDAP_USER_SEARCH_FIELD (see
// getLdapUserUniqueID in sync.js), so search over both attribute lists.
// The old code read only LDAP_UNIQUE_IDENTIFIER_FIELD and crashed
// (undefined.split) or built the invalid filter "(|(=value))" when it was
// unset/empty, so background sync never updated existing users.
const filter = buildUserIdFilter(
attribute,
escapedValue,
this.constructor.settings_get('LDAP_UNIQUE_IDENTIFIER_FIELD'),
this.constructor.settings_get('LDAP_USER_SEARCH_FIELD'),
);
if (!filter) {
Log.error('Can\'t search user by id: neither LDAP_UNIQUE_IDENTIFIER_FIELD nor LDAP_USER_SEARCH_FIELD is configured');
return;
}
const searchOptions = {
@@ -463,7 +493,21 @@ export default class LDAP {
}
if (this.options.group_filter_group_id_attribute !== '') {
filter.push(`(${this.options.group_filter_group_id_attribute}=${this.options.group_filter_group_name})`);
// #4036: LDAP_GROUP_FILTER_GROUP_NAME accepts a comma-separated list, and
// members of LDAP_SYNC_ADMIN_GROUPS may also log in when admin sync is on
// (previously an admin only in the admin group was locked out entirely).
const names = String(this.options.group_filter_group_name || '')
.split(',').map((s) => s.trim()).filter(Boolean);
if (this.constructor.settings_get('LDAP_SYNC_ADMIN_STATUS') === true) {
names.push(...String(this.constructor.settings_get('LDAP_SYNC_ADMIN_GROUPS') || '')
.split(',').map((s) => s.trim()).filter(Boolean));
}
const clauses = names.map((n) => `(${this.options.group_filter_group_id_attribute}=${n})`);
if (clauses.length === 1) {
filter.push(clauses[0]);
} else if (clauses.length > 1) {
filter.push(`(|${clauses.join('')})`);
}
}
filter.push(')');
packages/wekan-ldap/server/sync.js
+8 -3
View File
@@ -246,7 +246,10 @@ export async function syncUserData(user, ldapUser) {
const username = slug(getLdapUsername(ldapUser));
if (user && user._id && username !== user.username) {
log_info('Syncing user username', user.username, '->', username);
await Meteor.users.findOneAsync({ _id: user._id }, { $set: { username }});
// #4654: this was findOne/findOneAsync with the $set passed as the
// options argument, which performs no write — username changes in the
// directory were logged as synced but never saved.
await Meteor.users.updateAsync({ _id: user._id }, { $set: { username }});
}
}
@@ -338,10 +341,12 @@ export async function addLdapUser(ldapUser, username, password) {
userObject.ldap = true;
userObject._id = await Accounts.createUserAsync(userObject);
// Add the services.ldap identifiers
// Add the services.ldap identifiers. #4654: also persist idAttribute so
// the background sync (getUserById in ldap.js) can search by the exact
// attribute the id was taken from instead of guessing from settings.
await Meteor.users.updateAsync({ _id: userObject._id }, {
$set: {
'services.ldap': { id: uniqueId.value },
'services.ldap': { id: uniqueId.value, idAttribute: uniqueId.attribute },
'emails.0.verified': true,
'authenticationMethod': 'ldap',
}});
packages/wekan-ldap/server/userIdFilter.js
+49
View File
@@ -0,0 +1,49 @@
'use strict';
// #4654: build the LDAP search filter used to re-find an existing user by the
// stored services.ldap.id during background sync
// (LDAP_BACKGROUND_SYNC_KEEP_EXISTANT_USERS_UPDATED).
//
// The stored id is produced by getLdapUserUniqueID() in sync.js, which takes
// the first non-empty attribute from LDAP_UNIQUE_IDENTIFIER_FIELD and falls
// back to LDAP_USER_SEARCH_FIELD. The lookup therefore has to consider both
// settings too. Previously getUserById() used only LDAP_UNIQUE_IDENTIFIER_FIELD:
// when that variable was unset it crashed (undefined.split), aborting the whole
// sync run, and when it was set but empty it produced the invalid filter
// "(|(=value))" seen in the issue logs — either way existing users were never
// updated after the first import ("Can't sync user").
// Split a comma-separated attribute list setting into non-empty names.
// Mirrors the whitespace handling of getLdapUserUniqueID()/getBufferAttributes().
function splitFieldSetting(value) {
if (typeof value !== 'string' || value === '') {
return [];
}
return value.replace(/\s/g, '').split(',').filter(Boolean);
}
// attribute: the stored services.ldap.idAttribute when known (searched as-is);
// escapedValue: the id value already escaped for use inside an LDAP filter;
// uniqueIdentifierFieldSetting / userSearchFieldSetting: the raw settings values.
// Returns the filter string, or null when no usable attribute is configured.
function buildUserIdFilter(attribute, escapedValue, uniqueIdentifierFieldSetting, userSearchFieldSetting) {
if (attribute) {
return `(${attribute}=${escapedValue})`;
}
const fields = splitFieldSetting(uniqueIdentifierFieldSetting)
.concat(splitFieldSetting(userSearchFieldSetting));
const uniqueFields = [...new Set(fields)];
if (uniqueFields.length === 0) {
return null;
}
const filters = uniqueFields.map((field) => `(${field}=${escapedValue})`);
return filters.length === 1 ? filters[0] : `(|${filters.join('')})`;
}
module.exports = {
splitFieldSetting,
buildUserIdFilter,
};
packages/wekan-oidc/loginHandler.js
+38 -32
View File
@@ -3,8 +3,7 @@
// objString = ["Org","Team"] for method mapping
async function createObject(initArr, objString)
{
functionName = objString === "Org" ? 'setCreateOrgFromOidc' : 'setCreateTeamFromOidc';
creationString = 'setCreate'+ objString + 'FromOidc';
const functionName = objString === "Org" ? 'setCreateOrgFromOidc' : 'setCreateTeamFromOidc';
return await Meteor.callAsync(functionName,
initArr[0],//displayName
initArr[1],//desc
@@ -15,7 +14,7 @@ async function createObject(initArr, objString)
}
async function updateObject(initArr, objString)
{
functionName = objString === "Org" ? 'setOrgAllFieldsFromOidc' : 'setTeamAllFieldsFromOidc';
const functionName = objString === "Org" ? 'setOrgAllFieldsFromOidc' : 'setTeamAllFieldsFromOidc';
return await Meteor.callAsync(functionName,
initArr[0],//team || org Object
initArr[1],//displayName
@@ -32,7 +31,7 @@ async function updateObject(initArr, objString)
//e.g. collection = "team"
function contains(userObjs, obj, collection)
{
id = collection+'Id';
const id = collection+'Id';
if(typeof userObjs == "undefined" || !userObjs.length)
{
@@ -98,26 +97,33 @@ oauth2AdminStatusFromGroups: function (groups)
// isAdmin: [true, false] -> admin group becomes admin in wekan
// isOrganization: [true, false] -> creates org and adds to user
// displayName: "string"
// #4897: every variable in here used to be an implicit GLOBAL (teamArray,
// orgArray, isAdmin, teams, orgs, group, org, team, ...) shared by all
// concurrent OIDC logins. This function awaits inside its loop, so two logins
// interleaved and pushed each other's teams/orgs/admin flags onto the same
// arrays before writing them to the wrong user document. All state is local
// now, and the collection is addressed as Meteor.users instead of relying on
// a `users` global leaked by the caller.
addGroupsWithAttributes: async function (user, groups){
teamArray=[];
orgArray=[];
isAdmin = [];
teams = user.teams;
orgs = user.orgs;
for (group of groups)
const teamArray=[];
const orgArray=[];
const isAdmin = [];
const teams = user.teams;
const orgs = user.orgs;
for (const group of groups)
{
initAttributes = [
const initAttributes = [
group.displayName,
group.desc || group.displayName,
group.shortName ||group.displayName,
group.website || group.displayName, group.isActive || false];
isOrg = group.isOrganisation || false;
forceCreate = group.forceCreate|| false;
const isOrg = group.isOrganisation || false;
const forceCreate = group.forceCreate|| false;
isAdmin.push(group.isAdmin || false);
if (isOrg)
{
org = await Org.findOneAsync({"orgDisplayName": group.displayName});
let org = await Org.findOneAsync({"orgDisplayName": group.displayName});
if(org)
{
if(contains(orgs, org, "org"))
@@ -136,14 +142,14 @@ addGroupsWithAttributes: async function (user, groups){
{
continue;
}
orgHash = {'orgId': org._id, 'orgDisplayName': group.displayName};
const orgHash = {'orgId': org._id, 'orgDisplayName': group.displayName};
orgArray.push(orgHash);
}
else
{
//start team routine
team = await Team.findOneAsync({"teamDisplayName": group.displayName});
let team = await Team.findOneAsync({"teamDisplayName": group.displayName});
if (team)
{
if(contains(teams, team, "team"))
@@ -162,7 +168,7 @@ addGroupsWithAttributes: async function (user, groups){
{
continue;
}
teamHash = {'teamId': team._id, 'teamDisplayName': group.displayName};
const teamHash = {'teamId': team._id, 'teamDisplayName': group.displayName};
teamArray.push(teamHash);
}
}
@@ -170,32 +176,33 @@ addGroupsWithAttributes: async function (user, groups){
// hence user will get admin privileges in wekan
// E.g. Admin rights will be withdrawn if no group in oidc provider has isAdmin set to true
await users.updateAsync({ _id: user._id }, { $set: {isAdmin: isAdmin.some(i => (i === true))}});
teams = {'teams': {'$each': teamArray}};
orgs = {'orgs': {'$each': orgArray}};
await users.updateAsync({ _id: user._id }, { $push: teams});
await users.updateAsync({ _id: user._id }, { $push: orgs});
await Meteor.users.updateAsync({ _id: user._id }, { $set: {isAdmin: isAdmin.some(i => (i === true))}});
await Meteor.users.updateAsync({ _id: user._id }, { $push: {'teams': {'$each': teamArray}}});
await Meteor.users.updateAsync({ _id: user._id }, { $push: {'orgs': {'$each': orgArray}}});
// remove temporary oidc data from user collection
await users.updateAsync({ _id: user._id }, { $unset: {"services.oidc.groups": []}});
await Meteor.users.updateAsync({ _id: user._id }, { $unset: {"services.oidc.groups": []}});
return;
},
changeUsername: async function(user, name)
{
username = {'username': name};
if (user.username != username) await users.updateAsync({ _id: user._id }, { $set: username});
// #4897: `username` was an implicit global; use a local and update via
// Meteor.users so this no longer depends on a leaked `users` global.
if (user.username !== name) await Meteor.users.updateAsync({ _id: user._id }, { $set: {'username': name}});
},
changeFullname: async function(user, name)
{
username = {'profile.fullname': name};
if (user.username != username) await users.updateAsync({ _id: user._id }, { $set: username});
if ((user.profile || {}).fullname !== name) await Meteor.users.updateAsync({ _id: user._id }, { $set: {'profile.fullname': name}});
},
addEmail: async function(user, email)
{
user_email = user.emails || [];
var contained = false;
position = 0;
// #4897: `user_email` and `position` were implicit globals shared between
// concurrent logins — one login's email list could be spliced/unshifted by
// another login before being $set on this user's document.
let user_email = user.emails || [];
let contained = false;
let position = 0;
for (const [count, mail_hash] of Object.entries(user_email))
{
if (mail_hash['address'] === email)
@@ -213,8 +220,7 @@ addEmail: async function(user, email)
if(!contained)
{
user_email.unshift({'address': email, 'verified': true});
user_email = {'emails': user_email};
await users.updateAsync({ _id: user._id }, { $set: user_email});
await Meteor.users.updateAsync({ _id: user._id }, { $set: {'emails': user_email}});
}
}
}
packages/wekan-oidc/oidc_client.js
+21 -1
View File
@@ -16,9 +16,29 @@ Oidc.requestCredential = function (options, credentialRequestCompleteCallback) {
if (result) {
var config = result;
var credentialToken = Random.secret();
var loginStyle = OAuth._loginStyle('oidc', config, options);
// options
options = options || {};
// Fix #5695: OAUTH2_LOGIN_STYLE=redirect was never honored. The
// useraccounts social-login button always passes
// { loginStyle: 'popup' } (its client-side socialLoginStyle default),
// and OAuth._loginStyle() lets a caller option override the loginStyle
// stored in the service configuration. The admin's OAUTH2_LOGIN_STYLE
// env var is authoritative for redirect: when the service is configured
// with loginStyle 'redirect', drop the generic popup default so the
// full-page redirect flow is used. An explicit
// { loginStyle: 'redirect' } from a caller (the OIDC_REDIRECTION_ENABLED
// auto-login) keeps working regardless of the configured style, and
// popup remains the default when OAUTH2_LOGIN_STYLE is unset or popup.
if (config.loginStyle === 'redirect') {
options.loginStyle = 'redirect';
}
// OAuth._loginStyle still falls back to popup when sessionStorage is
// unavailable (e.g. Safari private mode), which the redirect flow needs.
var loginStyle = OAuth._loginStyle('oidc', config, options);
// loginStyle is a Meteor-internal choice, not an OAuth2/OIDC parameter;
// don't leak it to the provider in the authorization URL built below
// (the URL used to contain "...&loginStyle=popup&...", see #5695).
delete options.loginStyle;
options.client_id = config.clientId;
options.response_type = options.response_type || 'code';
options.redirect_uri = OAuth._redirectUri('oidc', config);
packages/wekan-oidc/oidc_server.js
+21 -7
View File
@@ -18,13 +18,23 @@ if (process.env.OAUTH2_CA_CERT !== undefined) {
console.log(e);
}
}
var profile = {};
var serviceData = {};
var userinfo = {};
OAuth.registerService('oidc', 2, null, async function (query) {
var debug = process.env.DEBUG === 'true';
// #4897: these MUST be fresh per-login objects. They used to be module-level
// (`var serviceData = {}` etc. shared by every login of every user), so:
// * fields a login did not overwrite leaked in from the PREVIOUS login of a
// DIFFERENT user (e.g. refreshToken is only set when the provider sends
// one, and idTokenWhitelistFields were Object.assign-ed cumulatively), and
// * two logins running concurrently (this handler awaits the token,
// userinfo and two Meteor.callAsync calls) interleaved writes to the SAME
// object, so a user could intermittently be created/updated with another
// user's id/email/username — the "web interface shows different data than
// MongoDB" reports.
var profile = {};
var serviceData = {};
var userinfo = {};
var token = await getToken(query);
if (debug) console.log('XXX: register token:', token);
@@ -121,7 +131,8 @@ OAuth.registerService('oidc', 2, null, async function (query) {
// therefore: keep admin privileges for wekan as before
if(Array.isArray(serviceData.groups) && serviceData.groups.length && typeof serviceData.groups[0] === "string" )
{
user = await Meteor.users.findOneAsync({'_id': serviceData.id});
// #4897: `user` was an implicit global shared by concurrent logins.
const user = await Meteor.users.findOneAsync({'_id': serviceData.id});
serviceData.groups.forEach(function(groupName, i)
{
@@ -373,8 +384,11 @@ Meteor.methods({
var propagateOidcData = process.env.PROPAGATE_OIDC_DATA || false;
if (propagateOidcData) {
users= Meteor.users;
user = await users.findOneAsync({'services.oidc.id': userId});
// #4897: `users`/`user` were implicit globals shared by concurrent
// logins; between the awaits below another login could reassign `user`,
// so this login's email/fullname/username got written onto ANOTHER
// user's document. loginHandler.js now uses Meteor.users directly.
const user = await Meteor.users.findOneAsync({'services.oidc.id': userId});
if(user) {
//updates/creates Groups and user admin privileges accordingly if not undefined
releases/migrate-mongodb-to-ferretdb.mjs
+358 -77
View File
@@ -99,6 +99,16 @@ const ATTACH_DIR = path.join(WRITABLE, 'files', 'attachments');
const AVATAR_DIR = path.join(WRITABLE, 'files', 'avatars');
const MARKER_COLL = '_wekan_migration'; // written to target when done
const SCHEMA_VER = 843; // v8.43
// #6473: version of the FILE (attachment/avatar) extraction logic. Bump this whenever a
// file-phase bug fix means already-migrated installs must be RE-SCANNED: a marker whose
// filesVersion is older than this triggers an automatic, incremental files-only REPAIR run
// (text data is left alone; only binaries/records that are missing are migrated). v2 =
// the #6473 fixes (copies.<bucket>.key resolution, files-collection-driven CFS extraction,
// gridFsFileId-without-storage-flag matching, <bucket>.files sweep).
const FILES_VERSION = 2;
// Run ONLY the file phases against an existing target (set by the snap's
// attachment-repair). Also turned on automatically when the marker's filesVersion is old.
const FILES_ONLY = process.env.FILES_ONLY === 'true';
// ── Progress state (shared between HTTP server and migration logic) ─────────
const state = {
@@ -259,9 +269,16 @@ const server = http.createServer((req, res) => {
}
});
// A background repair run (#6473) starts while WeKan itself is coming up on the web
// port, so the dashboard port can be taken (or 0 = ephemeral). Never crash over it —
// the dashboard is a convenience, the migration/repair itself matters.
server.on('error', (err) => {
console.log(`[migrate] Progress dashboard not available (${err.code || err.message}); continuing without it.`);
});
server.listen(PORT, () => {
console.log(`[migrate] Progress dashboard: http://localhost:${PORT}/`);
console.log(`[migrate] JSON status: http://localhost:${PORT}/migration-status`);
const actualPort = server.address()?.port ?? PORT;
console.log(`[migrate] Progress dashboard: http://localhost:${actualPort}/`);
console.log(`[migrate] JSON status: http://localhost:${actualPort}/migration-status`);
});
// ── Filesystem helpers ─────────────────────────────────────────────────────
@@ -614,11 +631,16 @@ function resolveCfsGridFsId(record, bucketName) {
* Each transformer is pure: takes a doc, returns a doc (or null to skip).
*/
function upgradeBoard(doc) {
// Only migrationVersion is stamped. The old comprehensiveMigrationCompleted /
// fixMissingListsCompleted stamps made WeKan's on-demand repair tools
// (fixMissingListsMigration, comprehensiveBoardMigration) permanently skip
// imported boards — while this importer covers LESS than those tools (e.g.
// cards with a dangling listId). The startup schema upgrade
// (server/lib/schemaUpgradeSteps.js) now rescues those on every platform,
// and the repair tools stay usable.
return {
...doc,
migrationVersion: SCHEMA_VER,
comprehensiveMigrationCompleted: true,
fixMissingListsCompleted: true,
};
}
@@ -728,13 +750,23 @@ async function copyCollection(srcDb, tgtDb, collName, transformer) {
const flush = async () => {
if (batch.length === 0) return;
if (!DRY_RUN) {
for (const doc of batch) {
try {
await tgt.replaceOne({ _id: doc._id }, doc, { upsert: true });
state.collections[collName].done++;
} catch (err) {
state.collections[collName].errors++;
pushError(`${collName}/${doc._id}: ${err.message}`);
// SPEED (#6473 follow-up: a big migration took 5 hours): ONE insertMany
// round trip per batch — the common case is a fresh, empty target where
// every insert succeeds. Only when the batch hits duplicates (a resumed
// or re-run migration) fall back to per-document replaceOne upserts for
// THIS batch, which also isolates any per-document error.
try {
await tgt.insertMany(batch, { ordered: false });
state.collections[collName].done += batch.length;
} catch {
for (const doc of batch) {
try {
await tgt.replaceOne({ _id: doc._id }, doc, { upsert: true });
state.collections[collName].done++;
} catch (err) {
state.collections[collName].errors++;
pushError(`${collName}/${doc._id}: ${err.message}`);
}
}
}
} else {
@@ -835,48 +867,67 @@ async function run() {
}
} catch (e) { /* keep default product name */ }
// Check idempotency marker
// Check idempotency marker. #6473: the marker also carries filesVersion — the version
// of the FILE extraction logic that produced the target's attachments/avatars. A
// COMPLETED migration whose filesVersion is older than FILES_VERSION is not re-done:
// it gets an automatic, INCREMENTAL files-only REPAIR — the text data (which may have
// been changed by users since the migration) is never touched, and only binaries and
// records that are actually missing are migrated; everything already on disk and
// pointed at by its record is verified and skipped.
const markerColl = tgtDb.collection(MARKER_COLL);
const marker = await markerColl.findOne({ _id: 'completed' });
const filesCurrent = !!marker && (marker.filesVersion || 1) >= FILES_VERSION;
let repairMode = FILES_ONLY;
if (marker && marker.schemaVersion >= SCHEMA_VER && !process.env.FORCE_MIGRATE) {
console.log('[migrate] Already migrated (schema v' + marker.schemaVersion + '). Exiting.');
state.phase = 'already-migrated';
state.phase_detail = 'Schema v' + marker.schemaVersion + ' already present in target';
state.success = true;
// A previous boot already migrated attachments, avatars and text data:
// reclaim the old MongoDB disk space now (runs once, tracked by the marker).
try { await compactSource(srcDb, markerColl); } catch (e) { pushError('compact: ' + e.message); }
state.finishedAt = new Date().toISOString();
await srcClient.close();
await tgtClient.close();
setTimeout(() => process.exit(0), 3000);
return;
if (filesCurrent && !FILES_ONLY) {
console.log('[migrate] Already migrated (schema v' + marker.schemaVersion + ', files v' + (marker.filesVersion || 1) + '). Exiting.');
state.phase = 'already-migrated';
state.phase_detail = 'Schema v' + marker.schemaVersion + ' already present in target';
state.success = true;
// A previous boot already migrated attachments, avatars and text data:
// reclaim the old MongoDB disk space now (runs once, tracked by the marker).
try { await compactSource(srcDb, markerColl); } catch (e) { pushError('compact: ' + e.message); }
state.finishedAt = new Date().toISOString();
await srcClient.close();
await tgtClient.close();
setTimeout(() => process.exit(0), 3000);
return;
}
repairMode = true;
console.log(`[migrate] Migration is complete (schema v${marker.schemaVersion}) but its file phase is v${marker.filesVersion || 1} < v${FILES_VERSION}: running an incremental files-only repair. Text data is left untouched; only missing attachments/avatars are migrated.`);
}
if (repairMode) {
state.phase = 'repairing-files';
state.phase_detail = 'Incremental attachment/avatar repair — text data untouched';
}
// ── 1. Build lookup maps needed for schema upgrades ────────────────────
state.phase = 'analyzing';
state.phase_detail = 'Building swimlane/list lookup maps…';
console.log('[migrate] Analyzing source schema…');
// boardId → first swimlaneId (for orphaned lists/cards)
// (skipped in repair mode: they only feed the text-copy transformers)
const defaultSwimlaneIdFor = {};
const swimlaneCursor = srcDb.collection('swimlanes').find({ archived: false }, {
projection: { _id: 1, boardId: 1, sort: 1 }, sort: { sort: 1 },
});
for await (const sw of swimlaneCursor) {
if (!defaultSwimlaneIdFor[sw.boardId]) {
defaultSwimlaneIdFor[sw.boardId] = String(sw._id);
}
}
await swimlaneCursor.close();
// listId → swimlaneId (so orphaned cards can inherit from their list)
const listSwimlaneMap = {};
const listCursor = srcDb.collection('lists').find({}, { projection: { _id: 1, swimlaneId: 1 } });
for await (const ls of listCursor) {
listSwimlaneMap[String(ls._id)] = ls.swimlaneId || '';
if (!repairMode) {
state.phase = 'analyzing';
state.phase_detail = 'Building swimlane/list lookup maps…';
console.log('[migrate] Analyzing source schema…');
// boardId → first swimlaneId (for orphaned lists/cards)
const swimlaneCursor = srcDb.collection('swimlanes').find({ archived: false }, {
projection: { _id: 1, boardId: 1, sort: 1 }, sort: { sort: 1 },
});
for await (const sw of swimlaneCursor) {
if (!defaultSwimlaneIdFor[sw.boardId]) {
defaultSwimlaneIdFor[sw.boardId] = String(sw._id);
}
}
await swimlaneCursor.close();
// listId → swimlaneId (so orphaned cards can inherit from their list)
const listCursor = srcDb.collection('lists').find({}, { projection: { _id: 1, swimlaneId: 1 } });
for await (const ls of listCursor) {
listSwimlaneMap[String(ls._id)] = ls.swimlaneId || '';
}
await listCursor.close();
}
await listCursor.close();
// ── 2. Ensure target filesystem directories exist ───────────────────────
if (!DRY_RUN) {
@@ -902,6 +953,10 @@ async function run() {
]);
// ── 4. Copy all plain collections ─────────────────────────────────────
// Repair mode NEVER touches text collections: users may have added/changed
// boards and cards on FerretDB since the original migration, and re-copying
// from the frozen MongoDB source would overwrite or resurrect data.
if (!repairMode) {
state.phase = 'migrating-collections';
for (const name of allColls) {
// FerretDB v1 rejects collection names containing a dot ("invalid key: '<name>' (key must
@@ -935,8 +990,10 @@ async function run() {
completedCollections.add(name);
saveCheckpoint(true); // persist progress to WRITABLE_PATH after each collection
}
} // end !repairMode (text collections)
// ── 5. Also ensure boards without a swimlane have one in the target ────
if (!repairMode) {
state.phase = 'ensuring-swimlanes';
state.phase_detail = 'Creating default swimlanes for boards that have none…';
if (!DRY_RUN) {
@@ -1011,9 +1068,10 @@ async function run() {
}
await orphanCursor.close();
}
} // end !repairMode (swimlane/orphan fixes)
// ── 7. Extract GridFS / CollectionFS binary files to filesystem ─────────
state.phase = 'migrating-files';
state.phase = repairMode ? 'repairing-files' : 'migrating-files';
// Close both clients and exit (staying up 60 s so the dashboard's disk-space error can be
// read) when the file phase stops for lack of disk space.
@@ -1024,6 +1082,32 @@ async function run() {
setTimeout(() => process.exit(1), 60_000);
};
// SPEED (#6473 follow-up): preload the TARGET's attachment/avatar metadata ONCE
// per bucket instead of one findOne round trip per file — per-document round
// trips were a main reason a big migration took 5 hours. The records are small
// metadata documents; a very large collection (>100k records) falls back to
// per-file lookups to bound memory.
const TGT_PRELOAD_LIMIT = 100_000;
const tgtRecCache = {};
async function getTgtRec(collName, id) {
if (tgtRecCache[collName] === undefined) {
tgtRecCache[collName] = null;
try {
const n = await tgtDb.collection(collName).countDocuments({});
if (n <= TGT_PRELOAD_LIMIT) {
const map = new Map();
const cur = tgtDb.collection(collName).find({}, { projection: { name: 1, versions: 1 } });
for await (const d of cur) map.set(String(d._id), d);
await cur.close();
tgtRecCache[collName] = map;
}
} catch { /* keep per-file lookups */ }
}
const map = tgtRecCache[collName];
if (map) return map.get(String(id)) || null;
try { return await tgtDb.collection(collName).findOne({ _id: id }); } catch { return null; }
}
// Measure up front how many files and how many BYTES must be extracted (both the
// CollectionFS GridFS files and the Meteor-Files-in-GridFS records), so the dashboard can
// show "file N of TOTAL" and a per-file bar, and so we can STOP before extracting anything
@@ -1049,8 +1133,11 @@ async function run() {
updateDiskFree(WRITABLE, true);
console.log(`[migrate] Files to extract: ${state.current.total} (${(state.filesTotalBytes / 1048576).toFixed(0)} MB)${state.diskFree >= 0 ? `; disk free ${(state.diskFree / 1048576).toFixed(0)} MB` : '; free space unknown — will stop on a write failure'}.`);
// Up-front check ONLY when free space is measurable. If it will not all fit, stop before
// extracting anything so a full disk can never corrupt MongoDB.
if (!DRY_RUN && state.filesTotalBytes > 0 && state.diskFree >= 0 && state.diskFree < state.filesTotalBytes + DISK_MIN_FREE) {
// extracting anything so a full disk can never corrupt MongoDB. NOT in repair mode: there
// most (often all) files are already on disk and need no new space, so requiring room for
// the full set again would falsely abort healthy repairs — the per-file
// ensureDiskForFile() guard still protects every actual write.
if (!DRY_RUN && !repairMode && state.filesTotalBytes > 0 && state.diskFree >= 0 && state.diskFree < state.filesTotalBytes + DISK_MIN_FREE) {
state.abort = true; state.success = false; state.phase = 'error';
state.additionalBytesNeeded = Math.max(0, state.filesTotalBytes + DISK_MIN_FREE - state.diskFree);
pushError(`FATAL: not enough disk space to migrate all files: ${(state.filesTotalBytes / 1048576).toFixed(0)} MB of attachments/avatars must be extracted, only ${(state.diskFree / 1048576).toFixed(0)} MB free — need ${(state.additionalBytesNeeded / 1048576).toFixed(0)} MB more. Stopped before extracting to avoid MongoDB data corruption.`);
@@ -1061,15 +1148,20 @@ async function run() {
async function migrateGridFs(bucketName, destDir, fileStateKey) {
const filesCollName = `cfs_gridfs.${bucketName}.files`;
if (!allColls.includes(filesCollName)) return;
const haveBucket = allColls.includes(filesCollName);
const haveFilerecords = allColls.includes(`cfs.${bucketName}.filerecord`);
// A CFS FileSystem-store database (ATTACHMENTS_STORE_PATH era, WeKan
// v3.12–v6.09) may have filerecords but NO cfs_gridfs collections at all —
// returning early here silently lost every such attachment.
if (!haveBucket && !haveFilerecords) return;
state.phase_detail = `Extracting ${bucketName} files from GridFS…`;
console.log(`[migrate] Extracting GridFS bucket: ${bucketName} → ${destDir}`);
console.log(`[migrate] Extracting CollectionFS bucket: ${bucketName} → ${destDir}`);
const bucket = new GridFSBucket(srcDb, { bucketName: `cfs_gridfs.${bucketName}` });
const filesCol = srcDb.collection(filesCollName);
const filesCol = haveBucket ? srcDb.collection(filesCollName) : null;
const bucket = haveBucket ? new GridFSBucket(srcDb, { bucketName: `cfs_gridfs.${bucketName}` }) : null;
state.files[fileStateKey].total = await filesCol.countDocuments();
state.files[fileStateKey].total = haveBucket ? await filesCol.countDocuments() : 0;
// #6473: join the CollectionFS filerecords (original name, board/card links)
// to the GridFS files by their RESOLVED GridFS id — but DRIVE the extraction
@@ -1078,13 +1170,20 @@ async function run() {
// the id living at copies.<bucket>.key (the real CollectionFS layout) — or a
// missing cfs.<bucket>.filerecord collection — silently skipped EVERY file
// while the migration still reported success.
//
// A key that is NOT a 24-hex ObjectId is a CFS **FileSystem store** key
// (ATTACHMENTS_STORE_PATH era): the value is the file's NAME on disk
// ("<collection>-<id>-<name>"), not a GridFS id — handled separately below.
const HEX24 = /^[0-9a-f]{24}$/i;
const recordByGridId = new Map();
const fsStoreRecords = [];
let danglingRecords = 0; // filerecords carrying no GridFS reference at all
if (allColls.includes(`cfs.${bucketName}.filerecord`)) {
if (haveFilerecords) {
const metaCursor = srcDb.collection(`cfs.${bucketName}.filerecord`).find({});
for await (const record of metaCursor) {
const gid = resolveCfsGridFsId(record, bucketName);
if (gid) recordByGridId.set(String(gid), record);
if (gid && HEX24.test(String(gid))) recordByGridId.set(String(gid), record);
else if (gid) fsStoreRecords.push({ record, key: String(gid) });
else danglingRecords++;
}
await metaCursor.close();
@@ -1093,12 +1192,17 @@ async function run() {
pushError(`${bucketName}: ${danglingRecords} CollectionFS filerecord(s) carry no GridFS reference; their binaries cannot be located.`);
state.files[fileStateKey].errors += danglingRecords;
}
state.current.total += fsStoreRecords.length;
state.files[fileStateKey].total += fsStoreRecords.length;
const matchedGridIds = new Set(); // hex filerecords whose GridFS file was found
let orphanFiles = 0; // GridFS files with no filerecord (extracted, kept on disk)
const gfCursor = filesCol.find({});
const gfCursor = haveBucket ? filesCol.find({}) : null;
if (gfCursor)
for await (const gf of gfCursor) {
const gridFsId = gf._id;
const record = recordByGridId.get(String(gridFsId)) || null;
if (record) matchedGridIds.add(String(gridFsId));
const original = (record && record.original) || {};
const recordId = record ? String(record._id) : String(gridFsId);
const originalName = original.name || (record && record.filename) || gf.filename || recordId;
@@ -1117,6 +1221,27 @@ async function run() {
continue;
}
// #6473: ALREADY MIGRATED? Check the TARGET itself — record present, version on
// 'fs', file actually on disk. This is what makes the automatic startup repair
// INCREMENTAL: servers whose migration worked verify-and-skip every file quickly,
// and only what is genuinely missing gets migrated.
if (!DRY_RUN && record) {
try {
const tgtRec = await getTgtRec(bucketName, record._id);
const tv = tgtRec && tgtRec.versions && tgtRec.versions.original;
if (tv && tv.storage === 'fs' && tv.path && fs.existsSync(tv.path)) {
fileIndex++;
state.files[fileStateKey].done++;
state.files[fileStateKey].bytes += Number(tv.size) || 0;
let onDiskSize = Number(tv.size) || 0;
try { onDiskSize = fs.statSync(tv.path).size; } catch { /* keep record size */ }
completedFiles.set(resumeKey, { path: tv.path, size: onDiskSize });
saveCheckpoint();
continue;
}
} catch { /* fall through to extraction */ }
}
// STOP (not skip) if this file plus the safety margin will not fit: a full disk can
// corrupt the still-running source MongoDB. Roll back and report how much more is needed.
if (!DRY_RUN && !ensureDiskForFile(destDir, size, originalName)) { await gfCursor.close(); onDiskAbort(); return; }
@@ -1172,10 +1297,110 @@ async function run() {
if (state.abort) { await gfCursor.close(); onDiskAbort(); return; }
}
await gfCursor.close();
if (gfCursor) await gfCursor.close();
if (orphanFiles > 0) {
console.log(`[migrate] ${bucketName}: ${orphanFiles} GridFS file(s) had no CollectionFS filerecord; extracted to ${destDir} without a database record.`);
}
// ── CFS FileSystem-store filerecords (ATTACHMENTS_STORE_PATH era, v3.12–v6.09) ──
// copies.<bucket>.key is the file's NAME under the CFS FileSystem store dir. Copy
// the binary into the current layout and create the Meteor-Files record.
let fsStoreMissing = 0;
for (const { record, key } of fsStoreRecords) {
const original = record.original || {};
const recordId = String(record._id);
const originalName = original.name || record.filename || recordId;
const resumeKey = `${bucketName}:${recordId}`;
const done = alreadyExtracted(resumeKey);
if (done) {
fileIndex++;
state.files[fileStateKey].done++;
state.files[fileStateKey].bytes += Number(done.size) || 0;
continue;
}
// Already migrated? (incremental repair — same check as the GridFS loop)
if (!DRY_RUN) {
try {
const tgtRec = await getTgtRec(bucketName, record._id);
const tv = tgtRec && tgtRec.versions && tgtRec.versions.original;
if (tv && tv.storage === 'fs' && tv.path && fs.existsSync(tv.path)) {
fileIndex++;
state.files[fileStateKey].done++;
state.files[fileStateKey].bytes += Number(tv.size) || 0;
let onDiskSize = Number(tv.size) || 0;
try { onDiskSize = fs.statSync(tv.path).size; } catch { /* keep record size */ }
completedFiles.set(resumeKey, { path: tv.path, size: onDiskSize });
saveCheckpoint();
continue;
}
} catch { /* fall through */ }
}
// Locate the binary: ATTACHMENTS_STORE_PATH (the env the old store used; the
// operator carries it over), an absolute key, or a cfs/files dir moved under
// WRITABLE_PATH.
const candidates = [];
if (path.isAbsolute(key)) candidates.push(key);
for (const dir of [process.env.ATTACHMENTS_STORE_PATH, process.env.AVATARS_STORE_PATH]) {
if (dir) candidates.push(path.join(dir, key));
}
candidates.push(path.join(WRITABLE, 'cfs', 'files', bucketName, key));
candidates.push(path.join(WRITABLE, 'files', 'cfs', bucketName, key));
let srcPath = null;
for (const c of candidates) {
try { if (fs.statSync(c).isFile()) { srcPath = c; break; } } catch { /* next */ }
}
if (!srcPath) { fsStoreMissing++; continue; }
const size = (() => { try { return fs.statSync(srcPath).size; } catch { return original.size || 0; } })();
const { fullPath, basename } = destFilePath(destDir, recordId, originalName);
if (!DRY_RUN && !ensureDiskForFile(destDir, size, originalName)) { onDiskAbort(); return; }
fileIndex++;
state.current = { active: true, kind: fileStateKey, name: originalName, size, done: 0, index: fileIndex, total: state.current.total };
if (DRY_RUN) {
state.files[fileStateKey].done++;
state.files[fileStateKey].bytes += size;
state.current.active = false;
continue;
}
try {
fs.copyFileSync(srcPath, fullPath);
state.files[fileStateKey].done++;
state.files[fileStateKey].bytes += size;
state.current.done = size;
const mfRecord = cfsRecordToMeteorFile(record, bucketName, basename);
mfRecord.path = fullPath;
mfRecord.size = size;
if (mfRecord.versions && mfRecord.versions.original) {
mfRecord.versions.original.path = fullPath;
mfRecord.versions.original.size = size;
mfRecord.versions.original.storage = 'fs';
}
mfRecord.meta = { ...(mfRecord.meta || {}), originalFilename: originalName, storedBasename: basename, source: 'cfs-migration' };
await tgtDb.collection(bucketName).replaceOne({ _id: record._id }, mfRecord, { upsert: true });
completedFiles.set(resumeKey, { path: fullPath, size });
saveCheckpoint();
} catch (err) {
if (err && (err.code === 'ENOSPC' || err.code === 'EDQUOT')) flagDiskFull('write failed: ' + err.code);
pushError(`CFS fs-store copy ${bucketName}/${recordId}: ${err.message}`);
state.files[fileStateKey].errors++;
try { fs.unlinkSync(fullPath); } catch { /* ignore */ }
} finally { state.current.active = false; }
if (state.abort) { onDiskAbort(); return; }
}
if (fsStoreMissing > 0) {
pushError(`${bucketName}: ${fsStoreMissing} CollectionFS FileSystem-store file(s) not found on disk — set ATTACHMENTS_STORE_PATH to the old store directory and re-run the repair.`);
state.files[fileStateKey].errors += fsStoreMissing;
}
// Hex filerecords whose GridFS binary no longer exists: report, never silently drop.
const missingBinaries = [...recordByGridId.keys()].filter(k => !matchedGridIds.has(k)).length;
if (missingBinaries > 0) {
pushError(`${bucketName}: ${missingBinaries} CollectionFS filerecord(s) reference a GridFS file that no longer exists; their binaries cannot be recovered.`);
state.files[fileStateKey].errors += missingBinaries;
}
}
await migrateGridFs('attachments', ATTACH_DIR, 'attachments');
@@ -1262,6 +1487,13 @@ async function run() {
// ── Pass A: record-driven (any version flagged gridfs or carrying gridFsFileId) ──
const cursor = col.find(MF_GRIDFS_QUERY);
for await (const doc of cursor) {
// #6473: the SOURCE record is frozen (MongoDB is never modified), so on a
// re-run it stays flagged gridfs forever. Ask the TARGET what actually still
// needs doing — that is what makes the automatic startup repair incremental.
let tgtDoc = null;
if (!DRY_RUN) {
tgtDoc = await getTgtRec(collName, doc._id);
}
for (const [versionName, version] of Object.entries(doc.versions || {})) {
if (!version || typeof version !== 'object') continue;
const marked = version.storage === 'gridfs' || (version.meta && version.meta.gridFsFileId);
@@ -1269,6 +1501,24 @@ async function run() {
const resumeKey = resumeKeyFor(String(doc._id), versionName);
seenByRecordPass.add(resumeKey);
// ALREADY MIGRATED: the target's version is on the filesystem and the file
// exists — verify-and-skip.
const tv = tgtDoc && tgtDoc.versions && tgtDoc.versions[versionName];
if (tv && tv.storage === 'fs' && tv.path && fs.existsSync(tv.path)) {
fileIndex++;
state.files[fileStateKey].done++;
state.files[fileStateKey].bytes += Number(tv.size) || 0;
let onDiskSize = Number(tv.size) || 0;
try { onDiskSize = fs.statSync(tv.path).size; } catch { /* keep record size */ }
completedFiles.set(resumeKey, { path: tv.path, size: onDiskSize });
saveCheckpoint();
continue;
}
// Repair mode: the record was DELETED in the target since the migration —
// do not resurrect its binary.
if (repairMode && !tgtDoc) continue;
const done = alreadyExtracted(resumeKey);
if (done) {
fileIndex++;
@@ -1307,11 +1557,16 @@ async function run() {
// gridfs but no gridFsFileId), which are exactly what this sweep recovers.
if (seenByRecordPass.has(resumeKey) && !unresolved.has(resumeKey)) continue;
const record = md.fileId ? await col.findOne({ _id: md.fileId }) : null;
// Ask the TARGET (the live database — the source is frozen) what this
// record looks like now.
const record = md.fileId ? await getTgtRec(collName, md.fileId) : null;
const recVersion = record && record.versions && record.versions[versionName];
// The record's version already lives happily on the filesystem: this bucket
// file is a stale leftover of an earlier move — leave both alone.
if (recVersion && recVersion.storage === 'fs' && recVersion.path && fs.existsSync(recVersion.path)) continue;
// Repair mode: the record was DELETED in the target since the migration —
// do not resurrect its binary.
if (repairMode && md.fileId && !record) continue;
const done = alreadyExtracted(resumeKey);
if (done) {
@@ -1344,30 +1599,49 @@ async function run() {
// ── 8. Write completion marker ─────────────────────────────────────────
if (!DRY_RUN) {
await markerColl.replaceOne(
{ _id: 'completed' },
{
_id: 'completed',
schemaVersion: SCHEMA_VER,
completedAt: new Date(),
sourceUrl: SOURCE_URL,
targetUrl: TARGET_URL,
dryRun: DRY_RUN,
collections: Object.fromEntries(
Object.entries(state.collections).map(([k, v]) => [k, { done: v.done, errors: v.errors }])
),
files: state.files,
},
{ upsert: true },
);
saveCheckpoint(true);
if (repairMode) {
// #6473: an incremental repair only STAMPS the marker's filesVersion (plus the
// repair stats); the original migration record — collections copied, when, from
// where — must stay intact, and a repair must never fabricate a "completed"
// marker where none exists.
if (marker) {
await markerColl.updateOne(
{ _id: 'completed' },
{ $set: { filesVersion: FILES_VERSION, filesRepairedAt: new Date(), filesRepair: state.files } },
).catch(e => pushError('marker filesVersion update: ' + e.message));
}
saveCheckpoint(true);
} else {
await markerColl.replaceOne(
{ _id: 'completed' },
{
_id: 'completed',
schemaVersion: SCHEMA_VER,
filesVersion: FILES_VERSION,
completedAt: new Date(),
sourceUrl: SOURCE_URL,
targetUrl: TARGET_URL,
dryRun: DRY_RUN,
collections: Object.fromEntries(
Object.entries(state.collections).map(([k, v]) => [k, { done: v.done, errors: v.errors }])
),
files: state.files,
},
{ upsert: true },
);
saveCheckpoint(true);
}
}
// ── 8b. Reclaim old MongoDB disk space now that everything is migrated ──
try { await compactSource(srcDb, markerColl); } catch (e) { pushError('compact: ' + e.message); }
// (skipped in repair mode — compact already ran after the original migration)
if (!repairMode) {
try { await compactSource(srcDb, markerColl); } catch (e) { pushError('compact: ' + e.message); }
}
// ── 9. Ensure indexes on commonly-queried fields ───────────────────────
if (!DRY_RUN) {
// (skipped in repair mode — they were created by the original migration)
if (!DRY_RUN && !repairMode) {
state.phase = 'creating-indexes';
state.phase_detail = 'Creating indexes on target…';
try {
@@ -1397,7 +1671,9 @@ async function run() {
if (totalErrors === 0) {
state.phase = 'completed';
state.success = true;
console.log('[migrate] ✓ Migration completed successfully.');
console.log(repairMode
? `[migrate] ✓ Attachment repair completed: ${state.files.attachments.done} attachments, ${state.files.avatars.done} avatars verified/migrated.`
: '[migrate] ✓ Migration completed successfully.');
} else {
state.phase = 'completed-with-errors';
// #6466: per-item errors (one unparsable document, one attachment/avatar that
@@ -1432,6 +1708,11 @@ Environment variables:
BATCH_SIZE Docs per write batch (default: 200)
DRY_RUN true = no writes (default: false)
FORCE_MIGRATE true = ignore done marker (default: false)
FILES_ONLY true = incremental attachment/avatar repair only: text
data is left untouched, already-migrated files are
verified and skipped, only missing binaries/records are
migrated. Also turned on automatically when the target's
migration marker has filesVersion < ${FILES_VERSION}.
Files are written to:
\$WRITABLE_PATH/files/attachments/
server/authentication.js
+44 -2
View File
@@ -1,5 +1,10 @@
import { ReactiveCache } from '/imports/reactiveCache';
const {
shouldRejectPasswordLogin,
LDAP_PASSWORD_LOGIN_DISABLED_REASON,
} = require('/server/lib/ldapPasswordLoginGuard');
// Authentication helpers — exported for use by API routes and model files
export const Authentication = {
async checkUserId(userId) {
@@ -84,6 +89,31 @@ Meteor.startup(() => {
return !user.loginDisabled;
});
// #4419 (Severity:Security): after a user is migrated from local password
// login to LDAP (authenticationMethod: 'ldap'), the stale local password in
// services.password would otherwise still work. Reject password-service
// logins for LDAP users — but only while LDAP is actually enabled, never for
// other services ('ldap', 'resume', 'oidc', 'cas', 'saml', …), never when
// LDAP_LOGIN_FALLBACK=true (that feature intentionally routes LDAP logins
// through the password service), and never when the operator opted out with
// LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN=true. Full decision logic and
// rationale live in server/lib/ldapPasswordLoginGuard.js.
Accounts.validateLoginAttempt(function(options) {
if (
shouldRejectPasswordLogin({
serviceName: options.type,
user: options.user,
env: process.env,
})
) {
throw new Meteor.Error(
'ldap-password-login-disabled',
LDAP_PASSWORD_LOGIN_DISABLED_REASON,
);
}
return true;
});
if (Meteor.isServer) {
if (
process.env.ORACLE_OIM_ENABLED === 'true' ||
@@ -94,7 +124,13 @@ Meteor.startup(() => {
{ service: 'oidc' },
{
$set: {
loginStyle: process.env.OAUTH2_LOGIN_STYLE || 'redirect',
// #5695: the client now honors a configured 'redirect' style, so
// the fallback here must stay 'popup' to keep popup the default
// behavior when OAUTH2_LOGIN_STYLE is not set.
loginStyle:
process.env.OAUTH2_LOGIN_STYLE === 'redirect'
? 'redirect'
: 'popup',
clientId: process.env.OAUTH2_CLIENT_ID,
secret: process.env.OAUTH2_SECRET,
serverUrl: process.env.OAUTH2_SERVER_URL,
@@ -116,7 +152,13 @@ Meteor.startup(() => {
{ service: 'oidc' },
{
$set: {
loginStyle: process.env.OAUTH2_LOGIN_STYLE || 'redirect',
// #5695: the client now honors a configured 'redirect' style, so
// the fallback here must stay 'popup' to keep popup the default
// behavior when OAUTH2_LOGIN_STYLE is not set.
loginStyle:
process.env.OAUTH2_LOGIN_STYLE === 'redirect'
? 'redirect'
: 'popup',
clientId: process.env.OAUTH2_CLIENT_ID,
secret: process.env.OAUTH2_SECRET,
serverUrl: process.env.OAUTH2_SERVER_URL,
server/imports.js
+3
View File
@@ -195,6 +195,9 @@ import '/server/migrations/fixMissingListsMigration';
import '/server/migrations/migrateAttachments';
import '/server/migrations/restoreAllArchived';
import '/server/migrations/restoreLostCards';
// #6473: startup schema upgrade — checks what old-version data is already
// migrated and migrates only the rest (all platforms, both databases).
import '/server/startupSchemaUpgrade';
// ----------------------------------------------------------------------------
// 12. Server — notifications
server/lib/ldapPasswordLoginGuard.js
+104
View File
@@ -0,0 +1,104 @@
// Pure, dependency-free helper (no Meteor imports) so it can be unit tested
// directly with plain Node. Used by the `Accounts.validateLoginAttempt` hook
// registered in server/authentication.js.
//
// #4419 (Severity:Security): when an instance migrates a user from local
// password login to LDAP (Admin Panel sets `authenticationMethod: 'ldap'`),
// the user's old `services.password` hash is left in place, so the stale —
// possibly weaker — local password KEEPS working alongside the LDAP password.
// The safe fix (per this repo's own triage in CHANGELOG.md "TODO Later") is to
// reject password-service login attempts for such users at validation time
// instead of destructively `$unset`-ing `services.password`.
//
// The decision is deliberately conservative. A password-service login attempt
// is rejected ONLY when ALL of the following hold:
//
// 1. The attempted login service is 'password' (`attempt.type` from
// Meteor's validateLoginAttempt). All other services — 'ldap', 'oidc',
// 'cas', 'saml', 'resume' (session token resume), … — are never touched,
// so existing sessions and every non-password login flow keep working.
// 2. The user document has `authenticationMethod === 'ldap'`. Users without
// the field, or with any other value ('password', 'oauth2', 'cas',
// 'saml', 'imported', …), are never rejected.
// 3. LDAP is actually enabled in this deployment: LDAP_ENABLE=true
// (same env check as isLdapEnabled() in server/models/settings.js). If
// LDAP has been turned off — e.g. an instance de-LDAPed or a directory
// decommissioned — password login must keep working so admins are never
// locked out.
// 4. LDAP_LOGIN_FALLBACK is NOT enabled. With LDAP_LOGIN_FALLBACK=true the
// wekan-ldap login handler *intentionally* falls back to the local
// password service and syncs the LDAP password into
// `services.password` on every LDAP login (see
// packages/wekan-ldap/server/loginHandler.js). That fallback surfaces in
// validateLoginAttempt with type 'password', so rejecting it would break
// the documented fallback feature.
// 5. The operator has not opted out. Setting
// LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN=true restores the old behavior
// (local password login stays valid for LDAP-migrated users), so no
// deployment can be hard-locked by upgrading WeKan. Document/announce
// this var wherever LDAP env vars are documented.
//
// The same rule applies to every user — admins are not treated differently.
'use strict';
// Error reason shown to the user when the guard rejects the login.
const LDAP_PASSWORD_LOGIN_DISABLED_REASON =
'Use LDAP login; local password login is disabled for this account';
// Mirrors the truthiness convention used across server/models/settings.js
// (isLdapEnabled etc.): env vars are the string 'true', but a literal boolean
// true is also accepted.
function envFlagIsTrue(value) {
return value === 'true' || value === true;
}
/**
* Decide whether a login attempt must be rejected because the account was
* migrated to LDAP but still carries a stale local password.
*
* @param {Object} params
* @param {string} params.serviceName - the login service of the attempt
* (Meteor's `attempt.type`), e.g. 'password', 'ldap', 'resume', 'oidc'.
* @param {Object|undefined} params.user - the user document of the attempt
* (may be undefined for unknown-user attempts).
* @param {Object|undefined} params.env - the environment (process.env).
* @return {boolean} true when the password-service login must be rejected.
*/
function shouldRejectPasswordLogin({ serviceName, user, env } = {}) {
const environment = env || {};
// (1) Only the local password service is ever guarded.
if (serviceName !== 'password') {
return false;
}
// (2) Only users explicitly migrated to LDAP.
if (!user || user.authenticationMethod !== 'ldap') {
return false;
}
// (3) Only when LDAP is enabled in this deployment; otherwise password
// login must keep working (never lock admins out of a de-LDAPed instance).
if (!envFlagIsTrue(environment.LDAP_ENABLE)) {
return false;
}
// (4) LDAP_LOGIN_FALLBACK intentionally logs LDAP users in through the
// password service; do not break that feature.
if (envFlagIsTrue(environment.LDAP_LOGIN_FALLBACK)) {
return false;
}
// (5) Operator opt-out: keep the pre-#4419-fix behavior.
if (envFlagIsTrue(environment.LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN)) {
return false;
}
return true;
}
module.exports = {
shouldRejectPasswordLogin,
LDAP_PASSWORD_LOGIN_DISABLED_REASON,
};
server/lib/schemaUpgradeSteps.js
+808
View File
@@ -0,0 +1,808 @@
// ============================================================================
// Startup schema upgrade (#6473 follow-up): bring TEXT DATA from EVERY old
// WeKan version to the newest database structure — on all platforms (Snap,
// Docker, Sandstorm, source) and regardless of database (MongoDB or
// FerretDB/SQLite, both speak the same wire protocol via the driver `Db`
// handle this module receives).
//
// WeKan v0.9–v8.00 ran `Migrations.add(...)` steps at startup; v8.01 disabled
// them (large databases caused long downtime) in favour of read-time
// compatibility — which covers the swimlane era but NOT everything. Data that
// jumped straight from an old version (or an old backup/Sandstorm grain) to
// current WeKan can still hold pre-migration shapes that today's code simply
// does not show: the text is in the database, but invisible.
//
// This module reinstates the safety net WITHOUT the downtime problem:
// - VERSION-GATED: the `_wekan_migration` marker stores the WeKan version and
// datetime of the last successful re-check. While the version is unchanged,
// a boot costs ONE findOne — the re-check is mandatory only after a new
// WeKan release (or WEKAN_FORCE_SCHEMA_UPGRADE=true).
// - After a version change it RE-CHECKS everything and migrates ONLY what is
// missing, using the fastest available query shapes for big databases
// (thousands of cards): bounded findOne/countDocuments existence probes,
// distinct() joins instead of full scans, and server-side updateMany
// batches instead of per-document round trips.
// - Every step is idempotent, so an interrupted boot resumes safely, and a
// step failure never blocks WeKan from starting.
// - Progress is published (getUpgradeState) for the migration dashboard,
// which shows the Admin Panel product name when one is set.
//
// Steps (each replicates a v8.00 `server/migrations.js` step that has no
// replacement in current WeKan, verified against `git show v8.00:server/migrations.js`):
// archived-flag-backfill old docs missing `archived` never match the
// archived:false view queries -> invisible
// swimlane-structure add-swimlanes (v0.65) + dangling-listId rescue +
// #1959/#1971 Swimlanes-view visibility rescue
// (cards under deleted/archived/foreign swimlanes)
// checklist-items-embedded add-checklist-items (v0.79): embedded
// checklist.items[] -> ChecklistItems collection
// customfields-boardIds mutate-boardIds-in-customfields (v2.49):
// scalar boardId -> boardIds array
// board-allows-defaults the add-*-allowed steps (v2.9–v5.43): missing
// allows* flags render as undefined=false and HIDE
// existing description/checklist/comment text
// board-members-isactive add-member-isactive-field: members without
// isActive are denied board access by
// isActiveMember()
// board-permission-lowercase lowercase-board-permission: 'PUBLIC' boards
// silently became member-only
// fs-path-heal filesystem attachments/avatars whose recorded
// path predates the current WRITABLE_PATH layout
// (v6.10-18 uploads/<coll>, v6.19-v8.4x
// WRITABLE_PATH/<coll>, CFS->ostrio temp files):
// locate the binary and repoint/copy it
//
// Pure, dependency-free (no Meteor imports) so it can be unit tested with
// plain Node against a driver-compatible Db object. The Meteor wrapper (and
// the /schema-upgrade-status dashboard) is server/startupSchemaUpgrade.js.
// ============================================================================
const fs = require('fs');
const path = require('path');
const crypto = require('crypto');
const { computeStoragePaths } = require('../../models/lib/attachmentStoragePath');
const MARKER_COLL = '_wekan_migration';
const MARKER_ID = 'schema-upgrade';
// Meteor-style random document ids (Random.id(): 17 chars, unmistakable set),
// so upgraded documents look exactly like app-created ones.
const ID_CHARS = '23456789ABCDEFGHJKLMNPQRSTWXYZabcdefghijkmnopqrstuvwxyz';
function randomId(len = 17) {
const bytes = crypto.randomBytes(len);
let out = '';
for (let i = 0; i < len; i++) out += ID_CHARS[bytes[i] % ID_CHARS.length];
return out;
}
// ── live progress state for the migration dashboard ─────────────────────────
const upgradeState = {
running: false,
product: 'WeKan', // Admin Panel product name when set
appVersion: '',
startedAt: null,
finishedAt: null,
currentStep: '',
gated: false, // true = skipped because the version was unchanged
steps: {}, // name -> {status: pending|checking|running|done|skipped|error, fixed, unresolved, error}
lastCheck: null, // {version, at} of the previous successful re-check
};
function getUpgradeState() { return upgradeState; }
function resetState(appVersion) {
upgradeState.running = true;
upgradeState.product = 'WeKan';
upgradeState.appVersion = appVersion || '';
upgradeState.startedAt = new Date();
upgradeState.finishedAt = null;
upgradeState.currentStep = '';
upgradeState.gated = false;
upgradeState.steps = {};
for (const s of steps) upgradeState.steps[s.name] = { status: 'pending', fixed: 0, unresolved: 0 };
}
// Board feature flags with schema defaultValue: true. SimpleSchema applies
// defaults ON INSERT ONLY, and the templates do raw property access — a board
// created before the flag existed reads `undefined`, which renders as FALSE
// and hides existing text (descriptions, checklists, comments, attachments).
// Kept in sync with models/boards.js by tests/schemaUpgradeSteps.test.cjs.
const BOARD_ALLOWS_TRUE_DEFAULTS = [
'allowsSubtasks', 'allowsSubtasksOnMinicard',
'allowsAttachments', 'allowsAttachmentsOnMinicard',
'allowsChecklists', 'allowsChecklistsOnMinicard',
'allowsComments',
'allowsDescriptionTitle', 'allowsDescriptionTitleOnMinicard',
'allowsDescriptionText',
'allowsCoverAttachmentOnMinicard',
'allowsActivities',
'allowsLabels', 'allowsLabelsOnMinicard',
'allowsCreator',
'allowsAssignee', 'allowsAssigneeOnMinicard',
'allowsMembers', 'allowsMembersOnMinicard',
'allowsRequestedBy', 'allowsRequestedByOnMinicard',
'allowsCardSortingByNumber',
'allowsShowLists',
'allowsAssignedBy', 'allowsAssignedByOnMinicard',
'allowsReceivedDate', 'allowsReceivedDateOnMinicard',
'allowsStartDate', 'allowsStartDateOnMinicard',
'allowsEndDate', 'allowsEndDateOnMinicard',
'allowsDueDate', 'allowsDueDateOnMinicard',
];
const MISSING_OR_EMPTY = { $in: [null, ''] }; // matches missing, null and ''
// ── individual steps ─────────────────────────────────────────────────────────
// Each step: { name, check(db,ctx) -> boolean (work exists),
// run(db,ctx) -> {fixed, unresolved} } — all idempotent, and all
// using bounded/batched queries so big databases stay fast.
const steps = [
{
// Old (LibreBoard-era) documents can lack the `archived` field entirely —
// and every board/list/card query in current WeKan filters with
// `archived: false`, which a MISSING field does not match. Such boards,
// swimlanes, lists and cards are invisible in BOTH the Swimlanes view and
// the Lists view although their data is intact. One updateMany per
// collection backfills the default. Must run FIRST: the visibility rescue
// below relies on `archived: false` matching.
name: 'archived-flag-backfill',
async check(db) {
for (const coll of ['boards', 'swimlanes', 'lists', 'cards']) {
if (await db.collection(coll).findOne({ archived: { $exists: false } }, { projection: { _id: 1 } })) return true;
}
return false;
},
async run(db) {
let fixed = 0;
for (const coll of ['boards', 'swimlanes', 'lists', 'cards']) {
const r = await db.collection(coll).updateMany(
{ archived: { $exists: false } },
{ $set: { archived: false } },
);
fixed += (r && r.modifiedCount) || 0;
}
return { fixed, unresolved: 0 };
},
},
{
// add-swimlanes (v0.65) + the dangling-listId rescue that the disabled
// repair tools used to provide, PLUS the Swimlanes-view visibility rescue
// of #1959/#1971: every board gets a swimlane, every list/card gets a
// swimlaneId, cards whose listId points nowhere are rescued to a visible
// list, and unarchived cards whose swimlaneId points at a DELETED,
// ARCHIVED or other-board swimlane (invisible in Swimlanes view) are
// reassigned to the board's first visible swimlane — so all lists and
// cards are visible in both the Swimlanes view and the Lists view.
name: 'swimlane-structure',
async check(db) {
// bounded existence probes — no scans. NOTE: lists with EMPTY swimlaneId
// are deliberately NOT backfilled — a shared board-wide list ('') renders
// in every swimlane via myLists()'s null/'' fallback, and stamping it to
// one swimlane would HIDE it from all other swimlanes in Swimlanes view.
if (await db.collection('cards').findOne({ swimlaneId: MISSING_OR_EMPTY }, { projection: { _id: 1 } })) return true;
// set-joins via distinct(): result sizes are bounded by the number of
// lists/swimlanes/boards, never by the number of cards
const boardIds = (await db.collection('boards').distinct('_id')).map(String);
const swimlaneBoards = new Set((await db.collection('swimlanes').distinct('boardId')).map(String));
if (boardIds.some(b => !swimlaneBoards.has(b))) return true;
const listIds = new Set((await db.collection('lists').distinct('_id')).map(String));
const usedListIds = (await db.collection('cards').distinct('listId')).map(String);
if (usedListIds.some(id => id && id !== 'null' && id !== 'undefined' && !listIds.has(id))) return true;
// #1959/#1971: any unarchived card under a deleted/archived/foreign swimlane?
const visibleSwimlaneOfBoard = await loadVisibleSwimlanes(db);
const usedSwimlaneIds = (await db.collection('cards').distinct('swimlaneId', { archived: false })).map(String);
for (const swId of usedSwimlaneIds) {
if (!swId || swId === 'null' || swId === 'undefined') continue;
const owner = visibleSwimlaneOfBoard.byId.get(swId);
if (!owner) return true; // deleted or archived swimlane
// same-board check needs the referencing boards (bounded by board count)
const boards = (await db.collection('cards').distinct('boardId', { swimlaneId: swId, archived: false })).map(String);
if (boards.some(b => b !== owner)) return true; // foreign swimlane
}
return false;
},
async run(db) {
let fixed = 0;
const now = new Date();
// a VISIBLE (unarchived, non-template) default swimlane per board —
// boards and swimlanes are few. #1959: assigning to any first swimlane
// could pick an archived one, and the card stayed invisible.
const boards = await db.collection('boards').find({}, { projection: { _id: 1 } }).toArray();
let visible = await loadVisibleSwimlanes(db);
const defaultSwimlaneOf = new Map(visible.firstOfBoard);
for (const b of boards) {
const boardId = String(b._id);
if (!defaultSwimlaneOf.has(boardId)) {
const sw = { _id: randomId(), title: 'Default', boardId, archived: false, sort: 0, type: 'swimlane', createdAt: now, modifiedAt: now, updatedAt: now };
await db.collection('swimlanes').insertOne(sw);
defaultSwimlaneOf.set(boardId, String(sw._id));
visible.byId.set(String(sw._id), boardId);
fixed++;
}
}
// missing swimlaneId on CARDS: ONE server-side updateMany per board — no
// per-document round trips, no matter how many thousands of cards.
// Lists are NOT stamped (see check() note: '' means shared across swimlanes).
for (const [boardId, swId] of defaultSwimlaneOf) {
const r = await db.collection('cards').updateMany(
{ boardId, swimlaneId: MISSING_OR_EMPTY },
{ $set: { swimlaneId: swId } },
);
fixed += (r && r.modifiedCount) || 0;
}
// dangling listId rescue: distinct() finds the bad ids (bounded by the
// number of lists ever referenced), then one updateMany per (bad id, board)
const lists = await db.collection('lists').find({}, { projection: { _id: 1, boardId: 1, swimlaneId: 1, sort: 1 } }).toArray();
const listIds = new Set(lists.map(l => String(l._id)));
const firstListOf = new Map();
for (const l of lists.sort((a, b2) => (a.sort || 0) - (b2.sort || 0))) {
if (!firstListOf.has(String(l.boardId))) firstListOf.set(String(l.boardId), l);
}
const rescuedListOf = new Map();
const badListIds = (await db.collection('cards').distinct('listId'))
.map(String).filter(id => id && id !== 'null' && id !== 'undefined' && !listIds.has(id));
for (const badId of badListIds) {
const boardsOfBad = (await db.collection('cards').distinct('boardId', { listId: badId })).map(String);
for (const boardId of boardsOfBad) {
let target = firstListOf.get(boardId) || rescuedListOf.get(boardId);
if (!target) {
target = { _id: randomId(), title: 'Rescued Data', boardId, swimlaneId: defaultSwimlaneOf.get(boardId) || '', archived: false, sort: 0, type: 'list', createdAt: now, modifiedAt: now, updatedAt: now };
await db.collection('lists').insertOne(target);
rescuedListOf.set(boardId, target);
}
const r = await db.collection('cards').updateMany(
{ boardId, listId: badId },
{ $set: { listId: String(target._id), swimlaneId: String(target.swimlaneId || defaultSwimlaneOf.get(boardId) || '') } },
);
fixed += (r && r.modifiedCount) || 0;
}
}
// #1959/#1971: unarchived cards whose swimlaneId points at a DELETED,
// ARCHIVED or other-board swimlane never render in the Swimlanes view.
// distinct() bounds the work by the number of referenced swimlanes; the
// fix is one updateMany per (bad swimlane, board).
const usedSwimlaneIds = (await db.collection('cards').distinct('swimlaneId', { archived: false })).map(String);
for (const swId of usedSwimlaneIds) {
if (!swId || swId === 'null' || swId === 'undefined') continue;
const owner = visible.byId.get(swId);
const boardsOfSw = (await db.collection('cards').distinct('boardId', { swimlaneId: swId, archived: false })).map(String);
for (const boardId of boardsOfSw) {
if (owner === boardId) continue; // visible swimlane on the right board
const target = defaultSwimlaneOf.get(boardId);
if (!target || target === swId) continue;
const r = await db.collection('cards').updateMany(
{ boardId, swimlaneId: swId, archived: false },
{ $set: { swimlaneId: target } },
);
fixed += (r && r.modifiedCount) || 0;
}
}
return { fixed, unresolved: 0 };
},
},
{
// v7.98 'per-swimlane lists' era: the migrate-lists-to-per-swimlane step
// stamped every list to the default swimlane (hiding it from every other
// swimlane in today's Swimlanes view), and the v8.07–v8.19 board-open
// repair DUPLICATED lists per swimlane (rendering duplicate columns per
// title in today's board-wide Lists view). Current WeKan renders shared
// lists (swimlaneId '') in every swimlane, so: merge same-title copies
// back into ONE shared list — cards keep their own swimlaneId, preserving
// per-swimlane grouping and order, and card/list drag between swimlanes
// keeps working — then clear the stamped swimlaneIds. Copies the user
// RENAMED differently per swimlane are deliberately kept as their own
// lists (merging them would lose the distinct names). Boards are detected
// by the era's own markers (fixMissingListsCompleted /
// comprehensiveMigrationCompleted), duplicate titles, or a list whose
// cards disagree with its stamped swimlane; healthy boards are untouched.
name: 'merge-per-swimlane-lists',
async check(db) {
if (await db.collection('boards').findOne(
{ $or: [{ fixMissingListsCompleted: true }, { comprehensiveMigrationCompleted: true }] },
{ projection: { _id: 1 } },
)) return true;
const boardIds = (await db.collection('lists').distinct('boardId', { swimlaneId: { $nin: [null, ''] }, type: { $ne: 'template-list' } })).map(String);
for (const boardId of boardIds) {
const lists = await db.collection('lists').find(
{ boardId, type: { $ne: 'template-list' } },
{ projection: { title: 1, swimlaneId: 1, archived: 1 } },
).toArray();
const seen = new Set();
for (const l of lists) {
if (l.archived === true) continue;
if (seen.has(l.title)) return true; // per-swimlane duplicate copies
seen.add(l.title);
}
for (const l of lists) {
if (!l.swimlaneId) continue;
// Shape A (v7.98 stamp): cards of another swimlane point at this list
if (await db.collection('cards').findOne(
{ listId: String(l._id), swimlaneId: { $nin: [String(l.swimlaneId), null, ''] } },
{ projection: { _id: 1 } },
)) return true;
}
}
return false;
},
async run(db) {
let fixed = 0;
const markerBoards = (await db.collection('boards').find(
{ $or: [{ fixMissingListsCompleted: true }, { comprehensiveMigrationCompleted: true }] },
{ projection: { _id: 1 } },
).toArray()).map(b => String(b._id));
const stampedBoards = (await db.collection('lists').distinct('boardId', { swimlaneId: { $nin: [null, ''] }, type: { $ne: 'template-list' } })).map(String);
for (const boardId of new Set([...markerBoards, ...stampedBoards])) {
const lists = await db.collection('lists').find(
{ boardId, type: { $ne: 'template-list' } },
{ projection: { title: 1, swimlaneId: 1, archived: 1, sort: 1, createdAt: 1 } },
).toArray();
// Only act on boards that actually show an era symptom (duplicate titles
// or a card/list swimlane mismatch or the marker) — a healthy natively
// per-swimlane board is left exactly as it is.
const unarchived = lists.filter(l => l.archived !== true);
const titleCount = new Map();
for (const l of unarchived) titleCount.set(l.title, (titleCount.get(l.title) || 0) + 1);
let mismatch = false;
if (!markerBoards.includes(boardId) && ![...titleCount.values()].some(n => n > 1)) {
for (const l of unarchived) {
if (!l.swimlaneId) continue;
if (await db.collection('cards').findOne(
{ listId: String(l._id), swimlaneId: { $nin: [String(l.swimlaneId), null, ''] } },
{ projection: { _id: 1 } },
)) { mismatch = true; break; }
}
if (!mismatch) continue;
}
// merge same-title unarchived copies into one canonical shared list
const groups = new Map();
for (const l of unarchived) {
const g = groups.get(l.title) || [];
g.push(l);
groups.set(l.title, g);
}
for (const group of groups.values()) {
if (group.length < 2) continue;
group.sort((a, b) =>
((a.swimlaneId ? 1 : 0) - (b.swimlaneId ? 1 : 0)) ||
((a.createdAt || 0) - (b.createdAt || 0)) ||
((a.sort || 0) - (b.sort || 0)));
const canonical = group[0];
for (const m of group.slice(1)) {
// cards KEEP their swimlaneId — only the column pointer moves
await db.collection('cards').updateMany({ listId: String(m._id) }, { $set: { listId: String(canonical._id) } });
try { await db.collection('activities').updateMany({ listId: String(m._id) }, { $set: { listId: String(canonical._id) } }); } catch { /* history relink is best-effort */ }
await db.collection('lists').deleteOne({ _id: m._id });
fixed++;
}
}
// shared lists render in EVERY swimlane via the '' fallback
const r = await db.collection('lists').updateMany(
{ boardId, type: { $ne: 'template-list' }, swimlaneId: { $nin: [null, ''] } },
{ $set: { swimlaneId: '' } },
);
fixed += (r && r.modifiedCount) || 0;
// drop the era markers so the on-demand repair tools are usable again
await db.collection('boards').updateOne(
{ _id: boardId },
{ $unset: { fixMissingListsCompleted: 1, fixMissingListsCompletedAt: 1, comprehensiveMigrationCompleted: 1 } },
);
}
return { fixed, unresolved: 0 };
},
},
{
// add-checklist-items (v0.79): before 2018 checklist items lived embedded
// in checklist.items[]; current WeKan reads ONLY the ChecklistItems
// collection, so embedded items are invisible although their text is in
// the database. Extract them exactly like the original migration did,
// then $unset the embedded array. The find is targeted ('items.0' exists),
// so only affected checklists are ever read.
name: 'checklist-items-embedded',
async check(db) {
return !!await db.collection('checklists').findOne({ 'items.0': { $exists: true } }, { projection: { _id: 1 } });
},
async run(db) {
let fixed = 0;
const now = new Date();
const cursor = db.collection('checklists').find({ 'items.0': { $exists: true } });
for await (const checklist of cursor) {
const items = (checklist.items || []).slice().sort((a, b) => (a.sort || 0) - (b.sort || 0));
// one query for already-extracted duplicates (interrupted-boot resume)
const existing = await db.collection('checklistItems')
.find({ checklistId: String(checklist._id) }, { projection: { title: 1, sort: 1, isFinished: 1 } }).toArray();
const seen = new Set(existing.map(i => `${i.title}\u0000${i.sort}\u0000${!!i.isFinished}`));
const docs = [];
for (let index = 0; index < items.length; index++) {
const item = items[index] || {};
const doc = {
_id: randomId(),
title: item.title ? item.title : 'Checklist',
sort: index,
isFinished: !!item.isFinished,
checklistId: String(checklist._id),
cardId: String(checklist.cardId || ''),
createdAt: now,
modifiedAt: now,
};
if (seen.has(`${doc.title}\u0000${doc.sort}\u0000${doc.isFinished}`)) continue;
docs.push(doc);
}
if (docs.length) {
// batched insert — one round trip per checklist, not per item
await db.collection('checklistItems').insertMany(docs, { ordered: false });
fixed += docs.length;
}
await db.collection('checklists').updateOne({ _id: checklist._id }, { $unset: { items: 1 } });
}
await cursor.close();
return { fixed, unresolved: 0 };
},
},
{
// mutate-boardIds-in-customfields (v2.49): custom field definitions used a
// scalar boardId; everything in current WeKan queries the boardIds array,
// so old definitions (names, dropdown texts) and their card values were
// orphaned.
name: 'customfields-boardIds',
async check(db) {
return !!await db.collection('customFields').findOne({ boardId: { $exists: true } }, { projection: { _id: 1 } });
},
async run(db) {
let fixed = 0;
const cursor = db.collection('customFields').find({ boardId: { $exists: true } });
for await (const cf of cursor) {
const update = { $unset: { boardId: '' } };
if (!Array.isArray(cf.boardIds) || cf.boardIds.length === 0) {
update.$set = { boardIds: [String(cf.boardId)] };
}
await db.collection('customFields').updateOne({ _id: cf._id }, update);
fixed++;
}
await cursor.close();
return { fixed, unresolved: 0 };
},
},
{
// The add-*-allowed steps (v2.9–v5.43): schema defaults fire on insert
// only, and the card-details templates do raw property access — a missing
// allows* flag hides existing description/checklist/comment/attachment
// text even though it is all still in the database. Backfill the
// defaultValue:true flags with server-side updateMany batches.
name: 'board-allows-defaults',
async check(db) {
return !!await db.collection('boards').findOne(
{ $or: BOARD_ALLOWS_TRUE_DEFAULTS.map(f => ({ [f]: { $exists: false } })) },
{ projection: { _id: 1 } },
);
},
async run(db) {
let fixed = 0;
for (const f of BOARD_ALLOWS_TRUE_DEFAULTS) {
const r = await db.collection('boards').updateMany(
{ [f]: { $exists: false } },
{ $set: { [f]: true } },
);
fixed += (r && r.modifiedCount) || 0;
}
return { fixed, unresolved: 0 };
},
},
{
// add-member-isactive-field: members without isActive are treated as
// INACTIVE by isActiveMember()/activeMembers — pre-2015 board members
// were silently denied access. Explicit isActive:false is respected.
// $elemMatch keeps the query server-side; if the backend rejects it we
// fall back to a projection scan of boards (a small collection).
name: 'board-members-isactive',
async check(db, ctx) {
try {
return !!await db.collection('boards').findOne(
{ members: { $elemMatch: { isActive: { $exists: false } } } },
{ projection: { _id: 1 } },
);
} catch (e) {
ctx.log(`board-members-isactive: $elemMatch probe unsupported (${e.message}); scanning boards`);
const cursor = db.collection('boards').find({ 'members.0': { $exists: true } }, { projection: { members: 1 } });
for await (const b of cursor) {
if ((b.members || []).some(m => m && m.isActive === undefined)) { await cursor.close(); return true; }
}
return false;
}
},
async run(db) {
let fixed = 0;
let cursor;
try {
cursor = db.collection('boards').find(
{ members: { $elemMatch: { isActive: { $exists: false } } } },
{ projection: { members: 1 } },
);
// touch the cursor so an unsupported operator throws HERE
await cursor.hasNext?.();
} catch {
cursor = db.collection('boards').find({ 'members.0': { $exists: true } }, { projection: { members: 1 } });
}
for await (const b of cursor) {
const members = b.members || [];
if (!members.some(m => m && m.isActive === undefined)) continue;
const patched = members.map(m => (m && m.isActive === undefined ? { ...m, isActive: true } : m));
await db.collection('boards').updateOne({ _id: b._id }, { $set: { members: patched } });
fixed++;
}
await cursor.close();
return { fixed, unresolved: 0 };
},
},
{
// lowercase-board-permission: LibreBoard-era 'PUBLIC'/'PRIVATE' values —
// current code compares permission === 'public', so an uppercase public
// board silently became member-only.
name: 'board-permission-lowercase',
async check(db) {
return !!await db.collection('boards').findOne(
{ permission: { $in: ['PUBLIC', 'PRIVATE', 'Public', 'Private'] } },
{ projection: { _id: 1 } },
);
},
async run(db) {
let fixed = 0;
for (const [from, to] of [['PUBLIC', 'public'], ['Public', 'public'], ['PRIVATE', 'private'], ['Private', 'private']]) {
const r = await db.collection('boards').updateMany({ permission: from }, { $set: { permission: to } });
fixed += (r && r.modifiedCount) || 0;
}
return { fixed, unresolved: 0 };
},
},
{
// fs-path-heal: filesystem-stored attachments/avatars whose recorded path
// predates the current layout. WeKan's historical fs locations/naming:
// v6.10-18 WRITABLE_PATH/uploads/<coll>/<name> (Meteor-Files temp)
// v6.10 WRITABLE_PATH/<oldId>-<name> (CFS->ostrio migration)
// v6.19-8.4x WRITABLE_PATH/<coll>/<id>[-original-...] (pre-/files layout)
// current <files root>/<coll>/<id> or <id>_<name> or <id>-<version>-<name>
// A record whose version.path no longer exists gets its binary located in
// those places and copied/repointed into the current directory. Records
// with NO storage flag and NO gridFsFileId (failed v6.10-18 GridFS copies)
// are healed the same way. Marked done only when nothing is left broken,
// so a temporarily-unmounted volume retries on the next boot.
name: 'fs-path-heal',
async check(db, ctx) {
return (await findBrokenFsVersions(db, ctx, true)).length > 0;
},
async run(db, ctx) {
const broken = await findBrokenFsVersions(db, ctx, false);
let fixed = 0, unresolved = 0;
for (const item of broken) {
const healed = await healOne(db, ctx, item);
if (healed) fixed++; else unresolved++;
}
return { fixed, unresolved };
},
},
];
// Visible swimlanes (not archived, not templates) — the ones the Swimlanes
// view actually renders. Swimlanes are few, so one projected fetch is cheap.
// byId: swimlaneId -> boardId; firstOfBoard: boardId -> swimlaneId (by sort).
async function loadVisibleSwimlanes(db) {
const byId = new Map();
const firstOfBoard = new Map();
const all = await db.collection('swimlanes')
.find({}, { projection: { _id: 1, boardId: 1, archived: 1, type: 1, sort: 1 } }).toArray();
for (const s of all.sort((a, b) => (a.sort || 0) - (b.sort || 0))) {
if (s.archived === true || s.type === 'template-swimlane') continue;
byId.set(String(s._id), String(s.boardId));
if (!firstOfBoard.has(String(s.boardId))) firstOfBoard.set(String(s.boardId), String(s._id));
}
return { byId, firstOfBoard };
}
// A version needs healing when it is (or defaults to) filesystem storage but
// its recorded path does not exist on disk.
function versionNeedsHeal(doc, version) {
if (!version || typeof version !== 'object') return false;
const hasGridRef = !!(version.meta && version.meta.gridFsFileId);
const isFs = version.storage === 'fs' || (!version.storage && !hasGridRef);
if (!isFs) return false;
if (version.path && fs.existsSync(version.path)) return false;
return true;
}
async function findBrokenFsVersions(db, ctx, firstOnly) {
const out = [];
for (const coll of ['attachments', 'avatars']) {
// The disk check cannot be expressed as a query, but the scan reads only
// the metadata fields it needs; attachments collections hold metadata
// documents (small), not binaries.
const cursor = db.collection(coll).find({}, { projection: { name: 1, versions: 1 } });
for await (const doc of cursor) {
for (const [versionName, version] of Object.entries(doc.versions || {})) {
if (versionNeedsHeal(doc, version)) {
out.push({ coll, doc, versionName, version });
if (firstOnly) { await cursor.close(); return out; }
}
}
}
await cursor.close();
}
return out;
}
async function healOne(db, ctx, { coll, doc, versionName, version }) {
const destDir = coll === 'avatars' ? ctx.paths.avatars : ctx.paths.attachments;
const writable = ctx.paths.writablePath;
const filesRoot = path.dirname(destDir); // <writable>[/files]
const id = String(doc._id);
const name = doc.name || id;
const base = version.path ? path.basename(String(version.path).replace(/\\/g, '/')) : '';
const candidates = [];
if (base) candidates.push(path.join(destDir, base));
candidates.push(path.join(destDir, id)); // v6.85+ naming
candidates.push(path.join(destDir, `${id}-${versionName}-${name}`)); // moveToStorage naming
candidates.push(path.join(destDir, `${id}_${name}`)); // importer naming
// v6.10-18 temp dir and v6.10 CFS->ostrio temp files lived directly under
// WRITABLE_PATH; on layouts where the current root added '/files', check both.
for (const root of new Set([filesRoot, writable])) {
if (base) candidates.push(path.join(root, 'uploads', coll, base));
candidates.push(path.join(root, 'uploads', coll, id));
candidates.push(path.join(root, `${id}-${name}`));
if (base) candidates.push(path.join(root, coll, base)); // pre-/files layout
candidates.push(path.join(root, coll, id));
}
let found = null;
for (const c of candidates) {
try { if (fs.statSync(c).isFile()) { found = c; break; } } catch { /* next */ }
}
if (!found) {
// last resort: prefix scan of the current directory
try {
const p1 = `${id}-${versionName}-`, p2 = `${id}-original-`, p3 = `${id}_`;
const hit = fs.readdirSync(destDir).find(e => e === id || e.startsWith(p1) || e.startsWith(p2) || e.startsWith(p3));
if (hit) found = path.join(destDir, hit);
} catch { /* ignore */ }
}
if (!found) return false;
let finalPath = found;
if (path.resolve(path.dirname(found)) !== path.resolve(destDir)) {
// copy (never move — the old dir may be shared/backed up) into the current layout
finalPath = path.join(destDir, `${id}-${versionName}-${path.basename(found)}`.slice(0, 220));
try {
fs.mkdirSync(destDir, { recursive: true });
fs.copyFileSync(found, finalPath);
} catch (e) {
ctx.log(`heal ${coll}/${id}: copy failed: ${e.message}`);
return false;
}
}
const set = {
[`versions.${versionName}.path`]: finalPath,
[`versions.${versionName}.storage`]: 'fs',
};
try { set[`versions.${versionName}.size`] = fs.statSync(finalPath).size; } catch { /* keep */ }
if (versionName === 'original') set.path = finalPath;
await db.collection(coll).updateOne({ _id: doc._id }, { $set: set });
return true;
}
// ── orchestrator ─────────────────────────────────────────────────────────────
/**
* Check what is already migrated and migrate all the rest.
*
* VERSION-GATED: the marker records {version, at} of the last successful
* re-check. While the running WeKan version is unchanged, the whole run is
* ONE findOne — a full re-check is mandatory only after a release upgrade
* (or with options.force / WEKAN_FORCE_SCHEMA_UPGRADE=true). A run that
* leaves unresolved work (or a failed step) does NOT record the version, so
* the next boot re-checks.
*
* @param db mongodb driver Db (MongoDB or FerretDB — same wire protocol)
* @param options {log?, writablePath?, appVersion?, force?}
* @return {{ran: string[], skipped: string[], results: Object, gated: boolean}}
*/
async function runSchemaUpgrade(db, options = {}) {
const log = options.log || (() => {});
const appVersion = options.appVersion || '';
const force = options.force || process.env.WEKAN_FORCE_SCHEMA_UPGRADE === 'true';
const ctx = {
log,
paths: computeStoragePaths(options.writablePath !== undefined ? options.writablePath : process.env.WRITABLE_PATH),
};
const markerColl = db.collection(MARKER_COLL);
const marker = (await markerColl.findOne({ _id: MARKER_ID })) || {};
resetState(appVersion);
upgradeState.lastCheck = marker.lastCheck || null;
// Migration dashboards show the Admin Panel product name when one is set.
try {
const s = await db.collection('settings').findOne({ productName: { $exists: true } }, { projection: { productName: 1 } });
if (s && typeof s.productName === 'string' && s.productName.trim()) upgradeState.product = s.productName.trim();
} catch { /* keep default */ }
// Version gate: unchanged release + previous re-check was clean -> one findOne, done.
if (!force && marker.lastCheck && marker.lastCheck.version === appVersion) {
log(`schema already re-checked for version ${appVersion} at ${marker.lastCheck.at}; skipping (set WEKAN_FORCE_SCHEMA_UPGRADE=true to force).`);
upgradeState.running = false;
upgradeState.gated = true;
upgradeState.finishedAt = new Date();
for (const s of steps) upgradeState.steps[s.name].status = 'skipped';
return { ran: [], skipped: steps.map(s => s.name), results: {}, gated: true };
}
const ran = [], skipped = [], results = {};
let clean = true;
for (const step of steps) {
const st = upgradeState.steps[step.name];
upgradeState.currentStep = step.name;
try {
st.status = 'checking';
const needed = await step.check(db, ctx);
if (!needed) {
st.status = 'skipped';
skipped.push(step.name);
continue;
}
log(`running ${step.name} ...`);
st.status = 'running';
const res = await step.run(db, ctx);
st.status = 'done';
st.fixed = res.fixed;
st.unresolved = res.unresolved || 0;
results[step.name] = res;
ran.push(step.name);
log(`${step.name}: fixed ${res.fixed}${res.unresolved ? `, unresolved ${res.unresolved} (will retry next start)` : ''}`);
if (res.unresolved) clean = false;
// per-step record for the dashboard/history
await markerColl.updateOne(
{ _id: MARKER_ID },
{ $set: { [`steps.${step.name}`]: { doneAt: new Date(), fixed: res.fixed, unresolved: res.unresolved || 0 } } },
{ upsert: true },
).catch(() => {});
} catch (e) {
// Never break startup over an upgrade step — log and retry next boot.
clean = false;
st.status = 'error';
st.error = e.message;
log(`${step.name} FAILED (will retry next start): ${e.message}`);
results[step.name] = { error: e.message };
}
}
upgradeState.currentStep = '';
upgradeState.running = false;
upgradeState.finishedAt = new Date();
// Record the re-check ONLY when everything is clean — an unresolved or
// failed step keeps the version un-stamped so the next boot re-checks.
if (clean) {
const lastCheck = { version: appVersion, at: new Date() };
upgradeState.lastCheck = lastCheck;
await markerColl.updateOne(
{ _id: MARKER_ID },
{ $set: { lastCheck } },
{ upsert: true },
).catch(e => log('marker lastCheck update failed: ' + e.message));
}
return { ran, skipped, results, gated: false };
}
module.exports = {
runSchemaUpgrade,
getUpgradeState,
steps,
BOARD_ALLOWS_TRUE_DEFAULTS,
randomId,
versionNeedsHeal,
MARKER_COLL,
MARKER_ID,
};
server/migrations/ensureValidSwimlaneIds.js
+23
View File
@@ -210,6 +210,29 @@ async function getOrCreateRescuedSwimlane(boardId) {
function addSwimlaneIdValidationHooks() {
// Card insert hook
Cards.before.insert(async function(userId, doc) {
// #1959/#1971: a client-supplied swimlaneId can point at a DELETED or
// ARCHIVED swimlane (adding cards in the Lists view reuses stale ids,
// and dragging re-assigns to a default swimlane that may be archived).
// Such cards exist but never render in the Swimlanes view. Validate the
// id and fall back to the board's first VISIBLE swimlane.
if (doc.swimlaneId) {
const valid = await Swimlanes.findOneAsync({
_id: doc.swimlaneId,
boardId: doc.boardId,
archived: false,
});
if (!valid) {
const fallback = await Swimlanes.findOneAsync({
boardId: doc.boardId,
type: { $ne: 'template-swimlane' },
archived: false,
}, { sort: { sort: 1 } });
if (fallback) {
console.warn(`Card insert pointed at a missing/archived swimlane ${doc.swimlaneId}; using ${fallback._id}`);
doc.swimlaneId = fallback._id;
}
}
}
if (!doc.swimlaneId) {
const board = await Boards.findOneAsync(doc.boardId);
if (board) {
server/models/boards.js
+43 -4
View File
@@ -8,6 +8,11 @@ import { Authentication } from '/server/authentication';
import { sendJsonResult } from '/server/apiMiddleware';
import { allowIsBoardAdmin, boardMemberRoleToFlags } from '/server/lib/utils';
import { reconcileBoardTeamMembers } from '/models/lib/reconcileBoardTeamMembers';
import {
isInvitedToBoard,
planQuitBoard,
planAcceptInvite,
} from '/models/lib/boardInvites';
import { buildBoardLabel } from '/models/lib/restLabel';
import { LABEL_COLORS } from '/models/metadata/colors';
import { filterUserBoards } from '/server/lib/boardListFilter';
@@ -204,12 +209,31 @@ Meteor.methods({
}
const userId = this.userId;
const index = board.memberIndex(userId);
if (index < 0) {
throw new Meteor.Error('error-board-notAMember');
const user = await ReactiveCache.getUser(userId);
// #4730: quitting (or declining an invitation to) a board must also clear
// the pending invitation, otherwise a follow-up acceptInvite call — the
// decline flow on the All Boards page issues one — re-activates the
// membership and the declined board stays in the personal overview. A user
// holding only a stale invitation (no member entry, e.g. after an admin
// removed them) may also quit: their invitation is cleared instead of
// throwing error-board-notAMember, so the board can always be removed
// from the personal overview.
const plan = planQuitBoard({
isMember: board.memberIndex(userId) >= 0,
isInvited: isInvitedToBoard(user && user.profile, boardId),
});
if (!plan.ok) {
throw new Meteor.Error(plan.error);
}
await board.removeMember(userId);
if (plan.pullInvite) {
await Users.updateAsync(userId, {
$pull: { 'profile.invitedBoards': boardId },
});
}
if (plan.removeMember) {
await board.removeMember(userId);
}
return true;
},
@@ -220,6 +244,20 @@ Meteor.methods({
throw new Meteor.Error('error-board-doesNotExist');
}
// #4730: only a user actually holding an invitation to this board may
// (re)activate their member entry. Previously this method set
// members.$.isActive = true unconditionally, which (a) re-activated the
// membership right after the decline flow's quitBoard call, so a declined
// board stayed in the personal overview, and (b) let a member an admin had
// removed re-add themselves by calling this method directly.
const user = await ReactiveCache.getUser(this.userId);
const plan = planAcceptInvite({
isInvited: isInvitedToBoard(user && user.profile, boardId),
});
if (!plan.ok) {
return false;
}
await Meteor.users.updateAsync(this.userId, {
$pull: {
'profile.invitedBoards': boardId,
@@ -238,6 +276,7 @@ Meteor.methods({
},
},
);
return true;
},
async myLabelNames() {
server/models/settings.js
+60 -13
View File
@@ -11,6 +11,12 @@ import { ensureIndex } from '/server/lib/mongoStartup';
import { Authentication } from '/server/authentication';
import { sendJsonResult } from '/server/apiMiddleware';
const { parseCardsLoadingEnv } = require('/models/lib/cardsLoading');
const {
normalizeInviteEmail,
isInvitationCodeSendable,
buildReinviteModifier,
shouldRemoveInvitationOnEmailFailure,
} = require('/models/lib/invitationCodeEmail');
const getReactiveCache = () => require('/imports/reactiveCache').ReactiveCache;
const getTAPi18n = () => require('/imports/i18n').TAPi18n;
@@ -45,8 +51,17 @@ function loadOidcConfig(service) {
return ServiceConfiguration.configurations.findOneAsync({ service });
}
async function sendInvitationEmail(_id) {
async function sendInvitationEmail(_id, { isNewInvitation = true } = {}) {
const icode = await getReactiveCache().getInvitationCode(_id);
// #4043: never send an invitation email without a code that will validate at
// sign-up (the sign-up lookup requires { code: <string>, valid: true }).
// Fail loudly instead of mailing a dead code.
if (!isInvitationCodeSendable(icode)) {
throw new Meteor.Error(
'invitation-code-invalid',
'Invitation email not sent: the invitation code is missing or no longer valid',
);
}
const author = await getReactiveCache().getCurrentUser();
try {
const authorUser = await getReactiveCache().getUser(icode.authorId);
@@ -74,7 +89,12 @@ async function sendInvitationEmail(_id) {
language: lang,
});
} catch (e) {
await InvitationCodes.removeAsync(_id);
// #4043: only roll back a code created by this very invite. A pre-existing
// invitation was already delivered in an earlier email; deleting it here
// would silently invalidate the code that email carries.
if (shouldRemoveInvitationOnEmailFailure({ isNewInvitation })) {
await InvitationCodes.removeAsync(_id);
}
throw new Meteor.Error('email-fail', e.message);
}
}
@@ -230,7 +250,12 @@ Meteor.methods({
throw new Meteor.Error('not-allowed');
}
for (const email of emails) {
for (const rawEmail of emails) {
// #4043: store the invitee address lowercase — the sign-up form
// lowercases the typed address and MongoDB string matching is case
// sensitive, so a mixed-case invitation document never matches at
// registration ("The invitation code doesn't exist").
const email = normalizeInviteEmail(rawEmail);
if (email && SimpleSchema.RegEx.Email.test(email)) {
const userExist = await getReactiveCache().getUser({ email });
if (userExist) {
@@ -243,12 +268,30 @@ Meteor.methods({
const invitation = await getReactiveCache().getInvitationCode({ email });
if (invitation) {
await InvitationCodes.updateAsync(invitation, {
$set: { boardsToBeInvited: boards },
});
await sendInvitationEmail(invitation._id);
// #4043: a re-invite must never re-send a stale code. When the
// stored invitation is no longer valid (or has no usable code), a
// fresh code overwrites it with valid restored to true, so the
// emailed code always passes the sign-up lookup
// { code, email, valid: true }.
const modifier = buildReinviteModifier(invitation, boards, () =>
getRandomNum(100000, 999999),
);
const updated = await InvitationCodes.updateAsync(
invitation._id,
modifier,
);
if (!updated) {
rc = -1;
throw new Meteor.Error(
'invitation-generated-fail',
'Failed to update invitation code',
);
}
await sendInvitationEmail(invitation._id, { isNewInvitation: false });
} else {
const code = getRandomNum(100000, 999999);
// String(...) so the stored code always matches the (string) code
// typed into the sign-up form, without relying on schema autoConvert.
const code = String(getRandomNum(100000, 999999));
const _id = await InvitationCodes.insertAsync({
code,
email,
@@ -397,11 +440,15 @@ Meteor.methods({
return !(process.env.PASSWORD_LOGIN_ENABLED === 'false');
},
isOidcRedirectionEnabled() {
return (
process.env.OIDC_REDIRECTION_ENABLED === 'true' &&
Object.keys(loadOidcConfig('oidc')).length > 0
);
// #5695: loadOidcConfig() returns a Promise since the Meteor 3 async
// migration (findOneAsync). Object.keys(<Promise>) is always [], so this
// method returned false even with OIDC_REDIRECTION_ENABLED=true and a
// configured oidc service, silently disabling the auto-redirect login.
// Await the config before inspecting it.
async isOidcRedirectionEnabled() {
if (process.env.OIDC_REDIRECTION_ENABLED !== 'true') return false;
const config = await loadOidcConfig('oidc');
return !!config && Object.keys(config).length > 0;
},
async getServiceConfiguration(service) {
server/models/users.js
+68 -1
View File
@@ -25,6 +25,45 @@ import { expiredNotificationActivityIds } from '/models/lib/notificationCleanup'
import { chooseInviteEmailLanguage } from '/models/lib/inviteEmailLanguage';
import { paginateDomains } from '/models/lib/domainTablePage';
import { orgsToAutoAddForEmail } from '/models/lib/orgAutoAddByDomain';
import {
gainedTeamIds,
newTeamBoardMemberEntry,
boardsToAddMemberTo,
} from '/models/lib/teamBoardMemberSync';
// #4593: when a team is assigned to a board (addBoardTeamPopup), every user in
// the team at that moment is pushed into board.members as a normal member — but
// a user added to the team afterwards never was, so they ended up with strictly
// less authority than their teammates: the publications let them view the board
// (teams.teamId match) while every interaction gate (board.hasMember,
// allowIsBoardMember*, Attachments.protected, board.isVisibleBy for export)
// only looks at board.members. This grants a user who just gained team(s) the
// same normal membership on every board those teams are assigned to. Existing
// member entries (even deactivated ones) are never touched, template boards are
// skipped, and team REMOVAL intentionally does not remove board members (a
// member may also have been invited individually; explicit cleanup remains the
// board admin's removeBoardTeamPopup action). Failures are logged, never fatal
// to the user update that triggered the sync.
const addUserToTeamBoards = async (userId, oldTeams, newTeams) => {
try {
const gained = gainedTeamIds(oldTeams, newTeams);
if (!gained.length) return;
const boards = await ReactiveCache.getBoards(
{ teams: { $elemMatch: { teamId: { $in: gained }, isActive: true } } },
{ fields: { _id: 1, type: 1, teams: 1, members: 1 } },
);
for (const boardId of boardsToAddMemberTo(boards, userId, gained)) {
// Guarded push: never create a duplicate entry if the user became a
// member through another path between the read above and this write.
await Boards.updateAsync(
{ _id: boardId, 'members.userId': { $ne: userId } },
{ $push: { members: newTeamBoardMemberEntry(userId) } },
);
}
} catch (error) {
console.error('addUserToTeamBoards failed:', error);
}
};
const getTAPi18n = () => require('/imports/i18n').TAPi18n;
const isSandstorm =
@@ -261,6 +300,12 @@ Meteor.methods({
if (updateData.orgs !== undefined) updateObject.orgs = updateData.orgs;
await Users.updateAsync(targetUserId, { $set: updateObject });
// #4593: a user who just gained team(s) must also gain membership of the
// boards those teams are assigned to, like the team's original members did.
if (updateData.teams !== undefined) {
await addUserToTeamBoards(targetUserId, targetUser.teams, updateData.teams);
}
},
async setListSortBy(value) {
@@ -698,6 +743,10 @@ Meteor.methods({
teams: userTeamsArray,
},
});
// #4593: a user created directly into team(s) must gain membership of
// the boards those teams are assigned to, like the teams' existing
// members already have.
await addUserToTeamBoards(user._id, [], userTeamsArray);
}
}
}
@@ -1219,7 +1268,25 @@ Accounts.onCreateUser(async (options, user) => {
existingUser.services[service] = user.services[service];
existingUser.emails = user.emails;
existingUser.username = user.username;
existingUser.profile = user.profile;
// #4560 ("User Profile is reset when changing the authentication method
// from LDAP to OIDC"): merge the profile, do not replace it. The previous
// wholesale `existingUser.profile = user.profile` re-inserted the linked
// account with ONLY the freshly built OIDC profile (initials, fullname,
// boardView), wiping profile.avatarUrl, profile.templatesBoardId and the
// template swimlane ids, language, notification/preference fields, etc. —
// the reporter's "templates and avatars disappeared". Keep every existing
// profile field, fill gaps from the OIDC-derived profile, and take the
// provider's fullname/initials only when the provider actually asserted a
// fullname (otherwise user.profile.fullname is just a fallback to the
// username and must not overwrite a real stored name).
existingUser.profile = {
...user.profile,
...(existingUser.profile || {}),
};
if (user.services.oidc.fullname) {
existingUser.profile.fullname = user.profile.fullname;
existingUser.profile.initials = user.profile.initials;
}
existingUser.authenticationMethod = user.authenticationMethod;
await Meteor.users.removeAsync({ _id: user._id });
server/publications/boards.js
+105
View File
@@ -10,7 +10,13 @@ import Org from "../../models/org";
import Team from "../../models/team";
import Attachments from '../../models/attachments';
import Boards from '/models/boards';
import Cards from '/models/cards';
import { localizeBoardMemberAvatars } from '/server/lib/localizeAvatar';
import {
showsCardCounterList,
countCardsByListId,
buildBoardTileData,
} from '/models/lib/boardTileData';
// When CARDS_LOADING=lazy (Admin Panel / Features), the board publication must
// NOT ship every card / checklist into the client's minimongo — each list loads
@@ -288,6 +294,105 @@ Meteor.methods({
const ids = boards.slice(start, start + perPage).map(b => b._id);
return { ids, total };
},
// #5174 / #4825 (follow-up to #4214): the data behind the All Boards board
// tiles — the per-list card-count line and the member avatar row. Computed
// ONCE per request, entirely server-side (one boards query, one lists query,
// one grouped card count), and returned as a plain map keyed by board id.
// The client stores it in a ReactiveVar it sets exactly once, instead of the
// old reactive getLists()/getCards() cursors inside the tile helpers that
// caused the "icons random dance" (#4214) and were therefore stubbed out —
// which in turn hid the counters/avatars for everyone.
//
// The per-board opt-in flags (allowsCardCounterList / allowsBoardMemberList,
// board sidebar "Show at All Boards page") are resolved here with strict
// semantics — missing flag means OFF — so every tile renders consistently
// (#4825). The Admin Panel hideCardCounterList / hideBoardMemberList
// settings remain enforced by the template on top of this.
async getAllBoardsTileData() {
const userId = this.userId;
if (!Match.test(userId, String) || !userId) {
return {};
}
const user = await ReactiveCache.getUser(userId);
if (!user) {
return {};
}
// Same visibility selector as getAllBoardsPage / the `boards` publication,
// restricted to real boards (template-container tiles never show these).
const selector = {
archived: false,
type: 'board',
$or: [
{ permission: 'public' },
{ members: { $elemMatch: { userId, isActive: true } } },
{ orgs: { $elemMatch: { orgId: { $in: user.orgIds() }, isActive: true } } },
{ teams: { $elemMatch: { teamId: { $in: user.teamIds() }, isActive: true } } },
{ domains: { $elemMatch: { domain: { $in: user.emailDomains() }, isActive: true } } },
],
};
let boards = await ReactiveCache.getBoards(
selector,
{
fields: {
_id: 1,
members: 1,
allowsCardCounterList: 1,
allowsBoardMemberList: 1,
},
},
true,
);
boards = typeof boards.fetchAsync === 'function'
? await boards.fetchAsync()
: (typeof boards.fetch === 'function' ? boards.fetch() : boards);
// Lists and card counts are only needed for boards that opted in to the
// card-counter line.
const countedBoardIds = boards
.filter(board => showsCardCounterList(board))
.map(board => board._id);
let lists = [];
const cardCounts = {};
if (countedBoardIds.length) {
lists = await ReactiveCache.getLists(
{ boardId: { $in: countedBoardIds }, archived: false },
{ fields: { _id: 1, boardId: 1, title: 1, sort: 1 } },
true,
);
lists = typeof lists.fetchAsync === 'function'
? await lists.fetchAsync()
: (typeof lists.fetch === 'function' ? lists.fetch() : lists);
try {
// One grouped count for all boards on the page instead of a count per
// list (and instead of shipping every card to the client, which is
// what the old reactive helpers effectively did).
const rows = await Cards.rawCollection()
.aggregate([
{ $match: { boardId: { $in: countedBoardIds }, archived: false } },
{ $group: { _id: '$listId', n: { $sum: 1 } } },
])
.toArray();
rows.forEach(row => {
if (row && row._id) cardCounts[row._id] = row.n;
});
} catch (error) {
// Aggregation may be unavailable on some storage backends; fall back
// to fetching only the cards' listId and folding in memory.
const cards = await Cards.find(
{ boardId: { $in: countedBoardIds }, archived: false },
{ fields: { listId: 1 } },
).fetchAsync();
Object.assign(cardCounts, countCardsByListId(cards));
}
}
return buildBoardTileData(boards, lists, cardCounts);
},
});
Meteor.publish('archivedBoards', async function() {
server/startupSchemaUpgrade.js
+96
View File
@@ -0,0 +1,96 @@
// ============================================================================
// Startup schema upgrade (#6473 follow-up): on start, check that data from
// every old WeKan version is migrated to the newest database structure and
// migrate only what is missing — all platforms (Snap, Docker, Sandstorm,
// source), both databases (MongoDB and FerretDB).
//
// VERSION-GATED: the `_wekan_migration` marker stores the WeKan version and
// datetime of the previous successful re-check. While the version is
// unchanged a boot costs ONE findOne; a full re-check is mandatory only after
// a new WeKan release. Force one with WEKAN_FORCE_SCHEMA_UPGRADE=true; opt
// out entirely with WEKAN_SKIP_SCHEMA_UPGRADE=true.
//
// The upgrade runs in the BACKGROUND (WeKan serves immediately) and publishes
// live progress at /schema-upgrade-status (HTML dashboard, ?json for raw
// state). The dashboard shows the Admin Panel product name when one is set.
//
// The actual steps live in server/lib/schemaUpgradeSteps.js (pure, driver-
// based, unit-tested by tests/schemaUpgradeSteps.test.cjs, batched
// updateMany/distinct queries so big databases stay fast).
// ============================================================================
import { Meteor } from 'meteor/meteor';
import { MongoInternals } from 'meteor/mongo';
import { WebApp } from 'meteor/webapp';
import { runSchemaUpgrade, getUpgradeState } from '/server/lib/schemaUpgradeSteps';
function escHtml(s) {
return String(s).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
// Live migration dashboard — same spirit as the MongoDB→FerretDB importer
// dashboards, available on every platform while WeKan runs.
WebApp.handlers.get('/schema-upgrade-status', (req, res) => {
const state = getUpgradeState();
if ('json' in (req.query || {})) {
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(state, null, 2));
return;
}
const rows = Object.entries(state.steps).map(([name, s]) => {
const color = s.status === 'error' ? '#f77' : s.status === 'running' || s.status === 'checking' ? '#fb7' : s.status === 'done' ? '#7f7' : '#aaa';
return `<tr><td>${escHtml(name)}</td><td style="color:${color}">${escHtml(s.status)}</td>`
+ `<td>${s.fixed || 0}</td><td class="${s.unresolved ? 'fail' : ''}">${s.unresolved || 0}</td>`
+ `<td>${s.error ? escHtml(s.error) : ''}</td></tr>`;
}).join('');
const product = escHtml(state.product || 'WeKan');
const phase = state.running
? `Checking / migrating… (${escHtml(state.currentStep || '')})`
: state.gated
? `Already re-checked for version ${escHtml(state.appVersion)} — nothing to do`
: state.finishedAt ? 'Completed' : 'Not started yet';
res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' });
res.end(`<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">${state.running ? '<meta http-equiv="refresh" content="3">' : ''}
<title>${product} Schema Upgrade</title>
<style>body{font-family:monospace;background:#111;color:#ddd;padding:1em 2em}h1{color:#7bf}
table{border-collapse:collapse}td,th{padding:4px 10px;border-bottom:1px solid #333;text-align:left}
th{color:#aaa;font-size:.85em}.fail{color:#f77}</style></head><body>
<h1>${product} Schema Upgrade</h1>
<p>Status: <strong>${phase}</strong></p>
<p style="color:#aaa">WeKan version: ${escHtml(state.appVersion || '?')}
${state.lastCheck ? ` &nbsp;·&nbsp; previous re-check: ${escHtml(state.lastCheck.version)} at ${escHtml(String(state.lastCheck.at))}` : ''}
${state.startedAt ? ` &nbsp;·&nbsp; started: ${escHtml(String(state.startedAt))}` : ''}
${state.finishedAt ? ` &nbsp;·&nbsp; finished: ${escHtml(String(state.finishedAt))}` : ''}</p>
<table><tr><th>Step</th><th>Status</th><th>Fixed</th><th>Unresolved</th><th>Error</th></tr>${rows}</table>
<p style="color:#aaa">A re-check is mandatory only after a new ${product} release; while the version
is unchanged, startup costs a single database read. Raw state: <a href="/schema-upgrade-status?json" style="color:#7bf">JSON</a></p>
</body></html>`);
});
Meteor.startup(() => {
if (process.env.WEKAN_SKIP_SCHEMA_UPGRADE === 'true') {
console.log('[schema-upgrade] skipped (WEKAN_SKIP_SCHEMA_UPGRADE=true)');
return;
}
// Deliberately NOT awaited: WeKan starts serving immediately; the upgrade
// runs in the background and reports at /schema-upgrade-status.
(async () => {
try {
const db = MongoInternals.defaultRemoteCollectionDriver()?.mongo?.db;
if (!db) {
console.warn('[schema-upgrade] no database handle at startup; skipping (will retry next start)');
return;
}
let appVersion = '';
try { appVersion = require('/package.json').version || ''; } catch { /* keep '' */ }
const { ran, gated } = await runSchemaUpgrade(db, {
appVersion,
log: (...args) => console.log('[schema-upgrade]', ...args),
});
if (!gated && ran.length > 0) {
console.log(`[schema-upgrade] upgraded old-version data: ${ran.join(', ')}`);
}
} catch (error) {
console.error('[schema-upgrade] failed (WeKan continues; retried next start):', error.message);
}
})();
});
snap-src/bin/attachment-repair
Executable
+193
View File
@@ -0,0 +1,193 @@
#!/bin/bash
# attachment-repair — automatic, incremental attachment/avatar repair (#6473).
#
# Older MongoDB → FerretDB migrations silently lost attachments (CollectionFS
# copies.<bucket>.key was never resolved; Meteor-Files records carrying only a
# gridFsFileId reference were skipped). The source MongoDB data was NEVER
# modified, so everything is recoverable. This script runs in the BACKGROUND at
# every WeKan start (wekan-control) when the snap is on FerretDB and, ONCE:
# 1. starts a temporary source mongod (7.x, or the bundled 3.2 for old data)
# on port 27099 — read-only use, the data is never modified;
# 2. runs the migration importer in FILES_ONLY mode against the LIVE FerretDB:
# text collections are left completely untouched, files already migrated
# are verified (record on 'fs' + file on disk) and skipped, and ONLY the
# missing attachments/avatars are migrated;
# 3. on success writes $SNAP_COMMON/.attachments-files-v2-done so it never
# runs again ("v2" matches FILES_VERSION in migrate-mongodb-to-ferretdb.mjs
# — bump both together when a future file-phase fix needs a re-scan).
# A failure leaves the marker absent, so the next start retries. WeKan starts
# and serves normally the whole time; repaired attachments appear as they are
# migrated (Meteor re-polls the queries).
set -u
COMMON="${SNAP_COMMON:-/var/snap/${SNAP_INSTANCE_NAME:-${SNAP_NAME:-wekan}}/common}"
MARKER="$COMMON/.attachments-files-v2-done"
LOCK="$COMMON/.attachment-repair.pid"
SRC_PORT="${REPAIR_SRC_PORT:-27099}" # temp source mongod (same as migration-control)
SRC_PIDFILE="$COMMON/attachment-repair-mongod.pid"
LOGF="$COMMON/attachment-repair-mongod.log"
NODE="$SNAP/bin/node"
DB_EVAL="$SNAP/bin/db-eval"
IMPORTER_MODERN="$SNAP/bin/migrate-mongodb-to-ferretdb.mjs"
IMPORTER_3="$SNAP/bin/migrate-mongo3-to-ferretdb.mjs"
MM="$SNAP/migratemongo" # MongoDB 3.2 tools (amd64)
MM_LIB="$MM/lib/x86_64-linux-gnu"
WAIT_SECS="${REPAIR_WAIT_SECS:-60}" # max wait for the live FerretDB
log() { echo "[attachment-repair] $*"; }
# ── Guards: run at most once, only when there is something to do ─────────────
[ -f "$MARKER" ] && exit 0
if [ -f "$COMMON/.wekan-maintenance" ]; then
log "maintenance mode is on; skipping."
exit 0
fi
# Only meaningful when WeKan runs on FerretDB. Before the switch, the normal
# migration path (migration-control, with the fixed importers) covers files.
if [ "$(snapctl get database 2>/dev/null)" != "ferretdb" ]; then
log "database is not ferretdb; nothing to repair yet."
exit 0
fi
if ! bash "$SNAP/bin/ferretdb-has-data"; then
log "no FerretDB (SQLite) data yet; skipping."
exit 0
fi
# No MongoDB source data left on this machine: there is nothing to repair FROM,
# and never will be — mark done so this never probes again.
if [ ! -f "$COMMON/WiredTiger" ] && ! ls "$COMMON"/collection-*.wt >/dev/null 2>&1; then
log "no MongoDB source data in $COMMON; nothing to repair. Marking done."
touch "$MARKER" 2>/dev/null || true
exit 0
fi
# Single instance across restarts (WeKan can be restarted while a repair runs).
if [ -f "$LOCK" ]; then
p="$(cat "$LOCK" 2>/dev/null)"
if [ -n "$p" ] && kill -0 "$p" 2>/dev/null; then
log "a repair is already running (pid $p); skipping."
exit 0
fi
fi
echo $$ > "$LOCK"
stop_src() {
if [ -f "$SRC_PIDFILE" ]; then
local p; p="$(cat "$SRC_PIDFILE" 2>/dev/null)"
if [ -n "$p" ]; then
"$DB_EVAL" shutdown "mongodb://127.0.0.1:$SRC_PORT/?directConnection=true" >/dev/null 2>&1 || true
sleep 1
kill "$p" 2>/dev/null || true
sleep 1
kill -9 "$p" 2>/dev/null || true
fi
rm -f "$SRC_PIDFILE"
fi
}
cleanup() { stop_src; rm -f "$LOCK"; }
trap cleanup EXIT INT TERM
# ── The LIVE FerretDB is the repair target ────────────────────────────────────
# Same address wekan-control builds MONGO_URL from (defaults 127.0.0.1:27019).
BIND_IP="${MONGODB_BIND_IP:-127.0.0.1}"
DB_PORT="${MONGODB_PORT:-27019}"
TARGET_URL="${REPAIR_TARGET_URL:-mongodb://$BIND_IP:$DB_PORT/wekan}"
waited=0
until "$DB_EVAL" ping "mongodb://$BIND_IP:$DB_PORT/?directConnection=true" >/dev/null 2>&1; do
waited=$((waited + 2))
if [ "$waited" -ge "$WAIT_SECS" ]; then
log "FerretDB not reachable on $BIND_IP:$DB_PORT after ${WAIT_SECS}s; will retry on the next start."
exit 1
fi
sleep 2
done
export LC_ALL=C
export LD_LIBRARY_PATH="$SNAP/lib:$SNAP/lib/x86_64-linux-gnu:$SNAP/lib/aarch64-linux-gnu:${LD_LIBRARY_PATH:-}"
# RAM-aware WiredTiger cache for the temporary source mongod, like
# migration-control: mongod's own ~50%-of-RAM default is too much next to the
# running FerretDB + WeKan (#6454). Quarter of (RAM - 1 GB), min 1, cap 8.
if [ -n "${MIGRATION_MONGOD_CACHE_GB:-}" ]; then
SRC_WT_CACHE_GB="${MIGRATION_MONGOD_CACHE_GB}"
else
_mem_kb=$(awk '/^MemTotal:/{print $2}' /proc/meminfo 2>/dev/null)
[ -z "${_mem_kb:-}" ] && _mem_kb=2097152
_mem_gb=$(( _mem_kb / 1024 / 1024 ))
SRC_WT_CACHE_GB=$(( (_mem_gb - 1) / 4 ))
[ "${SRC_WT_CACHE_GB}" -lt 1 ] && SRC_WT_CACHE_GB=1
[ "${SRC_WT_CACHE_GB}" -gt 8 ] && SRC_WT_CACHE_GB=8
fi
# ── Start the temporary source mongod (7.x first, then the 3.2 reader) ───────
start_mongod() { # $1 = mongod binary, $2 = LD_LIBRARY_PATH prefix, $3 = extra args, $4 = cpu-exec features
rm -f "$SRC_PIDFILE"
LD_LIBRARY_PATH="${2:+$2:}$LD_LIBRARY_PATH" bash "$SNAP/bin/cpu-exec" --features "${4:-}" "$1" \
--dbpath "$COMMON" --bind_ip 127.0.0.1 --port "$SRC_PORT" \
--logpath "$LOGF" --logappend --fork --pidfilepath "$SRC_PIDFILE" ${3:-} >/dev/null 2>&1
}
ready_via_node() {
local i=0 p
while [ $i -lt 45 ]; do
"$DB_EVAL" ping "mongodb://127.0.0.1:$SRC_PORT/?directConnection=true" >/dev/null 2>&1 && return 0
p="$(cat "$SRC_PIDFILE" 2>/dev/null)"
if [ -n "$p" ] && ! kill -0 "$p" 2>/dev/null; then return 1; fi
i=$((i+1)); sleep 1
done
return 1
}
ready_via_legacy() {
local i=0
while [ $i -lt 20 ]; do
LD_LIBRARY_PATH="$MM_LIB:$LD_LIBRARY_PATH" "$MM/bin/mongo" --port "$SRC_PORT" --quiet \
--eval 'db.adminCommand({ping:1}).ok' 2>/dev/null | grep -q 1 && return 0
i=$((i+1)); sleep 1
done
return 1
}
MONGOD7_CPU_FEATURES="x86_64=avx,aarch64=atomics"
rc=1
log "checking whether the source data opens with mongod 7 ..."
if start_mongod "$SNAP/bin/mongod" "" "--wiredTigerCacheSizeGB ${SRC_WT_CACHE_GB}" "$MONGOD7_CPU_FEATURES" && ready_via_node; then
log "modern MongoDB source. Running the incremental FILES_ONLY repair against $TARGET_URL ..."
NODE_PATH="$SNAP/programs/server/node_modules" \
SOURCE_MONGO_URL="mongodb://127.0.0.1:$SRC_PORT/wekan?directConnection=true" \
TARGET_MONGO_URL="$TARGET_URL" \
WRITABLE_PATH="$COMMON" \
FILES_ONLY=true \
MIGRATION_PORT=0 \
"$NODE" "$IMPORTER_MODERN"
rc=$?
else
stop_src
if [ -x "$MM/bin/mongod" ] && [ -x "$MM/bin/mongo" ] && [ -f "$IMPORTER_3" ] \
&& start_mongod "$MM/bin/mongod" "$MM_LIB" "" "" && ready_via_legacy; then
log "MongoDB 3.x source. Running the incremental FILES_ONLY repair against $TARGET_URL ..."
NODE_PATH="$SNAP/programs/server/node_modules" \
MONGO_BIN_DIR="$MM/bin" \
MONGO_LIB="$MM_LIB" \
SRC_PORT="$SRC_PORT" \
SRC_DB="wekan" \
TARGET_MONGO_URL="$TARGET_URL" \
FILES_DIR="$COMMON/files" \
FILES_ONLY=true \
MIGRATION_PORT=0 \
"$NODE" "$IMPORTER_3"
rc=$?
else
log "the MongoDB source data opens with neither mongod 7 nor the bundled 3.2 reader; will retry on the next start."
exit 1
fi
fi
if [ "$rc" -eq 0 ]; then
log "repair completed. Marking done ($MARKER)."
touch "$MARKER" 2>/dev/null || true
exit 0
fi
log "repair did not complete (importer rc=$rc); will retry on the next start."
exit 1
snap-src/bin/migrate-gridfs-to-fs.mjs
+35 -16
View File
@@ -118,36 +118,54 @@ let done = 0, skipped = 0, errors = 0;
async function migrateCollection(collName, destDir, bucketPrefix) {
const filesCollName = `${bucketPrefix}.files`;
const allColls = (await db.listCollections().toArray()).map(c => c.name);
if (!allColls.includes(filesCollName)) return;
// #6473: do NOT return early when the CollectionFS bucket is missing — a pure
// Meteor-Files database (any install born >= v6.10) has only <coll>.files,
// and returning here made this tool a silent no-op for it.
const haveCfsBucket = allColls.includes(filesCollName);
const haveMfBucket = allColls.includes(`${collName}.files`);
if (!haveCfsBucket && !haveMfBucket) return;
console.log(`[gridfs-migrate] Processing ${bucketPrefix} → ${destDir}`);
const bucket = new GridFSBucket(db, { bucketName: bucketPrefix });
console.log(`[gridfs-migrate] Processing ${collName} → ${destDir}`);
// #6473: each source remembers WHICH GridFS bucket holds its binary —
// CollectionFS filerecords point into cfs_gridfs.<coll>, Meteor-Files records
// point into the <coll> bucket. Reading them all through the CFS bucket made
// every Meteor-Files extraction fail.
const cfsBucket = haveCfsBucket ? new GridFSBucket(db, { bucketName: bucketPrefix }) : null;
const mfBucket = haveMfBucket ? new GridFSBucket(db, { bucketName: collName }) : null;
// Try CFS filerecord collection first, then fall back to Meteor-Files docs
const sources = [];
if (allColls.includes(`cfs.${collName}.filerecord`)) {
if (haveCfsBucket && allColls.includes(`cfs.${collName}.filerecord`)) {
const cursor = db.collection(`cfs.${collName}.filerecord`).find({});
for await (const rec of cursor) {
// #6473: resolve the GridFS id from copies.<bucket>.key too (real
// CollectionFS layout), not just original.gridFsFileId.
sources.push({ id: rec._id, gridFsId: resolveCfsGridFsId(rec, collName), name: rec.original?.name });
sources.push({ id: rec._id, gridFsId: resolveCfsGridFsId(rec, collName), name: rec.original?.name, bucket: cfsBucket, filesCollName });
}
await cursor.close();
}
// Also handle Meteor-Files docs marked storage:'gridfs' OR carrying a
// meta.gridFsFileId reference (#6473: the same rule WeKan's getFileStrategy
// uses — records with only the reference were previously missed).
const mfCursor = db.collection(collName).find({ $or: [
{ 'versions.original.storage': 'gridfs' },
{ 'versions.original.meta.gridFsFileId': { $exists: true, $ne: null } },
] });
for await (const doc of mfCursor) {
sources.push({ id: doc._id, gridFsId: doc.versions?.original?.meta?.gridFsFileId, name: doc.name });
if (mfBucket) {
const mfCursor = db.collection(collName).find({ $or: [
{ 'versions.original.storage': 'gridfs' },
{ 'versions.original.meta.gridFsFileId': { $exists: true, $ne: null } },
] });
for await (const doc of mfCursor) {
sources.push({ id: doc._id, gridFsId: doc.versions?.original?.meta?.gridFsFileId, name: doc.name, bucket: mfBucket, filesCollName: `${collName}.files` });
}
await mfCursor.close();
}
await mfCursor.close();
for (const src of sources) {
if (!src.gridFsId) { skipped++; continue; }
if (!src.gridFsId) {
// #6473: report — a GridFS-flagged record with no locatable binary is
// exactly how attachments go silently missing.
console.error(`[gridfs-migrate] ${collName}/${src.id}: GridFS reference missing; cannot extract.`);
errors++;
continue;
}
const originalName = src.name || String(src.id);
const { fullPath, basename } = uniquePath(destDir, String(src.id), originalName);
@@ -155,10 +173,11 @@ async function migrateCollection(collName, destDir, bucketPrefix) {
// Already extracted
if (fs.existsSync(fullPath)) { skipped++; continue; }
// Disk space check
// Disk space check (#6473: look the size up in the SOURCE's own files
// collection — CFS and Meteor-Files binaries live in different buckets)
let fileSize = 0;
try {
const meta = await db.collection(filesCollName).findOne({ _id: new ObjectId(src.gridFsId) });
const meta = await db.collection(src.filesCollName).findOne({ _id: new ObjectId(src.gridFsId) });
fileSize = meta?.length || 0;
} catch {}
if (!hasEnoughSpace(destDir, fileSize)) {
@@ -168,7 +187,7 @@ async function migrateCollection(collName, destDir, bucketPrefix) {
}
try {
const bytes = await extractFile(bucket, src.gridFsId, fullPath);
const bytes = await extractFile(src.bucket, src.gridFsId, fullPath);
done++;
console.log(`[gridfs-migrate] ${basename} (${(bytes/1024).toFixed(1)} KB)`);
snap-src/bin/migrate-mongo3-to-ferretdb.mjs
+53
View File
@@ -114,6 +114,12 @@ const SRC_DB = process.env.SRC_DB || 'wekan';
const TARGET_URL = process.env.TARGET_MONGO_URL || 'mongodb://127.0.0.1:27098/wekan';
const FILES_DIR = process.env.FILES_DIR || path.join(process.env.WRITABLE_PATH || '/data', 'files');
const PORT = parseInt(process.env.MIGRATION_PORT || '8080', 10);
// #6473: FILES_ONLY=true = incremental attachment/avatar repair against an existing,
// LIVE target (set by the snap's attachment-repair on startup): the text collections
// are left completely untouched (users may have changed them since the original
// migration), already-migrated files are verified and skipped, and only missing
// binaries/records are migrated.
const FILES_ONLY = process.env.FILES_ONLY === 'true';
// Optional: write the final migration state here as JSON so an admin UI can read
// it later (e.g. WeKan on Sandstorm — Admin Panel / Attachments / Sandstorm).
const STATUS_FILE = process.env.STATUS_FILE || '';
@@ -357,6 +363,10 @@ ${currentHtml}
<h2>Activity</h2><pre>${state.log.slice(-15).map(esc).join('\n') || '(waiting…)'}</pre>
<h2>Errors</h2>${state.errors.slice(-15).reverse().map(e => `<div style="color:#c0392b">${esc(e)}</div>`).join('') || '<p class="muted">None</p>'}
</div></body></html>`);
}).on('error', (e) => {
// A background FILES_ONLY repair runs while WeKan itself owns the web port (#6473):
// the dashboard is a convenience — never crash the repair over EADDRINUSE.
console.log(`[migrate3] progress dashboard not available (${e.code || e.message}); continuing without it.`);
}).listen(PORT, () => console.log(`[migrate3] progress at http://localhost:${PORT}`));
// ── run a mongo CLI tool with the old libraries on LD_LIBRARY_PATH ────────────
@@ -695,6 +705,12 @@ async function run() {
]);
// ── text collections (everything except the GridFS chunk/file collections) ──
// #6473: FILES_ONLY repair NEVER touches text collections — users may have
// added/changed boards and cards on FerretDB since the original migration, and
// re-copying from the frozen MongoDB source would overwrite or resurrect data.
if (FILES_ONLY) {
logline('FILES_ONLY: skipping text collections; incremental attachment/avatar repair only.');
} else {
state.phase = 'migrating-collections';
for (const name of all) {
// FerretDB v1 rejects collection names containing a dot ("invalid key: '<name>' (key must
@@ -723,6 +739,7 @@ async function run() {
completedCollections.add(name);
saveCheckpoint(true); // a collection boundary is worth a guaranteed write
}
} // end !FILES_ONLY (text collections)
// ── GridFS attachments + avatars -> filesystem ──────────────────────────────
state.phase = 'migrating-files';
@@ -775,6 +792,24 @@ async function run() {
state.files[key].done++; state.files[key].bytes += Number(alreadyDone.size) || 0;
continue;
}
// #6473: ALREADY MIGRATED? Check the TARGET itself — record present, version on
// 'fs', file actually on disk. This makes the automatic startup repair
// INCREMENTAL: healthy installs verify-and-skip, only what is missing migrates.
if (fr._id) {
try {
const tgtRec = await db.collection(bucket).findOne({ _id: fr._id });
const tv = tgtRec && tgtRec.versions && tgtRec.versions.original;
if (tv && tv.storage === 'fs' && tv.path && fs.existsSync(tv.path)) {
fileIndex++;
state.files[key].done++; state.files[key].bytes += Number(tv.size) || 0;
let onDiskSize = Number(tv.size) || 0;
try { onDiskSize = fs.statSync(tv.path).size; } catch { /* keep record size */ }
completedFiles.set(resumeKey, { path: tv.path, size: onDiskSize });
saveCheckpoint();
continue;
}
} catch { /* fall through to extraction */ }
}
if (!ensureDiskForFile(destDir, size, name)) { onDiskAbort(); return; }
// Stream the file from its chunks (sorted by n) straight to disk, one chunk at a
// time (bounded RAM), publishing live per-file progress for the dashboard.
@@ -856,6 +891,24 @@ async function run() {
state.files[key].done++; state.files[key].bytes += Number(alreadyDone.size) || 0;
continue;
}
// #6473: ALREADY MIGRATED? Verify against the TARGET record — version on 'fs'
// and the file on disk means nothing to do (incremental startup repair). And in
// FILES_ONLY mode a record DELETED from the target since the migration must not
// have its binary resurrected.
try {
const tgtRec = await db.collection(bucket).findOne({ _id: recId });
const tv = tgtRec && tgtRec.versions && tgtRec.versions[versionName];
if (tv && tv.storage === 'fs' && tv.path && fs.existsSync(tv.path)) {
fileIndex++;
state.files[key].done++; state.files[key].bytes += Number(tv.size) || 0;
let onDiskSize = Number(tv.size) || 0;
try { onDiskSize = fs.statSync(tv.path).size; } catch { /* keep record size */ }
completedFiles.set(resumeKey, { path: tv.path, size: onDiskSize });
saveCheckpoint();
continue;
}
if (FILES_ONLY && !tgtRec) continue;
} catch { /* fall through to extraction */ }
if (!ensureDiskForFile(destDir, size, name)) { onDiskAbort(); return; }
fileIndex++;
state.current = { active: true, kind: key, name, size, done: 0, index: fileIndex, total: state.current.total };
snap-src/bin/migration-control
+4
View File
@@ -241,6 +241,10 @@ finish_success() {
log "FerretDB SQLite verified (${sq:-?})${wal:+; WAL log present}."
log "Migration succeeded. Switching to FerretDB (SQLite); MongoDB will be shut down."
touch "$MARKER"
# #6473: this migration ran with the CURRENT (fixed) file extraction, so the startup
# attachment-repair has nothing to do — mark it done up front. Keep the marker name in
# sync with attachment-repair and FILES_VERSION in migrate-mongodb-to-ferretdb.mjs.
touch "$SNAP_COMMON/.attachments-files-v2-done" 2>/dev/null || true
# #6458/#6466: clear mongodb-control's start-failure counter — the migration
# completing is exactly the recovery those failures were counting toward.
rm -f "$SNAP_COMMON/.mongod-start-failures" 2>/dev/null || true
snap-src/bin/wekan-control
+8
View File
@@ -137,6 +137,14 @@ if [ "ferretdb" = "${DATABASE}" ]; then
sleep 5
done
echo "FerretDB is ready."
# #6473: SELF-HEALING attachments. Older MongoDB -> FerretDB migrations silently
# lost attachments/avatars (CollectionFS copies.<bucket>.key was never resolved;
# gridFsFileId-only Meteor-Files records were skipped). The MongoDB source data
# was never modified, so everything is recoverable: run an automatic, INCREMENTAL
# files-only repair in the background — it verifies what is already migrated,
# migrates only what is missing, never touches text data, runs once (marker
# $SNAP_COMMON/.attachments-files-v2-done) and never blocks WeKan from starting.
bash "$SNAP/bin/attachment-repair" > "$SNAP_COMMON/attachment-repair.log" 2>&1 &
elif [ "true" != "${DISABLE_MONGODB}" ]; then
# Make sure the MongoDB service is enabled and running first; if it was left
# stopped/disabled, enable and start it now so WeKan does not wait here forever.
snap-src/bin/wekan-force-migrate
+3
View File
@@ -29,6 +29,9 @@ snapctl stop "${svc}.mongodb" 2>/dev/null || true
# The source MongoDB data (WiredTiger files in $COMMON) is NOT touched.
rm -f "$COMMON/.migration-to-ferretdb-done"
rm -f "$COMMON/migration-progress.json"
# #6473: a forced fresh migration re-does the file phase too, and finish_success
# re-writes this marker — clear it so the startup attachment-repair state matches.
rm -f "$COMMON/.attachments-files-v2-done"
rm -rf "${COMMON:?}/files/db/"* 2>/dev/null || true
# Put the snap back on mongodb so the normal path (mongodb-control -> migration-control) runs;
snapcraft-core26.yaml
+7
View File
@@ -127,6 +127,13 @@ apps:
command: ./bin/wekan-force-migrate
plugs: [network, network-bind]
# #6473: incremental attachment/avatar repair (runs automatically in the
# background on every start; this command is for running it by hand, e.g.
# after deleting $SNAP_COMMON/.attachments-files-v2-done).
repair-attachments:
command: ./bin/attachment-repair
plugs: [network, network-bind]
maintenance:
command: ./bin/wekan-maintenance
plugs: [network, network-bind]
snapcraft.yaml
+7
View File
@@ -130,6 +130,13 @@ apps:
command: ./bin/wekan-force-migrate
plugs: [network, network-bind]
# #6473: incremental attachment/avatar repair (runs automatically in the
# background on every start; this command is for running it by hand, e.g.
# after deleting $SNAP_COMMON/.attachments-files-v2-done).
repair-attachments:
command: ./bin/attachment-repair
plugs: [network, network-bind]
maintenance:
command: ./bin/wekan-maintenance
plugs: [network, network-bind]
state.json
+1
View File
@@ -0,0 +1 @@
{"uuid":"35f0e3fa-6c42-4d90-b45d-feb8ef847671","telemetry":false}
tests/attachmentRepair.test.cjs
+279
View File
@@ -0,0 +1,279 @@
'use strict';
// Plain-Node BEHAVIORAL tests (no Meteor, needs bash — CI runners are Linux) for
// issue #6473's SELF-HEALING attachment repair: snap-src/bin/attachment-repair.
//
// Rolled out via snap auto-refresh to ~15k servers, the repair must run WITHOUT
// any manual command: wekan-control launches it in the background on every start
// when the snap runs on FerretDB; it verifies what is already migrated, migrates
// only what is missing (FILES_ONLY importer mode — text data untouched), marks
// itself done with $SNAP_COMMON/.attachments-files-v2-done, and retries on the
// next start after a failure. These tests drive the REAL bash script against a
// fake $SNAP tree with stubbed snapctl/db-eval/cpu-exec/node, asserting both the
// happy path and every guard (negative cases).
// Run: node tests/attachmentRepair.test.cjs
const assert = require('assert');
const fs = require('fs');
const os = require('os');
const path = require('path');
const { spawnSync } = require('child_process');
const repoRoot = path.resolve(__dirname, '..');
const read = rel => fs.readFileSync(path.join(repoRoot, rel), 'utf8');
const SCRIPT = path.join(repoRoot, 'snap-src/bin/attachment-repair');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// Build a disposable environment: fake $SNAP with stub tools, fake $SNAP_COMMON.
// opts: {database, ferretdbData, mongodbData, marker, maintenance, nodeExit}
function makeEnv(opts = {}) {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'attachment-repair-'));
const snap = path.join(root, 'snap');
const common = path.join(root, 'common');
const stubBin = path.join(root, 'stubs');
const record = path.join(root, 'record');
fs.mkdirSync(path.join(snap, 'bin'), { recursive: true });
fs.mkdirSync(common, { recursive: true });
fs.mkdirSync(stubBin, { recursive: true });
// Real guard helper, stubbed tools.
fs.copyFileSync(path.join(repoRoot, 'snap-src/bin/ferretdb-has-data'), path.join(snap, 'bin/ferretdb-has-data'));
// snapctl stub: `snapctl get database` prints the configured value.
fs.writeFileSync(path.join(stubBin, 'snapctl'), `#!/bin/sh
if [ "$1" = "get" ] && [ "$2" = "database" ]; then echo "${opts.database || 'ferretdb'}"; fi
exit 0
`, { mode: 0o755 });
// db-eval stub: ping/shutdown always succeed (FerretDB and temp mongod "ready").
fs.writeFileSync(path.join(snap, 'bin/db-eval'), `#!/bin/sh
echo "db-eval $@" >> "${record}"
exit 0
`, { mode: 0o755 });
// cpu-exec stub: "starts" the temp mongod — writes the pidfile named on the
// command line (like mongod --fork --pidfilepath does) and exits 0.
fs.writeFileSync(path.join(snap, 'bin/cpu-exec'), `#!/bin/bash
echo "cpu-exec $@" >> "${record}"
prev=""
for a in "$@"; do
if [ "$prev" = "--pidfilepath" ]; then echo 999999 > "$a"; fi
prev="$a"
done
exit 0
`, { mode: 0o755 });
// mongod binary must exist (start_mongod runs it through cpu-exec).
fs.writeFileSync(path.join(snap, 'bin/mongod'), '#!/bin/sh\nexit 0\n', { mode: 0o755 });
// node stub: records its argv and full environment, exits as configured —
// this is "the importer run".
fs.writeFileSync(path.join(snap, 'bin/node'), `#!/bin/sh
echo "node $@" >> "${record}"
env | grep -E '^(FILES_ONLY|TARGET_MONGO_URL|SOURCE_MONGO_URL|MIGRATION_PORT|WRITABLE_PATH)=' | sort >> "${record}"
exit ${Number.isInteger(opts.nodeExit) ? opts.nodeExit : 0}
`, { mode: 0o755 });
// The importer file path is passed to node; content is irrelevant here.
fs.writeFileSync(path.join(snap, 'bin/migrate-mongodb-to-ferretdb.mjs'), '// stub\n');
fs.writeFileSync(path.join(snap, 'bin/migrate-mongo3-to-ferretdb.mjs'), '// stub\n');
if (opts.ferretdbData !== false) {
fs.mkdirSync(path.join(common, 'files/db'), { recursive: true });
fs.writeFileSync(path.join(common, 'files/db/wekan.sqlite'), 'data');
}
if (opts.mongodbData !== false) {
fs.writeFileSync(path.join(common, 'WiredTiger'), 'WiredTiger');
}
if (opts.marker) fs.writeFileSync(path.join(common, '.attachments-files-v2-done'), '');
if (opts.maintenance) fs.writeFileSync(path.join(common, '.wekan-maintenance'), '');
return { root, snap, common, stubBin, record };
}
function runRepair(env, extraEnv = {}) {
return spawnSync('bash', [SCRIPT], {
encoding: 'utf8',
env: {
PATH: `${env.stubBin}:${process.env.PATH}`,
SNAP: env.snap,
SNAP_COMMON: env.common,
SNAP_NAME: 'wekan',
REPAIR_WAIT_SECS: '4',
...extraEnv,
},
});
}
const recordOf = env => (fs.existsSync(env.record) ? fs.readFileSync(env.record, 'utf8') : '');
const markerExists = env => fs.existsSync(path.join(env.common, '.attachments-files-v2-done'));
// ── happy path ────────────────────────────────────────────────────────────────
test('repairs once and marks done: temp mongod started, importer run FILES_ONLY against live FerretDB', () => {
const env = makeEnv({});
const r = runRepair(env);
assert.strictEqual(r.status, 0, r.stdout + r.stderr);
const rec = recordOf(env);
assert.ok(/cpu-exec .*mongod/.test(rec), 'temporary source mongod started via cpu-exec');
assert.ok(/node .*migrate-mongodb-to-ferretdb\.mjs/.test(rec), 'modern importer invoked');
assert.ok(rec.includes('FILES_ONLY=true'), 'importer runs in incremental files-only mode');
assert.ok(/TARGET_MONGO_URL=mongodb:\/\/127\.0\.0\.1:27019\/wekan/.test(rec), 'target is the LIVE FerretDB');
assert.ok(rec.includes('MIGRATION_PORT=0'), 'no clash with WeKan on the web port');
assert.ok(rec.includes(`WRITABLE_PATH=${env.common}`), 'files land in $SNAP_COMMON/files');
assert.ok(markerExists(env), 'success writes the done marker');
assert.ok(!fs.existsSync(path.join(env.common, '.attachment-repair.pid')), 'lock removed');
});
test('second run after success does nothing (marker respected)', () => {
const env = makeEnv({ marker: true });
const r = runRepair(env);
assert.strictEqual(r.status, 0);
assert.strictEqual(recordOf(env), '', 'no tool was invoked at all');
});
// ── failure: retry next start ─────────────────────────────────────────────────
test('negative: importer failure leaves NO marker so the next start retries', () => {
const env = makeEnv({ nodeExit: 1 });
const r = runRepair(env);
assert.notStrictEqual(r.status, 0, 'non-zero exit on failure');
assert.ok(!markerExists(env), 'failed repair must not be marked done');
assert.ok(!fs.existsSync(path.join(env.common, '.attachment-repair.pid')), 'lock removed even on failure');
});
// ── guards (negative cases) ───────────────────────────────────────────────────
test('negative: skips while still on MongoDB (normal migration path covers files)', () => {
const env = makeEnv({ database: 'mongodb' });
const r = runRepair(env);
assert.strictEqual(r.status, 0);
assert.ok(!markerExists(env), 'must not mark done — migration has not happened yet');
assert.ok(!/node /.test(recordOf(env)), 'importer not invoked');
});
test('negative: skips in maintenance mode', () => {
const env = makeEnv({ maintenance: true });
const r = runRepair(env);
assert.strictEqual(r.status, 0);
assert.ok(!markerExists(env));
assert.strictEqual(recordOf(env), '');
});
test('negative: skips when FerretDB has no data (nothing to repair INTO)', () => {
const env = makeEnv({ ferretdbData: false });
const r = runRepair(env);
assert.strictEqual(r.status, 0);
assert.ok(!markerExists(env), 'not marked done — the migration itself is still pending');
assert.ok(!/node /.test(recordOf(env)));
});
test('no MongoDB source data: marks done WITHOUT running anything (fresh installs)', () => {
const env = makeEnv({ mongodbData: false });
const r = runRepair(env);
assert.strictEqual(r.status, 0);
assert.ok(markerExists(env), 'nothing to repair from, ever — never probe again');
assert.ok(!/node /.test(recordOf(env)), 'importer not invoked');
});
test('negative: a live concurrent repair is not doubled (pid lock)', () => {
const env = makeEnv({});
fs.writeFileSync(path.join(env.common, '.attachment-repair.pid'), String(process.pid));
const r = runRepair(env);
assert.strictEqual(r.status, 0);
assert.ok(!/node /.test(recordOf(env)), 'importer not invoked while another repair runs');
assert.ok(!markerExists(env));
});
test('a STALE lock (dead pid) does not block the repair', () => {
const env = makeEnv({});
fs.writeFileSync(path.join(env.common, '.attachment-repair.pid'), '999999');
const r = runRepair(env);
assert.strictEqual(r.status, 0, r.stdout + r.stderr);
assert.ok(markerExists(env), 'repair ran despite the stale lock');
});
// ── wiring: the repair is started automatically and coordinated ───────────────
const wekanControl = read('snap-src/bin/wekan-control');
const migrationControl = read('snap-src/bin/migration-control');
const forceMigrate = read('snap-src/bin/wekan-force-migrate');
const importerModern = read('releases/migrate-mongodb-to-ferretdb.mjs');
const importer3 = read('snap-src/bin/migrate-mongo3-to-ferretdb.mjs');
test('wekan-control starts attachment-repair in the BACKGROUND on the ferretdb path', () => {
assert.ok(/attachment-repair[^\n]*&\s*$/m.test(wekanControl),
'must not block WeKan startup');
const ferretBranch = wekanControl.slice(wekanControl.indexOf('Waiting for FerretDB'));
assert.ok(ferretBranch.includes('attachment-repair'), 'launched after FerretDB is ready');
});
test('a fresh successful migration pre-marks the repair as done (fixed importer already ran)', () => {
const fn = migrationControl.match(/finish_success\(\) \{[\s\S]*?\n\}/);
assert.ok(fn, 'finish_success found');
assert.ok(fn[0].includes('.attachments-files-v2-done'));
});
test('wekan.migrate (forced fresh migration) clears the repair marker', () => {
assert.ok(forceMigrate.includes('rm -f "$COMMON/.attachments-files-v2-done"'));
});
test('modern importer: FILES_VERSION stamp exists and the marker gates on it', () => {
assert.ok(/const FILES_VERSION = 2;/.test(importerModern));
assert.ok(importerModern.includes('(marker.filesVersion || 1) >= FILES_VERSION'),
'old-filesVersion markers must trigger the automatic repair');
assert.ok(importerModern.includes('filesVersion: FILES_VERSION') ||
importerModern.includes('filesVersion: FILES_VERSION'),
'fresh migrations stamp the current files version');
});
test('modern importer: repair mode never touches text data or the migration record', () => {
assert.ok(importerModern.includes('end !repairMode (text collections)'));
assert.ok(importerModern.includes('end !repairMode (swimlane/orphan fixes)'));
assert.ok(/repairMode[\s\S]*?updateOne\(\s*\{ _id: 'completed' \},\s*\{ \$set: \{ filesVersion: FILES_VERSION/.test(importerModern),
'repair stamps filesVersion via $set instead of replacing the marker');
});
test('modern importer: repair skips the all-files up-front disk check (files already on disk)', () => {
assert.ok(/!repairMode && state\.filesTotalBytes > 0/.test(importerModern));
});
test('importers verify against the TARGET what is already migrated (incremental repair)', () => {
// modern: CFS phase + Meteor-Files pass A both check the target record's version
const modernChecks = importerModern.match(/tv && tv\.storage === 'fs' && tv\.path && fs\.existsSync\(tv\.path\)/g) || [];
assert.ok(modernChecks.length >= 2, `modern importer target-state checks (found ${modernChecks.length})`);
const mongo3Checks = importer3.match(/tv && tv\.storage === 'fs' && tv\.path && fs\.existsSync\(tv\.path\)/g) || [];
assert.ok(mongo3Checks.length >= 2, `mongo3 importer target-state checks (found ${mongo3Checks.length})`);
});
test('importers never resurrect records deleted since the migration (repair mode)', () => {
assert.ok(/repairMode && !tgtDoc\) continue/.test(importerModern), 'pass A skips deleted records');
assert.ok(/repairMode && md\.fileId && !record\) continue/.test(importerModern), 'pass B skips deleted records');
assert.ok(/FILES_ONLY && !tgtRec\) continue/.test(importer3));
});
test('mongo3 importer: FILES_ONLY skips text collections', () => {
assert.ok(/const FILES_ONLY\s*=\s*process\.env\.FILES_ONLY === 'true';/.test(importer3));
assert.ok(importer3.includes('end !FILES_ONLY (text collections)'));
});
test('importer dashboards survive the web port being taken by WeKan (background repair)', () => {
assert.ok(/server\.on\('error'/.test(importerModern));
assert.ok(/\.on\('error',/.test(importer3));
});
test('repair command is registered in both snapcraft yamls', () => {
for (const rel of ['snapcraft.yaml', 'snapcraft-core26.yaml']) {
const y = read(rel);
assert.ok(y.includes('repair-attachments:'), `${rel} registers the app`);
assert.ok(y.includes('./bin/attachment-repair'), `${rel} points at the script`);
}
});
console.log(`attachmentRepair.test.cjs: ${passed} tests passed`);
tests/boardInvites.test.cjs
+200
View File
@@ -0,0 +1,200 @@
'use strict';
// Plain-Node unit test (no Meteor) for the board invitation accept / decline
// decision helpers. Run: node tests/boardInvites.test.cjs
//
// Regression guard for #4730 ("Bugs in adding and removing users"):
//
// Case 2 — "User A declines the invitation but still sees board 1 in the
// personal overview": the decline handler on the All Boards page calls
// `quitBoard` (deactivates the member entry) and then `acceptInvite` (to clear
// the invitation). The old server-side `acceptInvite` unconditionally set
// `members.$.isActive = true`, immediately re-activating the just-deactivated
// membership, so the declined board stayed in the user's overview.
//
// Case 1 fallout — "Adding and removing user A again doesn't fix it": the same
// unconditional re-activation let a member an admin had removed re-add
// themselves by calling `acceptInvite` directly.
const assert = require('assert');
const {
isInvitedToBoard,
planQuitBoard,
planAcceptInvite,
} = require('../models/lib/boardInvites');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// --- isInvitedToBoard --------------------------------------------------------
test('isInvitedToBoard: true when profile.invitedBoards contains the board', () => {
assert.strictEqual(
isInvitedToBoard({ invitedBoards: ['b1', 'b2'] }, 'b1'),
true,
);
});
test('isInvitedToBoard: false when not invited / no profile / bad shapes', () => {
assert.strictEqual(isInvitedToBoard({ invitedBoards: ['b2'] }, 'b1'), false);
assert.strictEqual(isInvitedToBoard({}, 'b1'), false);
assert.strictEqual(isInvitedToBoard(null, 'b1'), false);
assert.strictEqual(isInvitedToBoard(undefined, 'b1'), false);
assert.strictEqual(isInvitedToBoard({ invitedBoards: 'b1' }, 'b1'), false);
assert.strictEqual(isInvitedToBoard({ invitedBoards: ['b1'] }, ''), false);
});
// --- planQuitBoard: POSITIVE -------------------------------------------------
test('a plain member quitting deactivates the membership', () => {
const plan = planQuitBoard({ isMember: true, isInvited: false });
assert.deepStrictEqual(plan, {
ok: true,
removeMember: true,
pullInvite: false,
});
});
test('#4730: declining (member + pending invite) deactivates AND clears the invite', () => {
const plan = planQuitBoard({ isMember: true, isInvited: true });
assert.deepStrictEqual(plan, {
ok: true,
removeMember: true,
pullInvite: true,
});
});
test('#4730: a stale invitation without a member entry can still be quit', () => {
// e.g. an admin removed the user before they responded to the invite; the
// old code threw error-board-notAMember and the invited board was stuck in
// the personal overview forever.
const plan = planQuitBoard({ isMember: false, isInvited: true });
assert.deepStrictEqual(plan, {
ok: true,
removeMember: false,
pullInvite: true,
});
});
// --- planQuitBoard: NEGATIVE -------------------------------------------------
test('quitting a board one is neither member of nor invited to is an error', () => {
const plan = planQuitBoard({ isMember: false, isInvited: false });
assert.strictEqual(plan.ok, false);
assert.strictEqual(plan.error, 'error-board-notAMember');
assert.strictEqual(plan.removeMember, false);
assert.strictEqual(plan.pullInvite, false);
});
// --- planAcceptInvite: POSITIVE ----------------------------------------------
test('accepting a real invitation activates the membership and clears the invite', () => {
const plan = planAcceptInvite({ isInvited: true });
assert.deepStrictEqual(plan, {
ok: true,
pullInvite: true,
activateMember: true,
});
});
// --- planAcceptInvite: NEGATIVE (the #4730 regression) -------------------------
test('#4730 regression: acceptInvite without an invitation must NOT activate membership', () => {
// The old code set members.$.isActive = true unconditionally. This is what
// re-activated a declined membership (decline = quitBoard + acceptInvite)
// and let removed members re-add themselves.
const plan = planAcceptInvite({ isInvited: false });
assert.strictEqual(plan.ok, false);
assert.strictEqual(
plan.activateMember,
false,
'a user without a pending invitation must never be (re)activated',
);
assert.strictEqual(plan.pullInvite, false);
});
// --- End-to-end simulation of the decline flow --------------------------------
// Mirrors how server/models/boards.js applies the plans, driving the exact
// client sequence from client/components/boards/boardsList.js
// ('click .js-decline-invite'): quitBoard(boardId) then acceptInvite(boardId).
function makeWorld() {
const world = {
board: { members: [{ userId: 'userA', isActive: true }] },
profile: { invitedBoards: ['board1'] },
};
world.memberOf = (userId) =>
world.board.members.some((m) => m.userId === userId);
world.activeMember = (userId) =>
world.board.members.some((m) => m.userId === userId && m.isActive);
world.quitBoard = (userId, boardId) => {
const plan = planQuitBoard({
isMember: world.memberOf(userId),
isInvited: isInvitedToBoard(world.profile, boardId),
});
if (!plan.ok) throw new Error(plan.error);
if (plan.pullInvite) {
world.profile.invitedBoards = world.profile.invitedBoards.filter(
(id) => id !== boardId,
);
}
if (plan.removeMember) {
world.board.members.forEach((m) => {
if (m.userId === userId) m.isActive = false;
});
}
return true;
};
world.acceptInvite = (userId, boardId) => {
const plan = planAcceptInvite({
isInvited: isInvitedToBoard(world.profile, boardId),
});
if (!plan.ok) return false;
world.profile.invitedBoards = world.profile.invitedBoards.filter(
(id) => id !== boardId,
);
world.board.members.forEach((m) => {
if (m.userId === userId) m.isActive = true;
});
return true;
};
return world;
}
test('#4730 Case 2: decline flow (quitBoard then acceptInvite) leaves the member INACTIVE', () => {
const world = makeWorld();
// The All Boards decline handler sequence:
assert.strictEqual(world.quitBoard('userA', 'board1'), true);
world.acceptInvite('userA', 'board1'); // must now be a no-op
assert.strictEqual(
world.activeMember('userA'),
false,
'declined membership must stay deactivated (board leaves the personal overview)',
);
assert.deepStrictEqual(
world.profile.invitedBoards,
[],
'the declined invitation must be removed from profile.invitedBoards',
);
});
test('#4730 Case 1: a removed member cannot re-activate themselves via acceptInvite', () => {
const world = makeWorld();
world.profile.invitedBoards = []; // invite already consumed earlier (accepted)
world.board.members[0].isActive = false; // admin removed the user
assert.strictEqual(world.acceptInvite('userA', 'board1'), false);
assert.strictEqual(
world.activeMember('userA'),
false,
'a removed member without an invitation must stay removed',
);
});
test('accept flow still works: invited user becomes an active member', () => {
const world = makeWorld();
world.board.members[0].isActive = false; // e.g. re-invited after removal
world.profile.invitedBoards = ['board1'];
assert.strictEqual(world.acceptInvite('userA', 'board1'), true);
assert.strictEqual(world.activeMember('userA'), true);
assert.deepStrictEqual(world.profile.invitedBoards, []);
});
console.log(`\n${passed} tests passed`);
tests/boardTileData.test.cjs
+226
View File
@@ -0,0 +1,226 @@
'use strict';
// Plain-Node unit test (no Meteor) for the All Boards board-tile data helpers.
// Run: node tests/boardTileData.test.cjs
//
// Regression guard for #5174 (screenshots: All Boards tiles show neither the
// per-list card-counter line nor the board-member avatars even though the
// Admin Panel "Hide ... on All Boards" settings say to show them) and #4825
// ("Show card count per list doesn't deactivate" — inconsistent per-tile
// rendering, members never shown). The old reactive tile helpers caused the
// #4214 "icons random dance" and were stubbed to return [], hiding the data
// for everyone. The fix computes the tile data once, server-side, through the
// pure helpers tested here (models/lib/boardTileData.js), consumed via the
// getAllBoardsTileData method and a set-once ReactiveVar on the client.
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const {
showsCardCounterList,
showsBoardMemberList,
formatListCount,
countCardsByListId,
buildBoardTileData,
} = require('../models/lib/boardTileData');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// --- per-board opt-in flags: POSITIVE ----------------------------------------
test('board with "Show card count per list" enabled shows the counter line', () => {
assert.strictEqual(showsCardCounterList({ allowsCardCounterList: true }), true);
});
test('board with "Show Board members avatars" enabled shows the member row', () => {
assert.strictEqual(showsBoardMemberList({ allowsBoardMemberList: true }), true);
});
// --- per-board opt-in flags: NEGATIVE (#4825 "doesn't deactivate") -----------
test('#4825: board with the checkbox OFF must NOT show the counter line', () => {
assert.strictEqual(showsCardCounterList({ allowsCardCounterList: false }), false);
assert.strictEqual(showsBoardMemberList({ allowsBoardMemberList: false }), false);
});
test('#4825: board created before the feature (missing flag) is consistently OFF', () => {
assert.strictEqual(showsCardCounterList({}), false);
assert.strictEqual(showsBoardMemberList({}), false);
// loose truthy values (old inconsistency source) are not honored either
assert.strictEqual(showsCardCounterList({ allowsCardCounterList: 'true' }), false);
assert.strictEqual(showsBoardMemberList({ allowsBoardMemberList: 1 }), false);
assert.strictEqual(showsCardCounterList(null), false);
assert.strictEqual(showsBoardMemberList(undefined), false);
});
// --- formatListCount ----------------------------------------------------------
test('list entries are formatted "Title: N"', () => {
assert.strictEqual(formatListCount('To Do', 3), 'To Do: 3');
});
test('lists with no cards (or bogus counts) show ": 0"', () => {
assert.strictEqual(formatListCount('Empty', undefined), 'Empty: 0');
assert.strictEqual(formatListCount('Empty', -1), 'Empty: 0');
assert.strictEqual(formatListCount('', 2), ': 2');
});
// --- countCardsByListId --------------------------------------------------------
test('card docs fold into per-list counts', () => {
const counts = countCardsByListId([
{ listId: 'l1' },
{ listId: 'l1' },
{ listId: 'l2' },
{ listId: null },
null,
]);
assert.deepStrictEqual(counts, { l1: 2, l2: 1 });
});
test('no cards means no counts (not a crash)', () => {
assert.deepStrictEqual(countCardsByListId([]), {});
assert.deepStrictEqual(countCardsByListId(undefined), {});
});
// --- buildBoardTileData: POSITIVE ---------------------------------------------
test('#5174: enabled board tile carries its per-list counts in board order', () => {
const boards = [
{
_id: 'b1',
allowsCardCounterList: true,
allowsBoardMemberList: true,
members: [
{ userId: 'u1', isActive: true },
{ userId: 'u2' }, // no isActive field -> active
{ userId: 'u3', isActive: false }, // ex-member -> filtered out
{ isActive: true }, // no userId -> filtered out
],
},
];
const lists = [
// deliberately out of order: sorted by `sort` ascending for the tile
{ _id: 'l2', boardId: 'b1', title: 'In Arbeit', sort: 2 },
{ _id: 'l1', boardId: 'b1', title: 'To Do', sort: 1 },
{ _id: 'l3', boardId: 'b1', title: 'Erledigt', sort: 3 },
];
const tiles = buildBoardTileData(boards, lists, { l1: 1, l2: 1 });
assert.deepStrictEqual(tiles.b1, {
showLists: true,
showMembers: true,
lists: ['To Do: 1', 'In Arbeit: 1', 'Erledigt: 0'],
memberIds: ['u1', 'u2'],
});
});
test('every board on the page gets an entry (consistent rendering)', () => {
const tiles = buildBoardTileData(
[
{ _id: 'on', allowsCardCounterList: true, members: [] },
{ _id: 'off', allowsCardCounterList: false, members: [] },
{ _id: 'legacy', members: [] },
],
[{ _id: 'l1', boardId: 'on', title: 'Done', sort: 0 }],
{},
);
assert.deepStrictEqual(Object.keys(tiles).sort(), ['legacy', 'off', 'on']);
assert.deepStrictEqual(tiles.on.lists, ['Done: 0']);
});
// --- buildBoardTileData: NEGATIVE ----------------------------------------------
test('#4825: disabled board tile carries NO list counts even when lists/cards exist', () => {
const tiles = buildBoardTileData(
[{ _id: 'b1', allowsCardCounterList: false, allowsBoardMemberList: false,
members: [{ userId: 'u1', isActive: true }] }],
[{ _id: 'l1', boardId: 'b1', title: 'To Do', sort: 0 }],
{ l1: 5 },
);
assert.strictEqual(tiles.b1.showLists, false);
assert.deepStrictEqual(tiles.b1.lists, []);
assert.strictEqual(tiles.b1.showMembers, false);
assert.deepStrictEqual(tiles.b1.memberIds, []);
});
test('legacy board (missing flags) tile is empty, same as disabled', () => {
const tiles = buildBoardTileData(
[{ _id: 'b1', members: [{ userId: 'u1' }] }],
[{ _id: 'l1', boardId: 'b1', title: 'To Do', sort: 0 }],
{ l1: 2 },
);
assert.deepStrictEqual(tiles.b1, {
showLists: false,
showMembers: false,
lists: [],
memberIds: [],
});
});
test('lists of unknown/foreign boards do not leak into other tiles', () => {
const tiles = buildBoardTileData(
[{ _id: 'b1', allowsCardCounterList: true, members: [] }],
[
{ _id: 'l1', boardId: 'b1', title: 'Mine', sort: 0 },
{ _id: 'lx', boardId: 'other', title: 'Foreign', sort: 0 },
{ _id: 'ly', title: 'No board', sort: 0 },
],
{ l1: 1, lx: 9 },
);
assert.deepStrictEqual(tiles.b1.lists, ['Mine: 1']);
assert.strictEqual(tiles.other, undefined);
});
test('degenerate inputs do not crash', () => {
assert.deepStrictEqual(buildBoardTileData(undefined, undefined, undefined), {});
assert.deepStrictEqual(buildBoardTileData([null, {}], [null], null), {});
});
// --- wiring: the fix is actually hooked up ------------------------------------
const repo = path.join(__dirname, '..');
const clientJs = fs.readFileSync(
path.join(repo, 'client/components/boards/boardsList.js'), 'utf8');
const clientJade = fs.readFileSync(
path.join(repo, 'client/components/boards/boardsList.jade'), 'utf8');
const serverJs = fs.readFileSync(
path.join(repo, 'server/publications/boards.js'), 'utf8');
test('server registers the getAllBoardsTileData method using the pure helpers', () => {
assert.ok(/async getAllBoardsTileData\(\)/.test(serverJs),
'getAllBoardsTileData method missing');
assert.ok(serverJs.includes("from '/models/lib/boardTileData'"),
'server must build the tile data through models/lib/boardTileData');
assert.ok(serverJs.includes('buildBoardTileData(boards, lists, cardCounts)'),
'server must return buildBoardTileData output');
});
test('client fetches the tile data ONCE (no reactive cursor loops, #4214)', () => {
assert.ok(clientJs.includes("Meteor.call('getAllBoardsTileData'"),
'boardsList.js must fetch tile data via the method');
assert.ok(clientJs.includes('boardTileDataVar'),
'boardsList.js must store tile data in a ReactiveVar');
// The old reactive helpers must NOT come back inside the tile helpers:
const helpersTail = clientJs.slice(clientJs.indexOf('boardLists(boardId)'));
const helperBody = helpersTail.slice(0, helpersTail.indexOf('isStarred'));
assert.ok(!/ReactiveCache\.getLists|ReactiveCache\.getCards/.test(helperBody),
'tile helpers must not use reactive getLists/getCards cursors (#4214)');
// ...and the stubs that hid the data for everyone must be gone:
assert.ok(!/boardLists\(boardId\)\s*{\s*return \[\];/.test(clientJs),
'boardLists stub must be replaced with real tile data');
});
test('template gates the tile blocks on the resolved per-board flags', () => {
assert.ok(clientJade.includes('if showBoardMemberList _id'),
'member row must be gated by showBoardMemberList');
assert.ok(clientJade.includes('if showCardCounterList _id'),
'counter line must be gated by showCardCounterList');
// the raw board fields are not published to the All Boards page, so the
// template must not gate on them directly anymore
assert.ok(!/^\s*if allowsBoardMemberList\s*$/m.test(clientJade)
&& !/^\s*if allowsCardCounterList\s*$/m.test(clientJade),
'template must not test the unpublished allows* fields directly');
// the Admin Panel settings stay respected on top
assert.ok(clientJade.includes('unless currentSetting.hideBoardMemberList'));
assert.ok(clientJade.includes('unless currentSetting.hideCardCounterList'));
});
console.log(`\nboardTileData: all ${passed} tests passed`);
tests/cjkLabelWrap.test.cjs
+137
View File
@@ -0,0 +1,137 @@
'use strict';
// Plain-Node regression guard (no Meteor) for issue #4023: in Japanese and
// Chinese UI the add-card form footer wrapped labels mid-word ("追加" broke
// onto two lines inside the submit button, "リンク" rendered as リン / ク in
// the "or link / search / template" footer). CJK text has no spaces, so
// browsers may break between any two characters unless the footer opts into
// `word-break: keep-all` (and `white-space: nowrap` for the short labels).
// Run: node tests/cjkLabelWrap.test.cjs
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const repoRoot = path.resolve(__dirname, '..');
const read = rel => fs.readFileSync(path.join(repoRoot, rel), 'utf8');
const formsCss = read('client/components/forms/forms.css');
const listBodyJade = read('client/components/lists/listBody.jade');
// Strip comments so assertions match real declarations only.
const css = formsCss.replace(/\/\*[\s\S]*?\*\//g, '');
// Split a stylesheet into { selector, body } rules (flat rules only; rules
// nested in at-rules keep their own selector text, which is fine here).
function rules(stylesheet) {
const out = [];
const re = /([^{}]+)\{([^{}]*)\}/g;
let m;
while ((m = re.exec(stylesheet)) !== null) {
out.push({ selector: m[1].trim(), body: m[2] });
}
return out;
}
function findRule(stylesheet, selectorPart, declaration) {
return rules(stylesheet).some(
r => r.selector.includes(selectorPart) && r.body.includes(declaration),
);
}
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
test('add-card footer (.add-controls) forbids mid-word CJK breaks via word-break: keep-all', () => {
assert.ok(
findRule(css, '.add-controls', 'word-break: keep-all'),
'.add-controls must declare word-break: keep-all',
);
});
test('edit footer (.edit-controls) gets the same keep-all guard (same bug class)', () => {
assert.ok(
findRule(css, '.edit-controls', 'word-break: keep-all'),
'.edit-controls must declare word-break: keep-all',
);
});
test('the keep-all footer can still wrap BETWEEN links (flex-wrap: wrap)', () => {
const flexWrapRule = rules(css).find(
r =>
r.selector.includes('.add-controls') &&
r.body.includes('flex-wrap: wrap'),
);
assert.ok(flexWrapRule, '.add-controls must declare flex-wrap: wrap');
assert.ok(
findRule(css, '.add-controls', 'display: flex'),
'.add-controls is a flex container, so flex-wrap is effective',
);
});
test('submit button label ("追加") never splits onto two lines', () => {
assert.ok(
findRule(css, '.add-controls button[type=submit]', 'white-space: nowrap'),
'submit button in .add-controls must declare white-space: nowrap',
);
});
test('footer link groups ("or リンク", "/ 検索", "/ テンプレート") stay on one line each', () => {
assert.ok(
findRule(css, '.add-controls .quiet', 'white-space: nowrap'),
'.add-controls .quiet must declare white-space: nowrap',
);
});
test('the footer markup this pins still exists (link/search/template in a .quiet span)', () => {
assert.ok(/span\.quiet\s*\n\s*\|\s*\{\{_ 'or'\}\}\s*\n\s*a\.js-link/.test(listBodyJade),
'addCardForm footer: span.quiet wrapping the "or" + js-link anchor');
assert.ok(/a\.js-search \{\{_ 'search'\}\}/.test(listBodyJade));
assert.ok(/a\.js-card-template \{\{_ 'template'\}\}/.test(listBodyJade));
assert.ok(/button\.primary\.confirm\(type="submit"\) \{\{_ 'add'\}\}/.test(listBodyJade));
});
test('negative: no global keep-all/nowrap that would change Latin or long-URL wrapping', () => {
const globalish = rules(css).filter(r =>
/(?:^|,)\s*(?:\*|html|body)\s*(?:,|$)/.test(r.selector),
);
for (const r of globalish) {
assert.ok(
!r.body.includes('word-break'),
`global selector "${r.selector}" must not set word-break`,
);
assert.ok(
!r.body.includes('white-space: nowrap'),
`global selector "${r.selector}" must not set white-space: nowrap`,
);
}
});
test('negative: keep-all is scoped to the footers, not sprayed across the stylesheet', () => {
const keepAllRules = rules(css).filter(r => r.body.includes('keep-all'));
assert.ok(keepAllRules.length >= 1, 'at least one keep-all rule exists');
for (const r of keepAllRules) {
assert.ok(
r.selector.split(',').every(s => /\.(add|edit)-controls/.test(s)),
`keep-all rule "${r.selector}" must only target .add-controls / .edit-controls`,
);
}
});
test('negative: fix does not use word-break: break-all (which would break Latin words)', () => {
const addControlRules = rules(css).filter(r =>
r.selector.includes('.add-controls'),
);
for (const r of addControlRules) {
assert.ok(
!r.body.includes('break-all'),
`"${r.selector}" must not use word-break: break-all`,
);
}
});
console.log(`\n${passed} tests passed`);
tests/invitationCodeEmail.test.cjs
+202
View File
@@ -0,0 +1,202 @@
'use strict';
// Plain-Node unit test (no Meteor) for the invitation-code email rules.
// Run: node tests/invitationCodeEmail.test.cjs
//
// Regression guard for #4043 ("Sometimes user get board invitation by mail
// without invitation code"): recipients received an invitation email whose
// code the sign-up form rejected with "The invitation code doesn't exist".
// Sign-up only accepts { code: <string from the form>, email, valid: true }
// (server/models/users.js onCreateUser), so the sending side
// (server/models/settings.js sendInvitation / sendInvitationEmail) must:
// - store the invitee email lowercase (Mongo matching is case sensitive);
// - on RE-INVITE, overwrite a stale/invalid code with a fresh valid one
// instead of re-mailing a code that can never validate;
// - refuse (loudly) to send an email for a missing/invalid code;
// - on SMTP failure, only delete a code created by this very invite — never
// a pre-existing one whose code was already delivered in an earlier email.
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const {
normalizeInviteEmail,
hasUsableCode,
isInvitationCodeSendable,
buildReinviteModifier,
shouldRemoveInvitationOnEmailFailure,
} = require('../models/lib/invitationCodeEmail');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// Deterministic stand-in for getRandomNum(100000, 999999).
const generateCode = () => 654321;
// Mimic the sign-up lookup selector { code, email, valid: true } from
// server/models/users.js onCreateUser, with Mongo's exact-match semantics
// (strict type and case sensitivity).
function signupLookupMatches(doc, typedCode, typedEmail) {
return (
doc.code === typedCode && doc.email === typedEmail && doc.valid === true
);
}
// --- POSITIVE ----------------------------------------------------------------
test('normalizeInviteEmail lowercases and trims the address', () => {
assert.strictEqual(
normalizeInviteEmail(' User@Example.COM \n'),
'user@example.com',
);
assert.strictEqual(normalizeInviteEmail('plain@example.com'), 'plain@example.com');
});
test('a valid pending invitation is sendable', () => {
assert.strictEqual(
isInvitationCodeSendable({ code: '123456', valid: true }),
true,
);
});
test('a legacy invitation without a valid field is still sendable', () => {
// Documents predating the `valid` defaultValue must keep working.
assert.strictEqual(isInvitationCodeSendable({ code: '123456' }), true);
});
test('re-invite of a STILL-VALID invitation keeps its code (earlier email stays usable)', () => {
const invitation = { _id: 'i1', code: '111111', valid: true, email: 'a@b.c' };
const modifier = buildReinviteModifier(invitation, ['board1'], generateCode);
assert.deepStrictEqual(modifier, { $set: { boardsToBeInvited: ['board1'] } });
// No code/valid churn: the code delivered in the first email still validates.
assert.strictEqual('code' in modifier.$set, false);
assert.strictEqual('valid' in modifier.$set, false);
});
test('#4043: re-invite of an INVALIDATED invitation overwrites it with a fresh valid code', () => {
const invitation = { _id: 'i1', code: '111111', valid: false, email: 'a@b.c' };
const modifier = buildReinviteModifier(invitation, ['board1', 'board2'], generateCode);
assert.deepStrictEqual(modifier.$set.boardsToBeInvited, ['board1', 'board2']);
assert.strictEqual(modifier.$set.code, '654321'); // String, not Number
assert.strictEqual(typeof modifier.$set.code, 'string');
assert.strictEqual(modifier.$set.valid, true);
// The document after the update matches what sign-up will look for.
const updatedDoc = { ...invitation, ...modifier.$set };
assert.strictEqual(signupLookupMatches(updatedDoc, '654321', 'a@b.c'), true);
});
test('SMTP failure on a NEW invitation rolls the code back (nothing was ever delivered)', () => {
assert.strictEqual(
shouldRemoveInvitationOnEmailFailure({ isNewInvitation: true }),
true,
);
});
// --- NEGATIVE ----------------------------------------------------------------
test('normalizeInviteEmail returns "" for non-string input (caller skips it)', () => {
assert.strictEqual(normalizeInviteEmail(null), '');
assert.strictEqual(normalizeInviteEmail(undefined), '');
assert.strictEqual(normalizeInviteEmail(123456), '');
assert.strictEqual(normalizeInviteEmail({}), '');
});
test('#4043: an invalidated invitation is NOT sendable', () => {
assert.strictEqual(
isInvitationCodeSendable({ code: '123456', valid: false }),
false,
);
});
test('#4043: a missing/empty/non-string code is NOT sendable', () => {
assert.strictEqual(isInvitationCodeSendable(null), false);
assert.strictEqual(isInvitationCodeSendable(undefined), false);
assert.strictEqual(isInvitationCodeSendable({ valid: true }), false);
assert.strictEqual(isInvitationCodeSendable({ code: '', valid: true }), false);
// A Number code can never match the string typed into the sign-up form.
assert.strictEqual(isInvitationCodeSendable({ code: 123456, valid: true }), false);
assert.strictEqual(hasUsableCode({ code: 123456 }), false);
});
test('re-invite regenerates when the stored code is a Number or missing', () => {
for (const invitation of [
{ _id: 'i1', code: 111111, valid: true }, // legacy Number code
{ _id: 'i1', valid: true }, // no code at all
{ _id: 'i1', code: '', valid: true }, // empty code
null, // no document
]) {
const modifier = buildReinviteModifier(invitation, ['b'], generateCode);
assert.strictEqual(modifier.$set.code, '654321');
assert.strictEqual(modifier.$set.valid, true);
}
});
test('a Number code in the document never matches the sign-up (string) lookup', () => {
// This is exactly why buildReinviteModifier/hasUsableCode require a string.
const doc = { code: 123456, email: 'a@b.c', valid: true };
assert.strictEqual(signupLookupMatches(doc, '123456', 'a@b.c'), false);
});
test('a mixed-case stored email never matches the lowercased sign-up address', () => {
const doc = { code: '123456', email: 'User@Example.com', valid: true };
assert.strictEqual(signupLookupMatches(doc, '123456', 'user@example.com'), false);
// ...which is why sendInvitation must store normalizeInviteEmail(rawEmail).
const stored = { ...doc, email: normalizeInviteEmail('User@Example.com') };
assert.strictEqual(signupLookupMatches(stored, '123456', 'user@example.com'), true);
});
test('#4043: SMTP failure on a RE-INVITE must NOT delete the pre-existing code', () => {
// Deleting it would invalidate the code already delivered in an earlier email.
assert.strictEqual(
shouldRemoveInvitationOnEmailFailure({ isNewInvitation: false }),
false,
);
assert.strictEqual(shouldRemoveInvitationOnEmailFailure({}), false);
assert.strictEqual(shouldRemoveInvitationOnEmailFailure(), false);
});
// --- WIRING ------------------------------------------------------------------
// The helpers only protect #4043 if server/models/settings.js actually uses
// them; guard the wiring so a refactor cannot silently regress.
test('server/models/settings.js is wired to the #4043 helpers', () => {
const src = fs.readFileSync(
path.join(__dirname, '..', 'server', 'models', 'settings.js'),
'utf8',
);
assert.ok(
src.includes("require('/models/lib/invitationCodeEmail')"),
'settings.js must require models/lib/invitationCodeEmail',
);
assert.ok(
src.includes('normalizeInviteEmail(rawEmail)'),
'sendInvitation must normalize the invitee email',
);
assert.ok(
src.includes('buildReinviteModifier(invitation, boards'),
'the re-invite branch must build its modifier via buildReinviteModifier',
);
assert.ok(
src.includes('isInvitationCodeSendable(icode)'),
'sendInvitationEmail must guard with isInvitationCodeSendable',
);
assert.ok(
src.includes('shouldRemoveInvitationOnEmailFailure({ isNewInvitation })'),
'the SMTP-failure rollback must be gated on isNewInvitation',
);
assert.ok(
src.includes("sendInvitationEmail(invitation._id, { isNewInvitation: false })"),
're-invites must be flagged as pre-existing invitations',
);
assert.ok(
!/catch \(e\) \{\s*await InvitationCodes\.removeAsync\(_id\);/.test(src),
'the unconditional remove-on-email-failure must not come back',
);
});
console.log(`\n${passed} tests passed`);
tests/ldapEncryptionSetting.test.cjs
+187
View File
@@ -0,0 +1,187 @@
'use strict';
// Plain-Node unit test (no Meteor) for LDAP_ENCRYPTION value normalization.
// Run: node tests/ldapEncryptionSetting.test.cjs
//
// Regression guard for #4158 ("Documentation/configuration of
// LDAP_ENCRYPTION"). Historically only the magic strings 'ssl' (= LDAPS) and
// 'tls' (= STARTTLS) did anything; the DOCUMENTED value 'true' was silently
// ignored (settings_get() JSON-parses it into boolean true, which matched
// neither string), and any typo/unknown value silently meant "no encryption".
//
// normalizeLdapEncryption(value) must:
// - map 'true'/boolean true -> LDAPS ('tls' mode);
// - keep the legacy values working with their HISTORICAL meaning
// ('ssl' -> LDAPS, 'tls' -> STARTTLS) plus a one-line deprecation notice
// naming the preferred value;
// - map 'starttls' -> STARTTLS;
// - map 'false'/''/unset/boolean false -> no encryption, no warning;
// - map anything else -> no encryption WITH a clear warning listing the
// accepted values (never silently treat an unknown value as enabled).
const assert = require('assert');
const {
normalizeLdapEncryption,
} = require('../packages/wekan-ldap/server/encryptionSetting');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// --- POSITIVE: LDAPS (immediate TLS) -----------------------------------------
test("'true' means LDAPS, no warning", () => {
const r = normalizeLdapEncryption('true');
assert.strictEqual(r.mode, 'tls');
assert.strictEqual(r.warning, undefined);
});
test('boolean true (settings_get JSON-parses the env string) means LDAPS', () => {
// ldap.js settings_get() turns the env string 'true' into boolean true;
// this is exactly the value that used to be silently ignored (#4158).
const r = normalizeLdapEncryption(true);
assert.strictEqual(r.mode, 'tls');
assert.strictEqual(r.warning, undefined);
});
test("'TRUE' (any case) means LDAPS", () => {
assert.strictEqual(normalizeLdapEncryption('TRUE').mode, 'tls');
assert.strictEqual(normalizeLdapEncryption('True').mode, 'tls');
});
test("legacy 'ssl' still means LDAPS (backwards compatible)", () => {
const r = normalizeLdapEncryption('ssl');
assert.strictEqual(r.mode, 'tls');
});
test("legacy 'ssl' logs a deprecation notice naming 'true'", () => {
const r = normalizeLdapEncryption('ssl');
assert.ok(typeof r.warning === 'string' && r.warning.length > 0);
assert.ok(r.warning.includes("'true'"), 'must name the preferred value');
assert.ok(/deprecat/i.test(r.warning), 'must say it is deprecated');
});
test("legacy 'SSL' (any case) still means LDAPS", () => {
const r = normalizeLdapEncryption('SSL');
assert.strictEqual(r.mode, 'tls');
assert.ok(r.warning, 'still deprecated regardless of case');
});
// --- POSITIVE: STARTTLS ------------------------------------------------------
test("'starttls' means STARTTLS, no warning", () => {
const r = normalizeLdapEncryption('starttls');
assert.strictEqual(r.mode, 'starttls');
assert.strictEqual(r.warning, undefined);
});
test("'StartTLS' (any case) means STARTTLS", () => {
assert.strictEqual(normalizeLdapEncryption('StartTLS').mode, 'starttls');
assert.strictEqual(normalizeLdapEncryption('STARTTLS').mode, 'starttls');
});
test("legacy 'tls' still means STARTTLS (its historical behavior)", () => {
// Backwards compatibility: 'tls' has ALWAYS meant STARTTLS in WeKan.
// Remapping it to LDAPS would break every existing STARTTLS setup.
const r = normalizeLdapEncryption('tls');
assert.strictEqual(r.mode, 'starttls');
});
test("legacy 'tls' logs a deprecation notice naming 'starttls'", () => {
const r = normalizeLdapEncryption('tls');
assert.ok(typeof r.warning === 'string' && r.warning.length > 0);
assert.ok(r.warning.includes("'starttls'"), 'must name the preferred value');
assert.ok(/deprecat/i.test(r.warning), 'must say it is deprecated');
});
// --- POSITIVE: no encryption -------------------------------------------------
test("'false' means no encryption, no warning", () => {
const r = normalizeLdapEncryption('false');
assert.strictEqual(r.mode, 'off');
assert.strictEqual(r.warning, undefined);
});
test('boolean false (settings_get JSON-parses the env string) means no encryption', () => {
const r = normalizeLdapEncryption(false);
assert.strictEqual(r.mode, 'off');
assert.strictEqual(r.warning, undefined);
});
test('empty string means no encryption, no warning', () => {
const r = normalizeLdapEncryption('');
assert.strictEqual(r.mode, 'off');
assert.strictEqual(r.warning, undefined);
});
test('unset (undefined) means no encryption, no warning', () => {
const r = normalizeLdapEncryption(undefined);
assert.strictEqual(r.mode, 'off');
assert.strictEqual(r.warning, undefined);
});
test('null means no encryption, no warning', () => {
const r = normalizeLdapEncryption(null);
assert.strictEqual(r.mode, 'off');
assert.strictEqual(r.warning, undefined);
});
test('surrounding whitespace is tolerated', () => {
assert.strictEqual(normalizeLdapEncryption(' true ').mode, 'tls');
assert.strictEqual(normalizeLdapEncryption(' starttls ').mode, 'starttls');
assert.strictEqual(normalizeLdapEncryption(' false ').mode, 'off');
});
// --- NEGATIVE: unknown values must warn, never silently enable/disable -------
test('NEGATIVE: unknown value means no encryption plus a clear warning', () => {
const r = normalizeLdapEncryption('bogus');
assert.strictEqual(r.mode, 'off');
assert.ok(typeof r.warning === 'string' && r.warning.length > 0,
'unknown values must warn, never fail silently');
assert.ok(r.warning.includes("'bogus'"), 'warning must echo the bad value');
});
test('NEGATIVE: the warning for unknown values lists all accepted values', () => {
const w = normalizeLdapEncryption('yes').warning;
assert.ok(w.includes("'true'"), "must list 'true'");
assert.ok(w.includes("'starttls'"), "must list 'starttls'");
assert.ok(w.includes("'false'"), "must list 'false'");
assert.ok(w.includes("'ssl'"), "must mention legacy 'ssl'");
assert.ok(w.includes("'tls'"), "must mention legacy 'tls'");
});
test('NEGATIVE: unknown value is never treated as encryption enabled', () => {
for (const bad of ['yes', 'on', '1', 'ldaps', 'sttarttls', 'enable', 'TLSv1.2']) {
const r = normalizeLdapEncryption(bad);
assert.strictEqual(r.mode, 'off', `'${bad}' must not silently enable encryption`);
assert.ok(r.warning, `'${bad}' must produce a warning`);
}
});
test('NEGATIVE: numeric values (settings_get converts digits) warn and stay off', () => {
// settings_get() turns '1'/'0' into Numbers; neither is a valid setting.
for (const bad of [1, 0]) {
const r = normalizeLdapEncryption(bad);
assert.strictEqual(r.mode, 'off');
assert.ok(r.warning, `${bad} must produce a warning`);
}
});
test("NEGATIVE: 'starttls' must NOT map to LDAPS and 'true' must NOT map to STARTTLS", () => {
assert.notStrictEqual(normalizeLdapEncryption('starttls').mode, 'tls');
assert.notStrictEqual(normalizeLdapEncryption('true').mode, 'starttls');
});
test('NEGATIVE: valid values never carry the invalid-value warning', () => {
for (const good of ['true', 'starttls', 'false', '', undefined, true, false]) {
const r = normalizeLdapEncryption(good);
assert.strictEqual(r.warning, undefined,
`'${good}' is valid and must not warn`);
}
});
console.log(`\n${passed} tests passed`);
tests/ldapPasswordLoginGuard.test.cjs
+184
View File
@@ -0,0 +1,184 @@
'use strict';
// Plain-Node unit test (no Meteor) for the #4419 LDAP password-login guard.
// Run: node tests/ldapPasswordLoginGuard.test.cjs
//
// Security issue #4419: after a user is migrated from local password login to
// LDAP (authenticationMethod: 'ldap'), the old local password hash stays in
// services.password and keeps working. The guard rejects password-service
// login attempts for such users — conservatively: only while LDAP is enabled,
// never for other login services, never when LDAP_LOGIN_FALLBACK routes LDAP
// logins through the password service on purpose, and with an operator
// opt-out (LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN=true) so no deployment can be
// hard-locked by the upgrade.
const assert = require('assert');
const {
shouldRejectPasswordLogin,
LDAP_PASSWORD_LOGIN_DISABLED_REASON,
} = require('../server/lib/ldapPasswordLoginGuard');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
const ldapUser = { _id: 'u1', username: 'alice', authenticationMethod: 'ldap' };
const passwordUser = {
_id: 'u2',
username: 'bob',
authenticationMethod: 'password',
};
const ldapEnabled = { LDAP_ENABLE: 'true' };
// --- Rejections (the fix) ---------------------------------------------------
test('reject: ldap user + password service + LDAP enabled', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: ldapUser,
env: ldapEnabled,
}),
true,
);
});
test('reject: same rule for admins (no admin exemption)', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: { ...ldapUser, isAdmin: true },
env: ldapEnabled,
}),
true,
);
});
test('reject: boolean true LDAP_ENABLE also counts as enabled', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: ldapUser,
env: { LDAP_ENABLE: true },
}),
true,
);
});
// --- Allowed cases (must never lock anyone out) ------------------------------
test('allow: ldap user + ldap service (the intended login path)', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'ldap',
user: ldapUser,
env: ldapEnabled,
}),
false,
);
});
test('allow: other services are never rejected (resume/oidc/cas/saml)', () => {
for (const serviceName of ['resume', 'oidc', 'cas', 'saml']) {
assert.strictEqual(
shouldRejectPasswordLogin({ serviceName, user: ldapUser, env: ldapEnabled }),
false,
`service ${serviceName} must not be rejected`,
);
}
});
test('allow: password user + password service', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: passwordUser,
env: ldapEnabled,
}),
false,
);
});
test('allow: ldap user + password service when LDAP is disabled (de-LDAPed instance, admins must not be locked out)', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: ldapUser,
env: { LDAP_ENABLE: 'false' },
}),
false,
);
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: ldapUser,
env: {},
}),
false,
'unset LDAP_ENABLE means LDAP disabled',
);
});
test('allow: operator opt-out LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN=true keeps old behavior', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: ldapUser,
env: {
LDAP_ENABLE: 'true',
LDAP_MIGRATION_ALLOW_PASSWORD_LOGIN: 'true',
},
}),
false,
);
});
test('allow: LDAP_LOGIN_FALLBACK=true intentionally uses the password service (wekan-ldap loginHandler fallback)', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: ldapUser,
env: { LDAP_ENABLE: 'true', LDAP_LOGIN_FALLBACK: 'true' },
}),
false,
);
});
test('allow: user without authenticationMethod', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: { _id: 'u3', username: 'carol' },
env: ldapEnabled,
}),
false,
);
});
test('allow: missing user / missing env / missing args never throw and never reject', () => {
assert.strictEqual(
shouldRejectPasswordLogin({
serviceName: 'password',
user: undefined,
env: ldapEnabled,
}),
false,
);
assert.strictEqual(
shouldRejectPasswordLogin({ serviceName: 'password', user: ldapUser }),
false,
'missing env means LDAP not enabled',
);
assert.strictEqual(shouldRejectPasswordLogin(), false, 'no args at all');
});
test('reason: a clear user-facing message is exported for the hook', () => {
assert.strictEqual(typeof LDAP_PASSWORD_LOGIN_DISABLED_REASON, 'string');
assert.ok(/LDAP/.test(LDAP_PASSWORD_LOGIN_DISABLED_REASON));
assert.ok(/password/i.test(LDAP_PASSWORD_LOGIN_DISABLED_REASON));
});
console.log(`\nldapPasswordLoginGuard: all ${passed} tests passed`);
tests/ldapUserIdFilter.test.cjs
+110
View File
@@ -0,0 +1,110 @@
'use strict';
// Plain-Node unit test for issue #4654: LDAP background sync
// (LDAP_BACKGROUND_SYNC_KEEP_EXISTANT_USERS_UPDATED) only worked once for new
// users. getUserById() built its search filter from LDAP_UNIQUE_IDENTIFIER_FIELD
// alone: unset -> TypeError (undefined.split) aborting the whole sync run;
// empty -> invalid filter "(|(=value))" (seen in the issue logs), so existing
// users were never found again ("Can't sync user"). The stored id can also come
// from LDAP_USER_SEARCH_FIELD (getLdapUserUniqueID fallback), so the filter must
// consider both settings.
// Run: node tests/ldapUserIdFilter.test.cjs
const assert = require('assert');
const {
splitFieldSetting,
buildUserIdFilter,
} = require('../packages/wekan-ldap/server/userIdFilter');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// ── splitFieldSetting ─────────────────────────────────────────────────────────
test('split: comma list with whitespace', () => {
assert.deepStrictEqual(splitFieldSetting(' guid , sAMAccountName '), ['guid', 'sAMAccountName']);
});
test('split: unset/empty/non-string means no fields', () => {
assert.deepStrictEqual(splitFieldSetting(undefined), []);
assert.deepStrictEqual(splitFieldSetting(''), []);
assert.deepStrictEqual(splitFieldSetting(null), []);
assert.deepStrictEqual(splitFieldSetting(123), []);
});
test('split: drops empty entries so no "(=value)" clause can be built', () => {
assert.deepStrictEqual(splitFieldSetting('a,,b,'), ['a', 'b']);
assert.deepStrictEqual(splitFieldSetting(' , '), []);
});
// ── buildUserIdFilter: positive cases ─────────────────────────────────────────
test('stored idAttribute wins and is used directly', () => {
assert.strictEqual(
buildUserIdFilter('sAMAccountName', '\\6d\\2e', 'guid', 'uid'),
'(sAMAccountName=\\6d\\2e)',
);
});
test('single unique identifier field gives a plain equality filter', () => {
assert.strictEqual(
buildUserIdFilter(undefined, '\\01\\02', 'objectGUID', ''),
'(objectGUID=\\01\\02)',
);
});
test('multiple fields are OR-ed', () => {
assert.strictEqual(
buildUserIdFilter(undefined, 'v', 'guid,entryUUID', 'uid'),
'(|(guid=v)(entryUUID=v)(uid=v))',
);
});
test('#4654 config: unset LDAP_UNIQUE_IDENTIFIER_FIELD falls back to LDAP_USER_SEARCH_FIELD', () => {
// The issue reporter had only LDAP_USER_SEARCH_FIELD=sAMAccountName set;
// the old code crashed on settings_get(...).split(',') here.
assert.strictEqual(
buildUserIdFilter(undefined, '\\6d\\2e\\75\\73\\65\\72', undefined, 'sAMAccountName'),
'(sAMAccountName=\\6d\\2e\\75\\73\\65\\72)',
);
});
test('duplicate attribute across both settings is de-duplicated', () => {
// The workaround from the issue comments: LDAP_UNIQUE_IDENTIFIER_FIELD set
// to the same attribute as LDAP_USER_SEARCH_FIELD.
assert.strictEqual(
buildUserIdFilter(undefined, 'v', 'sAMAccountName', 'sAMAccountName'),
'(sAMAccountName=v)',
);
});
// ── buildUserIdFilter: negative cases ─────────────────────────────────────────
test('empty unique identifier field never yields the invalid "(|(=value))" filter', () => {
// Old behavior: ''.split(',') -> [''] -> "(|(=m.user))" as in the issue log.
const filter = buildUserIdFilter(undefined, 'm.user', '', 'sAMAccountName');
assert.strictEqual(filter, '(sAMAccountName=m.user)');
assert.ok(!filter.includes('(='), 'filter must not contain an empty attribute name');
});
test('no usable attribute anywhere returns null instead of throwing or "(|)"', () => {
assert.strictEqual(buildUserIdFilter(undefined, 'v', undefined, undefined), null);
assert.strictEqual(buildUserIdFilter(undefined, 'v', '', ''), null);
assert.strictEqual(buildUserIdFilter(null, 'v', ' , ', ' '), null);
});
test('old implementation would have failed on the issue configuration', () => {
// Documents the regression this test guards against: the pre-fix code path.
const oldGetUserByIdFilter = (uniqueIdentifierSetting, escapedValue) => {
const fields = uniqueIdentifierSetting.split(','); // throws when unset
return `(|${fields.map((item) => `(${item}=${escapedValue})`).join('')})`;
};
assert.throws(() => oldGetUserByIdFilter(undefined, 'm.user'), TypeError);
assert.strictEqual(oldGetUserByIdFilter('', 'm.user'), '(|(=m.user))'); // invalid filter
});
console.log(`\n${passed} tests passed`);
tests/listWidthDefaults.test.cjs
+209
View File
@@ -0,0 +1,209 @@
'use strict';
// Plain-Node regression test (no Meteor) for issue #5659:
// "Bug: List Width settings do not affect Public Board".
//
// Root cause: the default list width was duplicated across the resolution
// paths and DISAGREED — models/users.js fell back to 270 (getListWidth and the
// anonymous/public-board getListWidthFromStorage path) while the client and
// the lists schema used 272 — so lists whose width resolved through different
// paths could render at different "defaults" on the same (public) board.
//
// The fix is models/lib/listWidth.js: ONE Meteor-free module holding the
// default/minimum width and the resolution order, imported by
// client/components/lists/list.js, client/components/lists/listHeader.js and
// models/users.js. This test pins:
// - with NO customization, every list resolves to the SAME default width,
// for members and for logged-out visitors of public boards alike;
// - a valid customized width (shared, personal, or fixed mode) still wins;
// - invalid / below-minimum values can NOT make one list differ;
// - the consumers really import the shared module (no local re-definitions).
//
// Run: ELECTRON_RUN_AS_NODE=1 <node> tests/listWidthDefaults.test.cjs
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const {
DEFAULT_LIST_WIDTH,
MIN_LIST_WIDTH,
isValidListWidth,
normalizeListWidth,
resolveListWidth,
} = require('../models/lib/listWidth');
const repoRoot = path.resolve(__dirname, '..');
const read = rel => fs.readFileSync(path.join(repoRoot, rel), 'utf8');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// --- The one default ---------------------------------------------------------
test('there is a single sane default width (>= minimum)', () => {
assert.strictEqual(typeof DEFAULT_LIST_WIDTH, 'number');
assert.strictEqual(typeof MIN_LIST_WIDTH, 'number');
assert.strictEqual(DEFAULT_LIST_WIDTH, 272);
assert.strictEqual(MIN_LIST_WIDTH, 270);
assert.ok(DEFAULT_LIST_WIDTH >= MIN_LIST_WIDTH);
});
test('with no customization, EVERY list gets the same width (public-board default)', () => {
// A logged-out visitor of a public board: no fixed mode, shared mode
// (allowsPersonalListWidth=false), no stored widths anywhere. Simulate a
// whole board of lists in every "empty" shape a list doc can have.
const uncustomizedLists = [
{}, // no width field at all (pre-6409 list)
{ width: undefined },
{ width: null },
];
const widths = uncustomizedLists.map(l =>
resolveListWidth({ sharedWidth: l.width, personalMode: false }),
);
widths.forEach(w => assert.strictEqual(w, DEFAULT_LIST_WIDTH));
// ... and the same holds in personal mode with nothing stored (anon
// localStorage empty / logged-in profile empty).
const personal = uncustomizedLists.map(l =>
resolveListWidth({
sharedWidth: l.width,
personalMode: true,
personalWidth: null,
}),
);
personal.forEach(w => assert.strictEqual(w, DEFAULT_LIST_WIDTH));
// ... and with no options at all.
assert.strictEqual(resolveListWidth(), DEFAULT_LIST_WIDTH);
assert.strictEqual(resolveListWidth({}), DEFAULT_LIST_WIDTH);
});
// --- Customized widths still win ---------------------------------------------
test('a customized shared width (lists.width) still wins for every viewer', () => {
assert.strictEqual(resolveListWidth({ sharedWidth: 400 }), 400);
// Shared widths apply in shared mode regardless of viewer login state — the
// resolver has no user input in shared mode at all.
assert.strictEqual(
resolveListWidth({ sharedWidth: 400, personalMode: false, personalWidth: 999 }),
400,
);
});
test('a personal width wins over the shared width in personal mode only', () => {
assert.strictEqual(
resolveListWidth({ sharedWidth: 400, personalMode: true, personalWidth: 350 }),
350,
);
// negative: personal width must NOT leak into shared mode
assert.strictEqual(
resolveListWidth({ sharedWidth: 400, personalMode: false, personalWidth: 350 }),
400,
);
// personal mode with no personal width falls back to the shared width
assert.strictEqual(
resolveListWidth({ sharedWidth: 400, personalMode: true, personalWidth: null }),
400,
);
});
test('fixed ("same width for all lists") mode overrides everything (#5729)', () => {
assert.strictEqual(
resolveListWidth({
fixedEnabled: true,
fixedWidth: 500,
sharedWidth: 400,
personalMode: true,
personalWidth: 350,
}),
500,
);
// negative: fixed width is ignored while the mode is off
assert.strictEqual(
resolveListWidth({ fixedEnabled: false, fixedWidth: 500, sharedWidth: 400 }),
400,
);
// fixed mode without a stored value uses the same single default
assert.strictEqual(
resolveListWidth({ fixedEnabled: true, fixedWidth: null, sharedWidth: 400 }),
DEFAULT_LIST_WIDTH,
);
});
// --- Invalid values can never make one list differ ---------------------------
test('below-minimum / bogus stored widths fall back to the default (negative)', () => {
// the lists schema historically allowed 100-1000, but the render minimum is
// 270; anything below must not make one list narrower than the rest.
[269, 100, 0, -5, NaN, Infinity, '400', {}, [], true].forEach(bad => {
assert.strictEqual(isValidListWidth(bad), false, `isValidListWidth(${String(bad)})`);
assert.strictEqual(normalizeListWidth(bad), DEFAULT_LIST_WIDTH);
assert.strictEqual(resolveListWidth({ sharedWidth: bad }), DEFAULT_LIST_WIDTH);
assert.strictEqual(
resolveListWidth({ fixedEnabled: true, fixedWidth: bad }),
DEFAULT_LIST_WIDTH,
);
});
// an invalid personal width falls back to the (valid) shared width
assert.strictEqual(
resolveListWidth({ sharedWidth: 400, personalMode: true, personalWidth: 100 }),
400,
);
});
test('boundary widths behave (negative + positive)', () => {
assert.strictEqual(isValidListWidth(MIN_LIST_WIDTH), true);
assert.strictEqual(isValidListWidth(MIN_LIST_WIDTH - 1), false);
assert.strictEqual(resolveListWidth({ sharedWidth: MIN_LIST_WIDTH }), MIN_LIST_WIDTH);
assert.strictEqual(
resolveListWidth({ sharedWidth: MIN_LIST_WIDTH - 1 }),
DEFAULT_LIST_WIDTH,
);
});
test('normalizeListWidth honors an explicit fallback', () => {
assert.strictEqual(normalizeListWidth(300, 400), 300);
assert.strictEqual(normalizeListWidth(null, 400), 400);
});
// --- The consumers really use the shared module ------------------------------
test('client list.js imports the shared module and defines no local default', () => {
const src = read('client/components/lists/list.js');
assert.ok(src.includes("from '/models/lib/listWidth'"), 'imports /models/lib/listWidth');
assert.ok(src.includes('resolveListWidth('), 'resolves widths via resolveListWidth');
// negative: the old duplicated constants must not come back
assert.ok(!/const\s+DEFAULT_LIST_WIDTH\s*=/.test(src), 'no local DEFAULT_LIST_WIDTH');
assert.ok(!/const\s+MIN_LIST_WIDTH\s*=/.test(src), 'no local MIN_LIST_WIDTH');
});
test('client listHeader.js imports the shared module and hardcodes no default', () => {
const src = read('client/components/lists/listHeader.js');
assert.ok(src.includes("from '/models/lib/listWidth'"), 'imports /models/lib/listWidth');
// negative: the old "?: 272" / ">= 270" literals must not come back
assert.ok(!/\breturn 272\b/.test(src), 'no hardcoded 272 default');
assert.ok(!/>=\s*270\b/.test(src), 'no hardcoded 270 minimum');
});
test('models/users.js no longer falls back to a DIFFERENT default (270)', () => {
const src = read('models/users.js');
assert.ok(src.includes("from '/models/lib/listWidth'"), 'imports /models/lib/listWidth');
// negative: these exact lines were the #5659 root cause
assert.ok(!/return 270;?\s*\/\/\s*TODO/.test(src), 'old getListWidth 270 default gone');
assert.ok(!/return 270;?\s*\/\/\s*Return default width/.test(src), 'old storage 270 default gone');
assert.ok(!/\breturn 270\b/.test(src), 'no width helper returns 270 anymore');
assert.ok(/getListWidth\(boardId, listId\) \{[\s\S]*?DEFAULT_LIST_WIDTH/.test(src),
'getListWidth falls back to the shared DEFAULT_LIST_WIDTH');
});
test('the lists schema default matches the shared default', () => {
const src = read('models/lists.js');
const m = src.match(/width:\s*\{[\s\S]*?defaultValue:\s*(\d+)/);
assert.ok(m, 'lists.width schema has a defaultValue');
assert.strictEqual(Number(m[1]), DEFAULT_LIST_WIDTH);
});
console.log(`\n${passed} tests passed`);
tests/migrationAttachmentExtraction.test.cjs
+29 -2
View File
@@ -124,10 +124,10 @@ test('modern importer drives CollectionFS extraction from cfs_gridfs.<bucket>.fi
const fn = modern.match(/async function migrateGridFs\(bucketName, destDir, fileStateKey\) \{[\s\S]*?\n \}/);
assert.ok(fn, 'migrateGridFs found');
// The loop must iterate the files collection, not the filerecords.
assert.ok(fn[0].includes('const gfCursor = filesCol.find({})'),
assert.ok(fn[0].includes('const gfCursor = haveBucket ? filesCol.find({}) : null'),
'extraction is driven by the GridFS files collection');
// The filerecord join must be optional (the collection can be missing).
assert.ok(fn[0].includes('if (allColls.includes(`cfs.${bucketName}.filerecord`))'),
assert.ok(fn[0].includes('if (haveFilerecords) {'),
'a missing cfs.<bucket>.filerecord collection must not skip extraction');
assert.ok(fn[0].includes('resolveCfsGridFsId(record, bucketName)'));
// Orphan binaries (no filerecord) are still saved to disk, not dropped.
@@ -171,6 +171,24 @@ test('modern importer reports GridFS-flagged versions whose binary was never fou
assert.ok(/unresolved\.size > 0/.test(modern), 'leftovers are reported, not silently dropped');
});
test('modern importer covers the CFS FileSystem-store era (ATTACHMENTS_STORE_PATH, v3.12-v6.09)', () => {
const fn = modern.match(/async function migrateGridFs\(bucketName, destDir, fileStateKey\) \{[\s\S]*?\n \}/);
assert.ok(fn[0].includes('haveFilerecords'),
'a database with filerecords but NO cfs_gridfs collections must still be processed');
assert.ok(!/if \(!allColls\.includes\(filesCollName\)\) return;/.test(fn[0]),
'the CFS-bucket-only gate silently dropped every FileSystem-store attachment');
assert.ok(fn[0].includes('HEX24'), 'non-ObjectId keys are FILENAMES, not GridFS ids');
assert.ok(fn[0].includes('fsStoreRecords'), 'FileSystem-store filerecords are handled');
assert.ok(fn[0].includes('ATTACHMENTS_STORE_PATH'),
'the old store dir is searched via its historical env var');
assert.ok(fn[0].includes('fsStoreMissing'), 'unfound FS-store binaries are REPORTED');
});
test('modern importer reports CFS filerecords whose GridFS binary vanished', () => {
assert.ok(modern.includes('matchedGridIds'), 'hex filerecords are matched against actual GridFS files');
assert.ok(/missingBinaries > 0/.test(modern), 'unmatched ones are reported, never silently dropped');
});
// --- mongo3 importer: created records must be attached to their cards ----------
const mongo3 = read(IMPORTERS.mongo3);
@@ -193,6 +211,15 @@ test('migrate-gridfs-to-fs matches meta.gridFsFileId without a storage flag', ()
assert.ok(/\$or: \[[\s\S]*?'versions\.original\.storage': 'gridfs'[\s\S]*?'versions\.original\.meta\.gridFsFileId'/.test(g2f));
});
test('migrate-gridfs-to-fs reads each source from its OWN GridFS bucket', () => {
assert.ok(g2f.includes('extractFile(src.bucket'),
'Meteor-Files binaries live in <coll>, CFS binaries in cfs_gridfs.<coll> — one bucket for all extracted nothing');
assert.ok(g2f.includes('haveMfBucket'),
'a pure Meteor-Files database (no cfs_gridfs collections) must not be a silent no-op');
assert.ok(!/\{ skipped\+\+; continue; \}\s*\n?\s*$/m.test(g2f.split('for (const src of sources)')[1].split('\n').slice(0, 8).join('\n')),
'a GridFS-flagged record with no reference is reported as an error, not silently skipped');
});
// --- snap wekan-database: point lost users at the actual re-migration ----------
const wekanDatabase = read('snap-src/bin/wekan-database');
tests/oauth2LoginStyle.test.cjs
+212
View File
@@ -0,0 +1,212 @@
'use strict';
// Plain-Node unit test (no Meteor) for the OAuth2/OIDC login style handling.
// Run: node tests/oauth2LoginStyle.test.cjs
//
// Regression guard for #5695 ("OAUTH2_LOGIN_STYLE=redirect: make fixes so that
// redirect would work"): the useraccounts social-login button always calls
// Meteor.loginWithOidc({ loginStyle: 'popup' }) (its client-side
// socialLoginStyle default), and OAuth._loginStyle() lets that caller option
// override the loginStyle stored in the service configuration. As a result the
// admin's OAUTH2_LOGIN_STYLE=redirect was never honored and a popup opened
// (the reporter even saw "...&loginStyle=popup&..." serialized into the
// authorization URL). Oidc.requestCredential must now:
// - use the full-page redirect flow when the service configuration says
// loginStyle 'redirect' (i.e. OAUTH2_LOGIN_STYLE=redirect), even though
// the login button passed { loginStyle: 'popup' };
// - keep popup as the default otherwise;
// - not leak the Meteor-internal loginStyle option into the provider URL.
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const vm = require('vm');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
// --- Load packages/wekan-oidc/oidc_client.js in a sandbox -------------------
// The file is a classic Meteor package file (globals, no imports): it assigns
// the `Oidc` global and only touches Meteor/OAuth/Random when
// requestCredential() runs, so it can be evaluated with small stubs.
const clientSrc = fs.readFileSync(
path.join(__dirname, '..', 'packages', 'wekan-oidc', 'oidc_client.js'),
'utf8',
);
// Faithful copy of Meteor's OAuth._loginStyle precedence (packages/oauth):
// options.loginStyle || config.loginStyle || 'popup', with a popup fallback
// when sessionStorage is unavailable (Safari private mode).
let sessionStorageAvailable = true;
let launched = null;
let serviceConfig = null;
const OAuth = {
_loginStyle(service, config, options) {
const loginStyle =
(options && options.loginStyle) || config.loginStyle || 'popup';
if (!['popup', 'redirect'].includes(loginStyle)) {
throw new Error(`Invalid login style: ${loginStyle}`);
}
if (loginStyle === 'redirect' && !sessionStorageAvailable) {
return 'popup';
}
return loginStyle;
},
_redirectUri: () => 'http://wekan.example/_oauth/oidc',
_stateParam: (loginStyle, credentialToken) =>
`st-${loginStyle}-${credentialToken}`,
launchLogin(opts) {
launched = opts;
},
};
const sandbox = {
Meteor: {
call(method, service, cb) {
assert.strictEqual(method, 'getServiceConfiguration');
assert.strictEqual(service, 'oidc');
cb(null, serviceConfig);
},
},
OAuth,
Random: { secret: () => 'tok123' },
ServiceConfiguration: {
ConfigError: class ConfigError extends Error {},
},
console,
};
vm.createContext(sandbox);
vm.runInContext(clientSrc, sandbox, { filename: 'oidc_client.js' });
assert.ok(sandbox.Oidc, 'oidc_client.js must define the Oidc global');
// Simulate one login attempt: `config` is what the server stored from
// OAUTH2_LOGIN_STYLE (server/authentication.js), `options` is what the caller
// (the useraccounts button or the auto-redirect) passed to loginWithOidc.
function requestWith(config, options) {
launched = null;
serviceConfig = Object.assign(
{
clientId: 'wekan',
serverUrl: 'https://auth.example',
authorizationEndpoint: '/authorize',
},
config,
);
sandbox.Oidc.requestCredential(options, () => {});
assert.ok(launched, 'OAuth.launchLogin must have been called');
return launched;
}
// --- POSITIVE: OAUTH2_LOGIN_STYLE=redirect is honored ------------------------
test('#5695: config redirect wins over the login button popup default', () => {
// The useraccounts social button always passes { loginStyle: 'popup' }.
const l = requestWith({ loginStyle: 'redirect' }, { loginStyle: 'popup' });
assert.strictEqual(l.loginStyle, 'redirect');
});
test('#5695: config redirect used when the caller passes no options', () => {
const l = requestWith({ loginStyle: 'redirect' }, undefined);
assert.strictEqual(l.loginStyle, 'redirect');
});
test('explicit redirect option (OIDC_REDIRECTION_ENABLED auto-login) works even with popup config', () => {
const l = requestWith({ loginStyle: 'popup' }, { loginStyle: 'redirect' });
assert.strictEqual(l.loginStyle, 'redirect');
});
test('redirect state param is built for the redirect flow', () => {
const l = requestWith({ loginStyle: 'redirect' }, { loginStyle: 'popup' });
assert.strictEqual(l.credentialToken, 'tok123');
assert.ok(l.loginUrl.includes(encodeURIComponent('st-redirect-tok123')));
});
// --- NEGATIVE: popup remains the default -------------------------------------
test('NEGATIVE: popup by default when the config has no loginStyle', () => {
const l = requestWith({}, { loginStyle: 'popup' });
assert.strictEqual(l.loginStyle, 'popup');
const l2 = requestWith({}, undefined);
assert.strictEqual(l2.loginStyle, 'popup');
});
test('NEGATIVE: OAUTH2_LOGIN_STYLE=popup keeps the popup flow', () => {
const l = requestWith({ loginStyle: 'popup' }, { loginStyle: 'popup' });
assert.strictEqual(l.loginStyle, 'popup');
});
test('NEGATIVE: redirect falls back to popup when sessionStorage is unavailable', () => {
// Meteor's redirect flow needs sessionStorage (Safari private mode has
// none); the config override must still go through that fallback.
sessionStorageAvailable = false;
const l = requestWith({ loginStyle: 'redirect' }, { loginStyle: 'popup' });
assert.strictEqual(l.loginStyle, 'popup');
sessionStorageAvailable = true;
});
test('NEGATIVE: loginStyle is not leaked into the authorization URL', () => {
// The reporter saw "...%26loginStyle%3Dpopup%26..." in the provider URL.
const l = requestWith({ loginStyle: 'redirect' }, { loginStyle: 'popup' });
assert.ok(!l.loginUrl.includes('loginStyle'), l.loginUrl);
const l2 = requestWith({ loginStyle: 'popup' }, { loginStyle: 'popup' });
assert.ok(!l2.loginUrl.includes('loginStyle'), l2.loginUrl);
});
// --- Server side: default stays popup, redirect only when asked --------------
const authSrc = fs.readFileSync(
path.join(__dirname, '..', 'server', 'authentication.js'),
'utf8',
);
test('server config maps OAUTH2_LOGIN_STYLE to redirect only when set to redirect', () => {
assert.ok(
authSrc.includes("process.env.OAUTH2_LOGIN_STYLE === 'redirect'"),
'authentication.js must check for the redirect value explicitly',
);
});
test("NEGATIVE: server no longer defaults the stored loginStyle to 'redirect'", () => {
// The old `process.env.OAUTH2_LOGIN_STYLE || 'redirect'` default would flip
// every deployment without the env var to redirect now that the client
// honors the stored style; the fallback must be 'popup'.
assert.ok(!authSrc.includes("OAUTH2_LOGIN_STYLE || 'redirect'"), authSrc.match(/OAUTH2_LOGIN_STYLE[^\n]*/g).join('\n'));
assert.ok(/loginStyle:\s*\n?\s*process\.env\.OAUTH2_LOGIN_STYLE === 'redirect'\s*\n?\s*\?\s*'redirect'\s*\n?\s*:\s*'popup'/.test(authSrc));
});
// --- Secondary ask: OIDC_REDIRECTION_ENABLED auto-login actually runs --------
const settingsSrc = fs.readFileSync(
path.join(__dirname, '..', 'server', 'models', 'settings.js'),
'utf8',
);
test('isOidcRedirectionEnabled awaits the async service configuration', () => {
// Object.keys(<Promise>) is always [] since the findOneAsync migration, so
// the unawaited version returned false even when properly configured.
const m = settingsSrc.match(/async isOidcRedirectionEnabled\(\)[\s\S]*?\n \},/);
assert.ok(m, 'isOidcRedirectionEnabled must be async');
assert.ok(m[0].includes("await loadOidcConfig('oidc')"));
assert.ok(!m[0].includes('Object.keys(loadOidcConfig'));
});
const layoutsSrc = fs.readFileSync(
path.join(__dirname, '..', 'client', 'components', 'main', 'layouts.js'),
'utf8',
);
test('auto-redirect login declares its options (no strict-mode ReferenceError)', () => {
// layouts.js is an ES module (strict mode); the undeclared `options = {...}`
// assignment threw a ReferenceError and killed the auto-redirect.
const m = layoutsSrc.match(/isOidcRedirectionEnabled[\s\S]*?\}\);/);
assert.ok(m, 'layouts.js must keep the isOidcRedirectionEnabled auto-login');
assert.ok(m[0].includes('const options = {'));
// No bare (undeclared) `options = {` statement may remain in the block.
assert.ok(!/^\s*options = \{/m.test(m[0]));
assert.ok(m[0].includes("loginStyle: AccountsTemplates.options.socialLoginStyle"));
assert.ok(m[0].includes('Meteor.loginWithOidc(options)'));
});
console.log(`\n${passed} passing`);
tests/oidcLoginStateIsolation.test.cjs
+249
View File
@@ -0,0 +1,249 @@
'use strict';
// Plain-Node unit test (no Meteor) for #4897 — OAuth2/OIDC users intermittently
// showing stale, missing or ANOTHER user's data.
// Run: node tests/oidcLoginStateIsolation.test.cjs
//
// Root cause: packages/wekan-oidc kept per-login state in shared module-level /
// implicit-global variables:
// * oidc_server.js declared `var profile = {}; var serviceData = {};
// var userinfo = {};` at MODULE scope, so every login of every user reused
// the same three objects. Fields not overwritten by the current login
// leaked in from the previous login (refreshToken, whitelisted id-token
// claims, email under provider-specific branches), and two logins running
// concurrently (the handler awaits the token, userinfo and two
// Meteor.callAsync calls) interleaved writes so a user could be
// created/updated with another user's id/email/username.
// * loginHandler.js used implicit globals (teamArray, orgArray, isAdmin,
// teams, orgs, user_email, ...) and a `users` collection global leaked by
// the caller, with the same cross-login contamination under concurrency.
//
// This test checks BOTH layers:
// 1. Source-level guard on oidc_server.js (an ES module we cannot require
// from plain Node): per-login state must be declared INSIDE the
// OAuth.registerService callback, never at module scope.
// 2. Behavioral tests on loginHandler.js (plain CommonJS): concurrent calls
// must not contaminate each other, and the collection must be addressed
// via Meteor.users instead of a leaked `users` global.
const assert = require('assert');
const fs = require('fs');
const path = require('path');
let passed = 0;
function test(name, fn) {
fn();
passed += 1;
console.log(' ok -', name);
}
async function testAsync(name, fn) {
await fn();
passed += 1;
console.log(' ok -', name);
}
// --- 1. Source-level guards on oidc_server.js --------------------------------
const oidcServerPath = path.join(
__dirname, '..', 'packages', 'wekan-oidc', 'oidc_server.js');
const oidcServerSrc = fs.readFileSync(oidcServerPath, 'utf8');
const registerIdx = oidcServerSrc.indexOf("OAuth.registerService('oidc'");
test('POSITIVE: oidc_server.js registers the oidc OAuth service', () => {
assert.ok(registerIdx > -1, 'OAuth.registerService(\'oidc\' not found');
});
test('POSITIVE: profile/serviceData/userinfo are declared INSIDE the login handler', () => {
for (const name of ['profile', 'serviceData', 'userinfo']) {
const decl = new RegExp(`(?:var|let|const)\\s+${name}\\s*=\\s*\\{\\}`);
const m = decl.exec(oidcServerSrc);
assert.ok(m, `no fresh-object declaration found for ${name}`);
assert.ok(
m.index > registerIdx,
`${name} is declared at module scope (index ${m.index} before ` +
`registerService at ${registerIdx}); per-login state would be shared ` +
'between all OIDC logins (#4897)',
);
}
});
test('NEGATIVE: no module-scope mutable login-state objects before registerService', () => {
const head = oidcServerSrc.slice(0, registerIdx);
assert.ok(
!/(?:var|let|const)\s+(?:profile|serviceData|userinfo)\s*=/.test(head),
'login state must not be declared before OAuth.registerService',
);
});
test('NEGATIVE: no implicit-global `user =` / `users =` assignments remain', () => {
// An assignment like `users= Meteor.users` or `user = await ...` without a
// declaration keyword creates a true global shared by concurrent logins.
const bareAssign = /^\s*users?\s*=[^=]/m;
assert.ok(
!bareAssign.test(oidcServerSrc),
'found a bare user/users assignment (implicit global) in oidc_server.js',
);
assert.ok(
/const user = await Meteor\.users\.findOneAsync/.test(oidcServerSrc),
'expected the user lookups to be const-scoped Meteor.users.findOneAsync calls',
);
});
// --- 2. Behavioral tests on loginHandler.js ----------------------------------
const loginHandlerPath = path.join(
__dirname, '..', 'packages', 'wekan-oidc', 'loginHandler.js');
test('NEGATIVE: loginHandler no longer relies on a leaked `users` global', () => {
const src = fs.readFileSync(loginHandlerPath, 'utf8');
assert.ok(
!/await\s+users\./.test(src),
'loginHandler.js still calls the leaked `users` global; it must use Meteor.users',
);
assert.ok(
/Meteor\.users\.updateAsync/.test(src),
'loginHandler.js should update via Meteor.users.updateAsync',
);
});
// Stub the Meteor/Org/Team globals the handler expects, recording every
// Users-collection write per user id.
const updatesByUser = {};
function resetUpdates() {
for (const k of Object.keys(updatesByUser)) delete updatesByUser[k];
}
const tick = () => new Promise((resolve) => setImmediate(resolve));
global.Meteor = {
users: {
async updateAsync(selector, modifier) {
await tick(); // widen the interleaving window, like a real DB write
(updatesByUser[selector._id] = updatesByUser[selector._id] || [])
.push(modifier);
},
},
async callAsync() {
await tick();
},
};
global.Org = {
async findOneAsync(sel) {
await tick();
return { _id: 'org-' + sel.orgDisplayName };
},
};
global.Team = {
async findOneAsync(sel) {
await tick();
return { _id: 'team-' + sel.teamDisplayName };
},
};
// Deliberately NO global.users: before the fix loginHandler depended on the
// caller leaking `users = Meteor.users` into the global scope, so calling any
// updater here would throw ReferenceError.
const {
addGroupsWithAttributes,
addEmail,
changeFullname,
changeUsername,
} = require('../packages/wekan-oidc/loginHandler');
(async () => {
await testAsync('POSITIVE: two CONCURRENT logins keep their teams/admin flags apart', async () => {
resetUpdates();
const userA = { _id: 'userA', teams: [], orgs: [] };
const userB = { _id: 'userB', teams: [], orgs: [] };
await Promise.all([
addGroupsWithAttributes(userA, [{ displayName: 'alpha', isAdmin: true }]),
addGroupsWithAttributes(userB, [{ displayName: 'beta' }]),
]);
const teamsPushedTo = (id) => {
const push = updatesByUser[id].find((m) => m.$push && m.$push.teams);
return push.$push.teams.$each.map((t) => t.teamId);
};
// Before the fix teamArray/isAdmin were implicit globals: the interleaved
// logins pushed each other's teams and admin flags onto the same arrays.
assert.deepStrictEqual(teamsPushedTo('userA'), ['team-alpha']);
assert.deepStrictEqual(teamsPushedTo('userB'), ['team-beta']);
const isAdminSet = (id) => {
const set = updatesByUser[id].find(
(m) => m.$set && 'isAdmin' in m.$set);
return set.$set.isAdmin;
};
assert.strictEqual(isAdminSet('userA'), true);
assert.strictEqual(isAdminSet('userB'), false,
'userB inherited userA\'s admin flag — shared login state (#4897)');
});
await testAsync('POSITIVE: two CONCURRENT addEmail calls write each email to its own user', async () => {
resetUpdates();
await Promise.all([
addEmail({ _id: 'userA', emails: [] }, 'a@example.com'),
addEmail({ _id: 'userB', emails: [] }, 'b@example.com'),
]);
const emailsSetOn = (id) =>
updatesByUser[id][0].$set.emails.map((e) => e.address);
assert.deepStrictEqual(emailsSetOn('userA'), ['a@example.com']);
assert.deepStrictEqual(emailsSetOn('userB'), ['b@example.com']);
});
await testAsync('POSITIVE: addEmail moves a known secondary email to the front, verified', async () => {
resetUpdates();
await addEmail(
{
_id: 'userA',
emails: [
{ address: 'old@example.com', verified: true },
{ address: 'new@example.com', verified: false },
],
},
'new@example.com',
);
const set = updatesByUser.userA[0].$set.emails;
assert.strictEqual(set[0].address, 'new@example.com');
assert.strictEqual(set[0].verified, true);
assert.strictEqual(set.length, 2);
});
await testAsync('NEGATIVE: addEmail does NOT rewrite emails when the address is already primary', async () => {
resetUpdates();
await addEmail(
{ _id: 'userA', emails: [{ address: 'same@example.com', verified: true }] },
'same@example.com',
);
assert.strictEqual(updatesByUser.userA, undefined,
'no update expected when the OIDC email is already the primary one');
});
await testAsync('POSITIVE: changeUsername/changeFullname update the right fields when changed', async () => {
resetUpdates();
await changeUsername({ _id: 'userA', username: 'old' }, 'new');
await changeFullname(
{ _id: 'userA', username: 'new', profile: { fullname: 'Old Name' } },
'New Name',
);
assert.deepStrictEqual(updatesByUser.userA[0], { $set: { username: 'new' } });
assert.deepStrictEqual(updatesByUser.userA[1], {
$set: { 'profile.fullname': 'New Name' },
});
});
await testAsync('NEGATIVE: changeUsername/changeFullname skip the write when nothing changed', async () => {
resetUpdates();
await changeUsername({ _id: 'userA', username: 'same' }, 'same');
await changeFullname(
{ _id: 'userA', profile: { fullname: 'Same Name' } },
'Same Name',
);
assert.strictEqual(updatesByUser.userA, undefined,
'no update expected when the OIDC values match the stored ones');
});
console.log(`\n${passed} passing`);
})().catch((err) => {
console.error(err);
process.exit(1);
});
tests/oidcProfileMerge.test.cjs
+276
View File
@@ -0,0 +1,276 @@
'use strict';
// Plain-Node regression guard (no Meteor) for issue #4560: "User Profile is
// reset when changing the authentication method from LDAP to OIDC".
//
// Root cause: the OIDC branch of Accounts.onCreateUser in
// server/models/users.js links an incoming OIDC login to an existing account
// (matched by verified email, opt-in via OAUTH2_MERGE_EXISTING_USERS) by
// deleting the existing document and re-inserting it. It used to do
//
// existingUser.profile = user.profile;
//
// i.e. REPLACE the stored profile with the freshly built OIDC-only profile
// ({ initials, fullname, boardView }). That wiped profile.avatarUrl,
// profile.templatesBoardId and the template swimlane ids, language and every
// other preference — the reporter's "templates and avatars disappeared".
// The fix merges instead of replacing: all existing profile fields survive,
// missing fields are filled from the OIDC-derived profile, and the provider's
// fullname/initials are applied only when the provider actually asserted a
// fullname (otherwise fullname is just a fallback to the username and must
// not overwrite a real stored name).
//
// The onCreateUser hook is extracted from server/models/users.js source text
// and exercised BEHAVIORALLY in a VM with stubs, so this fails if the merge
// regresses, not only if the code is reworded.
// Run: node tests/oidcProfileMerge.test.cjs
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const src = fs.readFileSync(
path.join(__dirname, '..', 'server', 'models', 'users.js'),
'utf8',
);
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
// --- Extract the Accounts.onCreateUser(...) registration --------------------
// The hook is a top-level statement; the first `});` at column 0 after its
// start closes it (all nested closers inside the hook are indented).
const m = src.match(/Accounts\.onCreateUser\(async \(options, user\) => \{[\s\S]*?\n\}\);/);
assert.ok(m, 'Accounts.onCreateUser hook found in server/models/users.js');
function buildHook(env) {
let hook = null;
const removed = [];
const context = {
process: { env: Object.assign({}, env) },
console,
Accounts: {
onCreateUser(fn) {
hook = fn;
},
},
Meteor: {
Error: class MeteorError extends Error {
constructor(code, reason) {
super(reason || code);
this.error = code;
}
},
users: {
removeAsync: async selector => {
removed.push(selector);
},
},
setTimeout: () => {},
},
Random: { id: () => 'rndid12345678' },
// Free variable used by the hook for brand-new users; a no-op keeps the
// creation path exercisable without an Orgs collection.
autoAddOrgsByDomain: async () => {},
ReactiveCache: null, // set per scenario below
InvitationCodes: { removeAsync: async () => {} },
};
vm.createContext(context);
vm.runInContext(m[0], context);
assert.ok(hook, 'hook registered');
return { hook, context, removed };
}
// The existing (LDAP-era) account whose profile must survive the switch.
function ldapUser() {
return {
_id: 'ldapUserId1',
username: 'jdoe',
emails: [{ address: 'jdoe@example.com', verified: true }],
authenticationMethod: 'ldap',
services: { ldap: { id: 'jdoe' } },
profile: {
fullname: 'John Doe',
initials: 'JD',
avatarUrl: 'http://wekan.example/cfs/files/avatars/abc123',
templatesBoardId: 'tplBoard111',
cardTemplatesSwimlaneId: 'swimCard222',
listTemplatesSwimlaneId: 'swimList333',
boardTemplatesSwimlaneId: 'swimBoard444',
language: 'ja',
boardView: 'board-view-lists',
hiddenSystemMessages: true,
},
};
}
// The incoming user document accounts-base hands to onCreateUser for an OIDC
// login (fresh _id, services.oidc from the OAuth handler).
function oidcIncoming(overrides) {
return {
_id: 'freshOidcId1',
createdAt: new Date(),
services: {
oidc: Object.assign(
{
id: 'oidc-sub-1',
username: 'jdoe@example.com',
fullname: 'John A. Doe',
email: 'jdoe@example.com',
email_verified: true,
},
overrides || {},
),
},
};
}
function stubCaches(context, existing) {
context.ReactiveCache = {
getUsers: async () => ({ countAsync: async () => 5 }),
getUser: async selector => {
if (
existing &&
selector &&
selector['emails.address'] === existing.emails[0].address
) {
return existing;
}
return undefined;
},
getCurrentSetting: async () => ({ disableRegistration: false }),
getInvitationCode: async () => undefined,
};
}
const MERGE_ENV = { OAUTH2_MERGE_EXISTING_USERS: 'true' };
// --- POSITIVE: the #4560 scenario -------------------------------------------
test('LDAP -> OIDC merge keeps avatar, templates board/swimlanes and preferences (#4560)', async () => {
const { hook, context, removed } = buildHook(MERGE_ENV);
const existing = ldapUser();
stubCaches(context, existing);
const result = await hook({}, oidcIncoming());
// Linked, not duplicated: the existing account (same _id) is re-inserted.
assert.strictEqual(result._id, 'ldapUserId1');
assert.ok(
removed.some(sel => sel && sel._id === 'ldapUserId1'),
'existing doc removed before re-insert (delete+reinsert keeps the _id)',
);
// The heart of #4560: profile is UPDATED, not REPLACED.
assert.strictEqual(result.profile.avatarUrl, 'http://wekan.example/cfs/files/avatars/abc123');
assert.strictEqual(result.profile.templatesBoardId, 'tplBoard111');
assert.strictEqual(result.profile.cardTemplatesSwimlaneId, 'swimCard222');
assert.strictEqual(result.profile.listTemplatesSwimlaneId, 'swimList333');
assert.strictEqual(result.profile.boardTemplatesSwimlaneId, 'swimBoard444');
assert.strictEqual(result.profile.language, 'ja');
assert.strictEqual(result.profile.boardView, 'board-view-lists');
assert.strictEqual(result.profile.hiddenSystemMessages, true);
// Provider asserted a fullname: identity fields follow the provider.
assert.strictEqual(result.profile.fullname, 'John A. Doe');
assert.strictEqual(result.profile.initials, 'JAD');
// Auth switch itself still happens.
assert.strictEqual(result.authenticationMethod, 'oauth2');
assert.strictEqual(result.services.oidc.id, 'oidc-sub-1');
assert.strictEqual(result.services.ldap.id, 'jdoe', 'old service data untouched');
});
test('provider without a fullname claim does not clobber the stored fullname', async () => {
const { hook, context } = buildHook(MERGE_ENV);
const existing = ldapUser();
stubCaches(context, existing);
const result = await hook({}, oidcIncoming({ fullname: undefined }));
// Without the guard, user.profile.fullname falls back to the username
// ('jdoe@example.com') and used to overwrite the real name.
assert.strictEqual(result.profile.fullname, 'John Doe');
assert.strictEqual(result.profile.initials, 'JD');
assert.strictEqual(result.profile.avatarUrl, 'http://wekan.example/cfs/files/avatars/abc123');
});
test('existing account without any profile gets the OIDC-derived one (no crash)', async () => {
const { hook, context } = buildHook(MERGE_ENV);
const existing = ldapUser();
delete existing.profile;
stubCaches(context, existing);
const result = await hook({}, oidcIncoming());
assert.strictEqual(result.profile.fullname, 'John A. Doe');
assert.strictEqual(result.profile.initials, 'JAD');
assert.strictEqual(result.profile.boardView, 'board-view-swimlanes');
});
// --- NEGATIVE: fail-closed linking rules stay intact -------------------------
test('NEGATIVE: merge stays rejected when OAUTH2_MERGE_EXISTING_USERS is unset', async () => {
const { hook, context } = buildHook({});
const existing = ldapUser();
stubCaches(context, existing);
await assert.rejects(
() => hook({}, oidcIncoming()),
err => err.error === 'oidc-email-already-in-use',
);
});
test('NEGATIVE: merge stays rejected when the provider did not verify the email', async () => {
const { hook, context } = buildHook(MERGE_ENV);
const existing = ldapUser();
stubCaches(context, existing);
await assert.rejects(
() => hook({}, oidcIncoming({ email_verified: false })),
err => err.error === 'oidc-email-already-in-use',
);
});
test('NEGATIVE: brand-new OIDC user (no matching account) still gets a fresh profile', async () => {
const { hook, context } = buildHook(MERGE_ENV);
stubCaches(context, null);
const incoming = oidcIncoming();
const result = await hook({}, incoming);
assert.strictEqual(result._id, 'freshOidcId1');
assert.strictEqual(result.authenticationMethod, 'oauth2');
assert.strictEqual(result.profile.fullname, 'John A. Doe');
assert.strictEqual(result.profile.boardView, 'board-view-swimlanes');
});
// --- Structural guard: the wholesale replacement must not come back ----------
test('source no longer contains the wholesale `existingUser.profile = user.profile`', () => {
assert.ok(
!/existingUser\.profile\s*=\s*user\.profile\s*;/.test(src),
'profile must be merged, not replaced',
);
});
// --- Runner ------------------------------------------------------------------
(async () => {
let passed = 0;
for (const { name, fn } of tests) {
try {
await fn();
passed += 1;
console.log(' ok -', name);
} catch (err) {
console.error(' FAIL -', name);
console.error(err);
process.exit(1);
}
}
console.log(`\nAll ${passed} oidcProfileMerge tests passed.`);
})();
tests/schemaUpgradeSteps.test.cjs
+706
View File
@@ -0,0 +1,706 @@
'use strict';
// Plain-Node BEHAVIORAL tests (no Meteor) for the startup schema upgrade
// (server/lib/schemaUpgradeSteps.js) — the #6473 follow-up that brings TEXT
// DATA from every old WeKan version to the newest database structure on all
// platforms and both databases. Guards the whole contract so later changes
// that would break migrating SOME data shape fail here:
// - every step migrates its old shape (positive) and leaves healthy/explicit
// data alone (negative);
// - the re-check is VERSION-GATED: mandatory only after a new WeKan release,
// a single findOne while the version is unchanged;
// - big-database speed: checks are bounded probes/distinct() set-joins and
// fixes are server-side updateMany batches (asserted via op counters);
// - a failing or unresolved step never blocks startup and forces the next
// boot to re-check.
//
// Runs against a minimal in-memory mongodb-driver-compatible Db fake (only
// the operators the module actually uses).
// Run: node tests/schemaUpgradeSteps.test.cjs
const assert = require('assert');
const fs = require('fs');
const os = require('os');
const path = require('path');
const repoRoot = path.resolve(__dirname, '..');
const read = rel => fs.readFileSync(path.join(repoRoot, rel), 'utf8');
const {
runSchemaUpgrade,
getUpgradeState,
BOARD_ALLOWS_TRUE_DEFAULTS,
randomId,
versionNeedsHeal,
MARKER_ID,
} = require('../server/lib/schemaUpgradeSteps');
let passed = 0;
function test(name, fn) {
const r = fn();
if (r && typeof r.then === 'function') {
return r.then(() => { passed += 1; console.log(' ok -', name); });
}
passed += 1;
console.log(' ok -', name);
return Promise.resolve();
}
// ── minimal in-memory Db fake (mongodb driver surface the module uses) ───────
function getPath(doc, key) {
return key.split('.').reduce((v, part) => (v == null ? v : v[part]), doc);
}
function matchCond(value, cond) {
if (cond && typeof cond === 'object' && !Array.isArray(cond)) {
if ('$exists' in cond) return cond.$exists === (value !== undefined);
if ('$in' in cond) {
return cond.$in.some(x => (x === null ? value === undefined || value === null : String(value) === String(x)));
}
if ('$nin' in cond) {
return !cond.$nin.some(x => (x === null ? value === undefined || value === null : String(value) === String(x)));
}
if ('$ne' in cond) {
return String(value) !== String(cond.$ne);
}
if ('$elemMatch' in cond) {
if (!Array.isArray(value)) return false;
return value.some(el => matches(el, cond.$elemMatch));
}
throw new Error('fake db: unsupported operator in ' + JSON.stringify(cond));
}
return String(value) === String(cond) || value === cond;
}
function matches(doc, filter) {
for (const [key, cond] of Object.entries(filter || {})) {
if (key === '$or') {
if (!cond.some(sub => matches(doc, sub))) return false;
continue;
}
if (!matchCond(getPath(doc, key), cond)) return false;
}
return true;
}
function applyUpdate(doc, update) {
for (const [k, v] of Object.entries(update.$set || {})) {
const parts = k.split('.');
let target = doc;
for (const p of parts.slice(0, -1)) {
if (typeof target[p] !== 'object' || target[p] === null) target[p] = {};
target = target[p];
}
target[parts[parts.length - 1]] = v;
}
for (const k of Object.keys(update.$unset || {})) {
const parts = k.split('.');
let target = doc;
for (const p of parts.slice(0, -1)) {
target = target && target[p];
if (!target) break;
}
if (target) delete target[parts[parts.length - 1]];
}
}
function fakeDb(initial = {}) {
const data = new Map(Object.entries(initial).map(([k, docs]) => [k, docs.map(d => JSON.parse(JSON.stringify(d)))]));
const ops = { findOne: 0, find: 0, distinct: 0, updateMany: 0, updateOne: 0, insertOne: 0, insertMany: 0, countDocuments: 0 };
const coll = name => {
if (!data.has(name)) data.set(name, []);
const docs = () => data.get(name);
return {
async findOne(filter = {}, opts = {}) {
ops.findOne++;
let found = docs().filter(d => matches(d, filter));
if (opts.sort) {
const [[k, dir]] = Object.entries(opts.sort);
found = found.sort((a, b) => ((a[k] || 0) - (b[k] || 0)) * dir);
}
return found[0] ? JSON.parse(JSON.stringify(found[0])) : null;
},
find(filter = {}) {
ops.find++;
const found = docs().filter(d => matches(d, filter)).map(d => JSON.parse(JSON.stringify(d)));
let i = 0;
return {
async toArray() { return found; },
async close() {},
[Symbol.asyncIterator]() {
return { async next() { return i < found.length ? { value: found[i++], done: false } : { value: undefined, done: true }; } };
},
};
},
async distinct(field, filter = {}) {
ops.distinct++;
const vals = new Set();
for (const d of docs()) if (matches(d, filter)) {
const v = getPath(d, field);
if (v !== undefined) vals.add(typeof v === 'string' ? v : String(v));
}
return [...vals];
},
async countDocuments(filter = {}) {
ops.countDocuments++;
return docs().filter(d => matches(d, filter)).length;
},
async deleteOne(filter) {
const i = docs().findIndex(d => matches(d, filter));
if (i >= 0) { docs().splice(i, 1); return { deletedCount: 1 }; }
return { deletedCount: 0 };
},
async insertOne(doc) { ops.insertOne++; docs().push(JSON.parse(JSON.stringify(doc))); return { insertedId: doc._id }; },
async insertMany(list) { ops.insertMany++; for (const d of list) docs().push(JSON.parse(JSON.stringify(d))); return { insertedCount: list.length }; },
async updateOne(filter, update, opts = {}) {
ops.updateOne++;
const target = docs().find(d => matches(d, filter));
if (target) { applyUpdate(target, update); return { modifiedCount: 1 }; }
if (opts.upsert) {
const fresh = { ...(filter._id ? { _id: filter._id } : {}) };
applyUpdate(fresh, update);
docs().push(fresh);
return { upsertedId: fresh._id, modifiedCount: 0 };
}
return { modifiedCount: 0 };
},
async updateMany(filter, update) {
ops.updateMany++;
let n = 0;
for (const d of docs()) if (matches(d, filter)) { applyUpdate(d, update); n++; }
return { modifiedCount: n };
},
};
};
return { collection: coll, _dump: name => data.get(name) || [], _ops: ops };
}
const tmpWritable = fs.mkdtempSync(path.join(os.tmpdir(), 'schema-upgrade-'));
const opts = { writablePath: tmpWritable, appVersion: 'v9.97.0' };
(async () => {
await test('randomId produces Meteor-style 17-char ids', () => {
const id = randomId();
assert.strictEqual(id.length, 17);
assert.ok(/^[23456789A-HJKLMNP-Zabcdefghijkmnop-z]+$/.test(id));
assert.notStrictEqual(randomId(), randomId());
});
// ── swimlane-structure ────────────────────────────────────────────────────────
await test('swimlane-structure: creates Default swimlane, backfills lists/cards, rescues dangling listId', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', title: 'Old board' }],
lists: [{ _id: 'l1', boardId: 'b1', title: 'Todo' }], // no swimlaneId
cards: [
{ _id: 'c1', boardId: 'b1', listId: 'l1', title: 'ok card' }, // no swimlaneId
{ _id: 'c2', boardId: 'b1', listId: 'GONE', title: 'orphan' },// dangling listId
],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('swimlane-structure'));
const sw = db._dump('swimlanes');
assert.strictEqual(sw.length, 1);
assert.strictEqual(sw[0].title, 'Default');
const l1 = db._dump('lists').find(l => l._id === 'l1');
assert.strictEqual(l1.swimlaneId, undefined,
'lists are NOT stamped: empty/missing swimlaneId means SHARED across swimlanes');
const c1 = db._dump('cards').find(c => c._id === 'c1');
assert.strictEqual(c1.swimlaneId, String(sw[0]._id));
const c2 = db._dump('cards').find(c => c._id === 'c2');
assert.strictEqual(c2.listId, 'l1', 'orphan rescued to the board\'s first list');
});
await test('swimlane-structure: creates a Rescued Data list when the board has none', async () => {
const db = fakeDb({
boards: [{ _id: 'b1' }],
swimlanes: [{ _id: 's1', boardId: 'b1', title: 'Default', sort: 0 }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'GONE', swimlaneId: 's1' }],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('swimlane-structure'));
const rescued = db._dump('lists').find(l => l.title === 'Rescued Data');
assert.ok(rescued, 'Rescued Data list created');
assert.strictEqual(db._dump('cards')[0].listId, String(rescued._id));
});
await test('speed: card fixes are server-side updateMany batches, never per-card round trips', async () => {
const manyCards = [];
for (let i = 0; i < 500; i++) manyCards.push({ _id: 'c' + i, boardId: 'b1', listId: 'l1', title: 't' + i });
const db = fakeDb({
boards: [{ _id: 'b1' }],
lists: [{ _id: 'l1', boardId: 'b1' }],
cards: manyCards, // 500 cards all missing swimlaneId
});
await runSchemaUpgrade(db, opts);
assert.ok(db._dump('cards').every(c => c.swimlaneId), 'all 500 cards fixed');
assert.ok(db._ops.updateOne < 20,
`per-document updateOne calls must stay constant-ish (got ${db._ops.updateOne}) — thousands of cards must not mean thousands of round trips`);
assert.ok(db._ops.updateMany >= 2, 'fixes go through updateMany');
});
await test('negative: swimlane-structure does not run on healthy data', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }],
swimlanes: [{ _id: 's1', boardId: 'b1', archived: false }],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 's1', archived: false }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 's1', archived: false }],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(!res.ran.includes('swimlane-structure'));
assert.ok(!res.ran.includes('archived-flag-backfill'));
});
// ── views visibility: Swimlanes view + Lists view (#1959, #1971) ─────────────
await test('archived-flag-backfill: docs missing `archived` become visible to archived:false view queries', async () => {
const db = fakeDb({
boards: [{ _id: 'b1' }], // LibreBoard-era: no archived field
swimlanes: [{ _id: 's1', boardId: 'b1' }],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 's1' }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 's1' }],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('archived-flag-backfill'));
for (const coll of ['boards', 'swimlanes', 'lists', 'cards']) {
assert.strictEqual(db._dump(coll)[0].archived, false, `${coll} now matches archived:false`);
}
});
await test('negative: archived-flag-backfill never flips explicitly archived docs', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: true }],
swimlanes: [{ _id: 's1', boardId: 'b1', archived: false }],
});
await runSchemaUpgrade(db, opts);
assert.strictEqual(db._dump('boards')[0].archived, true, 'archived board stays archived');
});
await test('#1959: card under a DELETED swimlane is rescued to the board\'s visible swimlane', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }],
swimlanes: [{ _id: 'sVisible', boardId: 'b1', archived: false, sort: 0 }],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 'sVisible', archived: false }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 'sDELETED', archived: false }],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('swimlane-structure'));
assert.strictEqual(db._dump('cards')[0].swimlaneId, 'sVisible', 'card visible in Swimlanes view again');
});
await test('#1971: card under an ARCHIVED swimlane is rescued to a visible one', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }],
swimlanes: [
{ _id: 'sArchived', boardId: 'b1', archived: true, sort: 0 },
{ _id: 'sVisible', boardId: 'b1', archived: false, sort: 1 },
],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 'sVisible', archived: false }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 'sArchived', archived: false }],
});
await runSchemaUpgrade(db, opts);
assert.strictEqual(db._dump('cards')[0].swimlaneId, 'sVisible');
});
await test('#1971: when EVERY swimlane is archived, a visible Default is created for the rescue', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }],
swimlanes: [{ _id: 'sArchived', boardId: 'b1', archived: true, sort: 0 }],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 'sArchived', archived: false }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 'sArchived', archived: false }],
});
await runSchemaUpgrade(db, opts);
const visible = db._dump('swimlanes').find(s => s.archived === false);
assert.ok(visible, 'a visible Default swimlane was created');
assert.strictEqual(db._dump('cards')[0].swimlaneId, String(visible._id));
});
await test('negative: ARCHIVED cards under archived swimlanes are left alone (intentionally hidden)', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }],
swimlanes: [
{ _id: 'sArchived', boardId: 'b1', archived: true, sort: 0 },
{ _id: 'sVisible', boardId: 'b1', archived: false, sort: 1 },
],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 'sVisible', archived: false }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 'sArchived', archived: true }],
});
await runSchemaUpgrade(db, opts);
assert.strictEqual(db._dump('cards')[0].swimlaneId, 'sArchived', 'archived card untouched');
});
await test('cross-board: card pointing at ANOTHER board\'s swimlane is pulled back to its own board', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }, { _id: 'b2', archived: false }],
swimlanes: [
{ _id: 's1', boardId: 'b1', archived: false, sort: 0 },
{ _id: 's2', boardId: 'b2', archived: false, sort: 0 },
],
lists: [{ _id: 'l1', boardId: 'b1', swimlaneId: 's1', archived: false }],
cards: [{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 's2', archived: false }],
});
await runSchemaUpgrade(db, opts);
assert.strictEqual(db._dump('cards')[0].swimlaneId, 's1');
});
await test('merge-per-swimlane-lists: v8.x duplicate list copies merged into one shared list', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false, fixMissingListsCompleted: true }],
swimlanes: [
{ _id: 's1', boardId: 'b1', archived: false, sort: 0 },
{ _id: 's2', boardId: 'b1', archived: false, sort: 1 },
],
lists: [
{ _id: 'l1', boardId: 'b1', title: 'Todo', swimlaneId: 's1', archived: false, createdAt: 1 },
{ _id: 'l2', boardId: 'b1', title: 'Todo', swimlaneId: 's2', archived: false, createdAt: 2 },
],
cards: [
{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 's1', archived: false },
{ _id: 'c2', boardId: 'b1', listId: 'l2', swimlaneId: 's2', archived: false },
],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('merge-per-swimlane-lists'));
const lists = db._dump('lists');
assert.strictEqual(lists.length, 1, 'duplicate copy deleted');
assert.strictEqual(lists[0].swimlaneId, '', 'canonical list is SHARED (renders in every swimlane)');
const cards = db._dump('cards');
assert.ok(cards.every(c => c.listId === 'l1'), 'cards repointed to the canonical list');
assert.strictEqual(cards[0].swimlaneId, 's1', 'cards KEEP their swimlane (per-swimlane grouping preserved)');
assert.strictEqual(cards[1].swimlaneId, 's2');
const b = db._dump('boards')[0];
assert.strictEqual(b.fixMissingListsCompleted, undefined, 'era marker cleared so repair tools work again');
});
await test('merge-per-swimlane-lists: v7.98 stamped lists (Shape A) become shared again', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false }],
swimlanes: [
{ _id: 's1', boardId: 'b1', archived: false, sort: 0 },
{ _id: 's2', boardId: 'b1', archived: false, sort: 1 },
],
lists: [{ _id: 'l1', boardId: 'b1', title: 'Todo', swimlaneId: 's1', archived: false }],
cards: [{ _id: 'c2', boardId: 'b1', listId: 'l1', swimlaneId: 's2', archived: false }], // other swimlane's card
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('merge-per-swimlane-lists'));
assert.strictEqual(db._dump('lists')[0].swimlaneId, '', 'stamped list unshared so every swimlane renders it');
assert.strictEqual(db._dump('cards')[0].swimlaneId, 's2', 'card untouched');
});
await test('negative: merge-per-swimlane-lists leaves healthy and RENAMED per-swimlane lists alone', async () => {
const db = fakeDb({
boards: [{ _id: 'b1', archived: false, fixMissingListsCompleted: true }],
swimlanes: [
{ _id: 's1', boardId: 'b1', archived: false, sort: 0 },
{ _id: 's2', boardId: 'b1', archived: false, sort: 1 },
],
lists: [
{ _id: 'l1', boardId: 'b1', title: 'Todo (team A)', swimlaneId: 's1', archived: false },
{ _id: 'l2', boardId: 'b1', title: 'Todo (team B)', swimlaneId: 's2', archived: false },
],
cards: [
{ _id: 'c1', boardId: 'b1', listId: 'l1', swimlaneId: 's1', archived: false },
{ _id: 'c2', boardId: 'b1', listId: 'l2', swimlaneId: 's2', archived: false },
],
});
await runSchemaUpgrade(db, opts);
const lists = db._dump('lists');
assert.strictEqual(lists.length, 2, 'differently RENAMED copies are kept (their names carry meaning)');
assert.ok(lists.every(l => l.swimlaneId === ''), 'both become shared');
assert.ok(db._dump('cards').every((c, i) => c.listId === ['l1', 'l2'][i]), 'cards keep their own list');
});
await test('write path (#1959/#1971): card insert hook validates client-supplied swimlaneIds', () => {
const hook = read('server/migrations/ensureValidSwimlaneIds.js');
assert.ok(hook.includes('#1959/#1971'), 'the visibility fix is documented at the hook');
const insertHook = hook.match(/Cards\.before\.insert\([\s\S]*?\n \}\);/)[0];
assert.ok(/if \(doc\.swimlaneId\) \{/.test(insertHook),
'a PROVIDED swimlaneId must be validated, not only a missing one');
assert.ok(insertHook.includes('archived: false'), 'must resolve to a VISIBLE swimlane');
assert.ok(insertHook.includes('boardId: doc.boardId'), 'must belong to the same board');
});
// ── checklist-items-embedded ─────────────────────────────────────────────────
await test('checklist-items-embedded: extracts pre-v0.79 embedded items to ChecklistItems', async () => {
const db = fakeDb({
checklists: [{
_id: 'ck1', cardId: 'c1', title: 'Todo',
items: [
{ title: 'second', sort: 2, isFinished: true },
{ title: 'first', sort: 1, isFinished: false },
{ sort: 3 }, // untitled item — original migration used 'Checklist'
],
}],
});
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('checklist-items-embedded'));
const items = db._dump('checklistItems');
assert.strictEqual(items.length, 3);
assert.deepStrictEqual(items.map(i => i.title), ['first', 'second', 'Checklist'], 'sorted by embedded sort');
assert.deepStrictEqual(items.map(i => i.sort), [0, 1, 2], 're-numbered like the v8.00 migration');
assert.strictEqual(items[1].isFinished, true);
assert.ok(items.every(i => i.checklistId === 'ck1' && i.cardId === 'c1' && i._id.length === 17));
assert.strictEqual(db._dump('checklists')[0].items, undefined, 'embedded array removed');
});
await test('negative: checklist extraction does not duplicate items on a re-run after a crash', async () => {
const db = fakeDb({
checklists: [{ _id: 'ck1', cardId: 'c1', items: [{ title: 'a', sort: 0 }, { title: 'b', sort: 1 }] }],
// 'a' was already extracted before the interruption:
checklistItems: [{ _id: 'x1', checklistId: 'ck1', cardId: 'c1', title: 'a', sort: 0, isFinished: false }],
});
await runSchemaUpgrade(db, opts);
const items = db._dump('checklistItems');
assert.strictEqual(items.length, 2, 'no duplicate of "a"');
});
// ── customfields-boardIds ────────────────────────────────────────────────────
await test('customfields-boardIds: scalar boardId becomes boardIds array', async () => {
const db = fakeDb({ customFields: [{ _id: 'cf1', name: 'Severity', boardId: 'b1' }] });
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('customfields-boardIds'));
const cf = db._dump('customFields')[0];
assert.deepStrictEqual(cf.boardIds, ['b1']);
assert.strictEqual(cf.boardId, undefined);
});
await test('negative: customfields with an existing boardIds array only lose the stale scalar', async () => {
const db = fakeDb({ customFields: [{ _id: 'cf1', boardId: 'old', boardIds: ['b1', 'b2'] }] });
await runSchemaUpgrade(db, opts);
const cf = db._dump('customFields')[0];
assert.deepStrictEqual(cf.boardIds, ['b1', 'b2'], 'existing array untouched');
assert.strictEqual(cf.boardId, undefined);
});
// ── board-allows-defaults ────────────────────────────────────────────────────
await test('board-allows-defaults: missing true-default flags are backfilled, explicit false respected', async () => {
const db = fakeDb({ boards: [
{ _id: 'b1', allowsComments: false }, // user turned comments OFF
{ _id: 'b2' }, // ancient board: nothing set
] });
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('board-allows-defaults'));
const [b1, b2] = db._dump('boards');
assert.strictEqual(b1.allowsComments, false, 'explicit user choice untouched');
assert.strictEqual(b1.allowsChecklists, true, 'missing flag backfilled');
assert.strictEqual(b2.allowsDescriptionText, true);
assert.strictEqual(b2.allowsAttachments, true);
});
await test('the true-defaults list stays in sync with models/boards.js', () => {
const src = read('models/boards.js');
const re = /(allows[A-Za-z0-9]+):\s*\{[^}]*?defaultValue:\s*(true|false)/g;
const fromSchema = [];
let m;
while ((m = re.exec(src))) if (m[2] === 'true') fromSchema.push(m[1]);
assert.deepStrictEqual([...BOARD_ALLOWS_TRUE_DEFAULTS].sort(), fromSchema.sort(),
'BOARD_ALLOWS_TRUE_DEFAULTS must match the schema defaults (update both together)');
});
// ── board-members-isactive ───────────────────────────────────────────────────
await test('board-members-isactive: missing isActive becomes true, explicit false stays false', async () => {
const db = fakeDb({ boards: [
{ _id: 'b1', members: [{ userId: 'u1' }, { userId: 'u2', isActive: false }, { userId: 'u3', isActive: true }] },
] });
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('board-members-isactive'));
const members = db._dump('boards')[0].members;
assert.strictEqual(members[0].isActive, true, 'pre-2015 member regains access');
assert.strictEqual(members[1].isActive, false, 'deactivated member stays deactivated');
assert.strictEqual(members[2].isActive, true);
});
// ── board-permission-lowercase ───────────────────────────────────────────────
await test('board-permission-lowercase: PUBLIC boards become public again (batched updateMany)', async () => {
const db = fakeDb({ boards: [
{ _id: 'b1', permission: 'PUBLIC' },
{ _id: 'b2', permission: 'private' },
] });
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('board-permission-lowercase'));
assert.strictEqual(db._dump('boards')[0].permission, 'public');
assert.strictEqual(db._dump('boards')[1].permission, 'private', 'already-lowercase untouched');
});
// ── fs-path-heal ─────────────────────────────────────────────────────────────
await test('fs-path-heal: stale-path record repointed to the binary found under the old uploads layout', async () => {
const attachDir = path.join(tmpWritable, 'files', 'attachments');
const oldDir = path.join(tmpWritable, 'files', 'uploads', 'attachments');
fs.mkdirSync(attachDir, { recursive: true });
fs.mkdirSync(oldDir, { recursive: true });
fs.writeFileSync(path.join(oldDir, 'legacy.bin'), 'legacy-bytes');
const db = fakeDb({ attachments: [{
_id: 'a1', name: 'legacy.bin',
versions: { original: { storage: 'fs', path: '/nonexistent/old/legacy.bin', size: 12 } },
}] });
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('fs-path-heal'));
const a = db._dump('attachments')[0];
assert.ok(a.versions.original.path.startsWith(attachDir), 'repointed into the current layout');
assert.ok(fs.existsSync(a.versions.original.path), 'binary copied');
assert.strictEqual(fs.readFileSync(a.versions.original.path, 'utf8'), 'legacy-bytes');
assert.strictEqual(a.versions.original.storage, 'fs');
});
await test('fs-path-heal: flagless record without gridFsFileId (failed v6.10-18 GridFS copy) is healed too', () => {
assert.strictEqual(versionNeedsHeal({}, { size: 1, path: '/gone' }), true, 'no flag + no gridFsFileId + missing path');
assert.strictEqual(versionNeedsHeal({}, { size: 1, meta: { gridFsFileId: 'x' } }), false, 'gridfs reference -> not an fs heal case');
assert.strictEqual(versionNeedsHeal({}, { storage: 's3', path: '/gone' }), false, 'cloud storage untouched');
});
await test('negative: fs-path-heal with an unfindable binary leaves the version UN-STAMPED so the next boot re-checks', async () => {
const db = fakeDb({ avatars: [{
_id: 'av1', name: 'gone.png',
versions: { original: { storage: 'fs', path: '/nonexistent/gone.png' } },
}] });
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.ran.includes('fs-path-heal'));
assert.strictEqual(res.results['fs-path-heal'].unresolved, 1);
const marker = await db.collection('_wekan_migration').findOne({ _id: MARKER_ID });
assert.ok(!marker || !marker.lastCheck || marker.lastCheck.version !== opts.appVersion,
'unresolved work must keep the version un-stamped (re-check next boot)');
});
// ── version gating: re-check mandatory only after a NEW RELEASE ───────────────
await test('same WeKan version: second boot is gated to a single findOne (no scans, no fixes)', async () => {
const db = fakeDb({});
const first = await runSchemaUpgrade(db, opts);
assert.strictEqual(first.gated, false);
const marker = await db.collection('_wekan_migration').findOne({ _id: MARKER_ID });
assert.strictEqual(marker.lastCheck.version, opts.appVersion, 'version + datetime recorded');
assert.ok(marker.lastCheck.at, 'datetime of the re-check saved');
// plant NEW old-shape data; same version -> must NOT be scanned or touched
await db.collection('checklists').insertOne({ _id: 'ck9', cardId: 'c9', items: [{ title: 'x', sort: 0 }] });
const before = { ...db._ops };
const second = await runSchemaUpgrade(db, opts);
assert.strictEqual(second.gated, true, 'gated: version unchanged');
assert.strictEqual(second.ran.length, 0);
assert.strictEqual(db._dump('checklistItems').length, 0, 'no work performed while gated');
assert.ok(db._ops.findOne - before.findOne <= 2, 'gated boot costs at most the marker read (+product name)');
assert.strictEqual(db._ops.updateMany, before.updateMany, 'no writes while gated');
});
await test('NEW WeKan version: the re-check is mandatory and picks up newly-broken data', async () => {
const db = fakeDb({});
await runSchemaUpgrade(db, opts); // v9.97.0 clean
await db.collection('checklists').insertOne({ _id: 'ck9', cardId: 'c9', items: [{ title: 'x', sort: 0 }] });
const res = await runSchemaUpgrade(db, { ...opts, appVersion: 'v9.98.0' });
assert.strictEqual(res.gated, false, 'new release -> full re-check');
assert.ok(res.ran.includes('checklist-items-embedded'));
assert.strictEqual(db._dump('checklistItems').length, 1, 'the new old-shape data was migrated');
const marker = await db.collection('_wekan_migration').findOne({ _id: MARKER_ID });
assert.strictEqual(marker.lastCheck.version, 'v9.98.0');
});
await test('force option overrides the version gate', async () => {
const db = fakeDb({});
await runSchemaUpgrade(db, opts);
await db.collection('boards').insertOne({ _id: 'b1', permission: 'PUBLIC' });
const res = await runSchemaUpgrade(db, { ...opts, force: true });
assert.strictEqual(res.gated, false);
assert.strictEqual(db._dump('boards')[0].permission, 'public');
});
await test('negative: unresolved work prevents the version stamp, so the SAME version re-checks next boot', async () => {
const db = fakeDb({ avatars: [{ _id: 'av1', name: 'gone.png', versions: { original: { storage: 'fs', path: '/nonexistent/gone.png' } } }] });
await runSchemaUpgrade(db, opts); // unresolved heal
const res2 = await runSchemaUpgrade(db, opts); // same version
assert.strictEqual(res2.gated, false, 'must re-check because the previous run was not clean');
});
// ── orchestrator semantics ───────────────────────────────────────────────────
await test('healthy database: everything skipped, version stamped', async () => {
const db = fakeDb({});
const res = await runSchemaUpgrade(db, opts);
assert.strictEqual(res.ran.length, 0);
const marker = await db.collection('_wekan_migration').findOne({ _id: MARKER_ID });
assert.ok(marker && marker.lastCheck, 'lastCheck written so later boots cost one findOne');
});
await test('a throwing step never breaks the run: later steps still execute, version stays un-stamped', async () => {
const db = fakeDb({ boards: [{ _id: 'b1', permission: 'PUBLIC' }] });
const real = db.collection;
db.collection = name => {
const c = real(name);
if (name === 'checklists') return { ...c, findOne: async () => { throw new Error('boom'); } };
return c;
};
const res = await runSchemaUpgrade(db, opts);
assert.ok(res.results['checklist-items-embedded'].error, 'failure recorded');
assert.ok(res.ran.includes('board-permission-lowercase'), 'later steps still ran');
assert.strictEqual(db._dump('boards')[0].permission, 'public');
const marker = await db.collection('_wekan_migration').findOne({ _id: MARKER_ID });
assert.ok(!marker || !marker.lastCheck, 'failed step keeps the version un-stamped');
});
// ── progress state for the migration dashboard ───────────────────────────────
await test('dashboard state: progress, per-step results, and the product name when set', async () => {
const db = fakeDb({
settings: [{ _id: 's1', productName: 'AcmeBoards' }],
boards: [{ _id: 'b1', permission: 'PUBLIC' }],
});
await runSchemaUpgrade(db, opts);
const state = getUpgradeState();
assert.strictEqual(state.product, 'AcmeBoards', 'dashboard shows the Admin Panel product name, not WeKan');
assert.strictEqual(state.running, false);
assert.ok(state.startedAt && state.finishedAt);
assert.strictEqual(state.steps['board-permission-lowercase'].status, 'done');
assert.strictEqual(state.steps['board-permission-lowercase'].fixed, 1);
assert.strictEqual(state.steps['checklist-items-embedded'].status, 'skipped');
});
await test('negative: no product name set -> dashboard says WeKan', async () => {
const db = fakeDb({});
await runSchemaUpgrade(db, opts);
assert.strictEqual(getUpgradeState().product, 'WeKan');
});
// ── wiring ───────────────────────────────────────────────────────────────────
await test('the startup wrapper is imported, non-blocking, and serves the dashboard route', () => {
assert.ok(read('server/imports.js').includes("import '/server/startupSchemaUpgrade'"));
const wrapper = read('server/startupSchemaUpgrade.js');
assert.ok(wrapper.includes('runSchemaUpgrade'));
assert.ok(wrapper.includes('Meteor.startup'));
assert.ok(wrapper.includes('WEKAN_SKIP_SCHEMA_UPGRADE'), 'opt-out env var exists');
assert.ok(wrapper.includes("WebApp.handlers.get('/schema-upgrade-status'"), 'dashboard route registered');
assert.ok(wrapper.includes('state.product'), 'dashboard uses the product name');
assert.ok(/Deliberately NOT awaited/i.test(wrapper), 'upgrade must not block WeKan from serving');
assert.ok(wrapper.includes("require('/package.json').version"), 'version for the release gate');
});
await test('the importer no longer stamps the flags that disabled the repair tools', () => {
const importer = read('releases/migrate-mongodb-to-ferretdb.mjs');
const fn = importer.match(/function upgradeBoard\(doc\) \{[\s\S]*?\n\}/)[0];
assert.ok(!fn.includes('comprehensiveMigrationCompleted: true'));
assert.ok(!fn.includes('fixMissingListsCompleted: true'));
assert.ok(fn.includes('migrationVersion: SCHEMA_VER'));
});
await test('speed: importer text phase batches with insertMany and only falls back per-document', () => {
const importer = read('releases/migrate-mongodb-to-ferretdb.mjs');
const flush = importer.match(/const flush = async \(\) => \{[\s\S]*?\n \};/)[0];
assert.ok(flush.includes('insertMany(batch, { ordered: false })'),
'one round trip per batch — per-document replaceOne made big migrations take hours');
assert.ok(flush.includes('replaceOne({ _id: doc._id }, doc, { upsert: true })'),
'per-document fallback for resumed/re-run migrations still exists');
});
await test('speed: importer file phases use the preloaded target map, not per-file findOne', () => {
const importer = read('releases/migrate-mongodb-to-ferretdb.mjs');
assert.ok(importer.includes('async function getTgtRec(collName, id)'));
assert.ok((importer.match(/await getTgtRec\(/g) || []).length >= 4,
'CFS phase, FS-store phase, pass A and pass B all use the cache');
assert.ok(importer.includes('TGT_PRELOAD_LIMIT'),
'very large collections fall back to per-file lookups to bound memory');
});
console.log(`schemaUpgradeSteps.test.cjs: ${passed} tests passed`);
})().catch(e => { console.error('FAIL:', e); process.exit(1); });