mirror of
https://github.com/wekan/wekan.git
synced 2026-11-07 11:02:33 -05:00
Lauri Ojansivu
d56bb34490
On a seeded board of 4,000 cards, 12,000 comments, 20,000 checklist items and 32,000 activities, served from FerretDB with SQLite as the Snap is: closing a card took about 4 s and now takes 0.1 s, the board finished loading in 17.6 s and now in 6.2 s, a card open re-ran about 7,700 computations and now about 1,000, and FerretDB used 13 s of CPU and 724 MB while the board opened, now 3 s and 280 MB. - The client data cache queued one more stop callback each time a helper re-ran, and every check asked Tracker for dependents, which on close was 710 walks of 3 ms; there is now one callback per computation and key and one timer per key (imports/lib/dataCache.js). - Opening a card writes the user's last-viewed card, and avatars, @mentions, list and swimlane headers read the whole user, so everything re-ran; they read only the fields they show. The comment, checklist and subtask indexes were one object every minicard depended on and are now per card (imports/lib/keyedGroupIndex.js). - The lazy card window kept a full copy of every card of a list per window; it keeps the id and the sort keys, runs its id query once instead of five times, and sends comment text only when minicards show comments (server/publications/cardsWindow.js, models/lib/cardWindowFields.js). Comment reactions and legacy attachment lookups come with the opened card only. - Attachment downloads honour Range for filesystem, GridFS and cloud storage (206, 416, If-Range); uploads to cloud storage are spooled to a temporary file and streamed instead of gathered into one Buffer; the API copy checks the size limit before reading; the legacy attachment migration streams, and no longer stores "[object Promise]" as a file; the board export no longer concatenates buffers quadratically. The FerretDB fork pushes dotted-path filters such as meta.cardId down to SQLite (they were each a full scan) and decodes only the fields a count filters on. docs/Features/Admin-Panel/Problems/Large-Boards.md describes how a large board loads and how to size a server with 8 GB of RAM. Thanks to markusst1982 and xet7 ! Fixes #6745.
62 lines
2.5 KiB
JavaScript
62 lines
2.5 KiB
JavaScript
'use strict';
|
|
|
|
// #6745: HTTP Range requests (RFC 9110 §14) for attachment downloads. Pure, so
|
|
// tests/attachmentRangeStreaming.test.cjs runs it without a server.
|
|
//
|
|
// Without Range support a browser that plays a video, or resumes a download,
|
|
// had to receive the whole file from the start each time: seeking in a 1 GB
|
|
// video re-read 1 GB from storage. With it, the server streams just the bytes
|
|
// asked for, straight from the storage backend (fs, GridFS and the cloud
|
|
// adapters all read a byte range).
|
|
//
|
|
// Only a SINGLE range is served. A multi-range request ("bytes=0-1,5-9") is
|
|
// answered with the whole file (null), which RFC 9110 allows and which avoids
|
|
// building a multipart body.
|
|
|
|
// -> { start, end } (inclusive), 'unsatisfiable', or null (serve the whole file).
|
|
function parseRange(header, size) {
|
|
if (typeof header !== 'string' || !Number.isSafeInteger(size) || size < 0) return null;
|
|
const match = /^\s*bytes\s*=\s*(\d*)\s*-\s*(\d*)\s*$/i.exec(header);
|
|
if (!match) return null; // not bytes, several ranges, or malformed
|
|
const [, first, last] = match;
|
|
if (first === '' && last === '') return null;
|
|
|
|
let start;
|
|
let end;
|
|
if (first === '') {
|
|
// Suffix range: the last N bytes.
|
|
const suffix = Number(last);
|
|
if (!Number.isSafeInteger(suffix)) return null;
|
|
if (suffix === 0) return 'unsatisfiable';
|
|
start = Math.max(0, size - suffix);
|
|
end = size - 1;
|
|
} else {
|
|
start = Number(first);
|
|
end = last === '' ? size - 1 : Number(last);
|
|
if (!Number.isSafeInteger(start) || !Number.isSafeInteger(end)) return null;
|
|
if (last !== '' && end < start) return null; // invalid: ignored, whole file
|
|
if (start >= size) return 'unsatisfiable';
|
|
end = Math.min(end, size - 1);
|
|
}
|
|
if (size === 0 || start >= size) return 'unsatisfiable';
|
|
return { start, end };
|
|
}
|
|
|
|
// The headers of a 206 Partial Content answer for `range` of a `size`-byte file.
|
|
function partialContentHeaders(range, size) {
|
|
return {
|
|
'Content-Range': `bytes ${range.start}-${range.end}/${size}`,
|
|
'Content-Length': String(range.end - range.start + 1),
|
|
'Accept-Ranges': 'bytes',
|
|
};
|
|
}
|
|
|
|
// An If-Range precondition holds when it names the current ETag; a date (or a
|
|
// different tag) means "send the whole file". No If-Range: the range applies.
|
|
function ifRangeAllows(ifRange, etag) {
|
|
if (ifRange === undefined || ifRange === null || ifRange === '') return true;
|
|
return String(ifRange).trim() === etag;
|
|
}
|
|
|
|
module.exports = { parseRange, partialContentHeaders, ifRangeAllows };
|