Compare commits

..
Author SHA1 Message Date
Graham Steffaniak a996b39117 keep package log 2026-11-14 20:46:09 -06:00
Graham Steffaniak 2180d6625f don't update package lock 2026-11-14 20:44:51 -06:00
Graham SteffaniakandCursor f9c31d321b refactor(i18n): sync and reorder locale keys
- Reorder sorted_keys output consistently across 26 locale files
- No source strings changed, key order only
- Splits be2b16b0d's i18n hunk; base is 493f8f741 (post-merge lockfix)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-14 19:59:34 -06:00
Graham SteffaniakandCursor 493f8f7417 fix(frontend): restore cross-platform rolldown entries in package-lock
Regenerate the lockfile from dev's baseline so npm ci installs native
@rolldown bindings on arm64 Docker builders (slim image frontend build).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-14 18:30:18 -06:00
Graham SteffaniakandCursor 449f39ddf9 Merge branch 'main' into dev/v2.2.0
Bring v2.1.5 security and bugfixes into v2.2.0 while keeping go-media,
embedded previews, and virtual-scroll frontend architecture.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-11-14 15:45:47 -06:00
Graham Steffaniak 885adcb3a2 Beta/v2.1.5 (#3389) 2026-11-13 19:32:18 -05:00
Graham Steffaniak aaa34e9bb3 Fix generation counters (#3391) 2026-11-13 18:17:06 -06:00
Graham Steffaniak 844f8f9fdd address coderabbit blocker comments (#3390) 2026-11-13 18:03:02 -06:00
Graham Steffaniak 9aba7b55ca updated dependencies (#3387) 2026-11-13 15:54:19 -06:00
Graham Steffaniak 7964b30e5d Fix checksum authenticated users on shares (#3386) 2026-11-13 15:19:53 -06:00
Graham Steffaniakandgtsteffaniak b46ffd410f media: pure-Go keyframe index, HEIC rotation, WMV/AVIF routing and a shared media cache (#3380)
Co-authored-by: gtsteffaniak <gtsteffaniak@users.noreply.github.com>
2026-11-13 14:55:17 -06:00
Graham Steffaniak 618de3fbcc Fix GHSA-823x-gh7f-grmf (#3385) 2026-11-13 14:49:39 -06:00
Graham Steffaniak a68fa0ea15 fix GHSA-jg32-ppcc-ph62 archive extraction symlink gap (#3384) 2026-11-13 14:14:10 -06:00
Graham Steffaniak 3bf3d2ce92 fix GHSA-2546-vf4j-vh9v symlink previews (#3383) 2026-11-13 14:03:44 -06:00
Graham Steffaniak f6cc16d1c8 fix subtitles GHSA-hfvf-2639-p2mq (#3382) 2026-11-13 13:45:38 -06:00
Kevin B. 49ba28208e Add media info tab, video tags and some cleanup (#3364) 2026-11-13 13:37:14 -06:00
Graham Steffaniak 8a1b735e0d add more cache invalidaton for realpath (#3381) 2026-11-13 13:00:15 -06:00
A-d-r-iandGraham Steffaniak 10a26832dd fix(i18n): improve French translations (#3356)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-11-13 11:44:14 -06:00
Kevin B. 0dfab77d6d update frontend deps v2.1.5 (#3378) 2026-11-13 11:43:57 -06:00
Vladimir 7bd9c67eb7 fix(sidebar): keep source selector stationary during horizontal tree scrolling (#3368) 2026-11-13 11:40:09 -06:00
009be1a06d fix(http): restore Unix socket reverse-proxy behavior after #3309 (#3365)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
2026-11-13 11:39:42 -06:00
danielroizen 272a0fab9d fix(preview): respect EXIF orientation on embedded JPEG thumbnails (#3358) 2026-11-13 11:39:28 -06:00
Lykinandxunguangzlj-cloud 63a517de1f fix(picker): open the parent folder on a single click (#3355)
Co-authored-by: xunguangzlj-cloud <293371022+xunguangzlj-cloud@users.noreply.github.com>
2026-11-13 11:39:04 -06:00
Lykinandxunguangzlj-cloud e39b534531 fix(users): show password-change requirement in edit preferences (#3354)
Co-authored-by: xunguangzlj-cloud <293371022+xunguangzlj-cloud@users.noreply.github.com>
2026-11-13 11:38:14 -06:00
Graham Steffaniak 5d25220e94 fix advanced user toggle duplicate (#3353) 2026-11-07 10:52:59 -06:00
8355682773 fix(upload): don't pause while the server saves, and make the stall timeout a setting (#3346)
Co-authored-by: FandacqkaCorp <275647464+FandacqkaCorp@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-11-07 10:49:55 -06:00
Graham Steffaniak 48c1ce709b updated changelog v2.1.5 (#3350) 2026-11-06 17:34:04 -06:00
Graham Steffaniak 43682dc809 fix(users): keep requested profile values when creating a user (#3349) 2026-11-06 16:09:36 -06:00
Xyleme-git 80fba42076 Update fr.json (#3343) 2026-11-06 15:51:28 -06:00
Hromad-aandClaude Opus 5.5 a01ab6901c i18n(cz): fix Czech translations (#3333)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 15:27:14 -06:00
Geekice1andGeekice1 e652538ea8 i18n: fix and improve Italian translation (it.json) (#3332)
Co-authored-by: Geekice1 <geekice1@users.noreply.github.com>
2026-11-06 15:26:49 -06:00
Peter Gao c974da7002 fix(i18n):Update Chinese translations for UI elements (#3330) 2026-11-06 15:26:37 -06:00
Konstantinos Chatzikokolakis eaee271b96 Square image previews: avoid the full-resolution crop copy in imaging.Fill (#3340) 2026-11-06 15:13:20 -06:00
HaneandClaude Opus 5.5 0d6b592046 fix(upload): serialize directory creation before uploading files (#3321)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 15:10:52 -06:00
1791bb7a36 feat(cache): add server.cacheDirMaxSizeMB to cap the preview cache size (#3314)
Co-authored-by: FandacqkaCorp <275647464+FandacqkaCorp@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-06 15:08:40 -06:00
86d4994152 fix(preview): open RAW, HEIC and rendered documents in the viewer when previews are disabled (#3313)
Co-authored-by: FandacqkaCorp <275647464+FandacqkaCorp@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-06 15:03:43 -06:00
Graham Steffaniak 8415d5f1ff Fix install button (#3312) 2026-10-31 08:27:12 -05:00
Graham Steffaniak 85f934208c replace button with link 2026-10-31 08:20:44 -05:00
1022ecafef fix(frontend): address CodeRabbit follow-ups from #3276 and #3303 (#3311)
Co-authored-by: FandacqkaCorp <275647464+FandacqkaCorp@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-31 08:17:44 -05:00
Graham Steffaniak 062ac9fb2d Beta/v2.1.4 (#3307) 2026-10-30 19:14:33 -05:00
Graham SteffaniakandGraham Steffaniak cecdd79fcc Restrict Unix socket forwarded-header trust with peer credentials (#3309)
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
2026-10-30 18:40:10 -05:00
Graham Steffaniak ac50ecfd18 fix: CodeRabbit follow-ups from #3307 (quick-win bugfixes) (#3308) 2026-10-30 17:56:50 -05:00
Graham Steffaniak 71cc2f3f1b fixed linting 2026-10-30 16:30:00 -05:00
Graham Steffaniak 1b3f47f176 fix sensitive data GHSA-892f-9h5p-mfc8 (#3306) 2026-10-30 16:25:25 -05:00
Graham Steffaniak 219b64b5ec addressed deep copy issue 2026-10-30 15:37:45 -05:00
Graham Steffaniak 3ef4928ed7 fix: prevent visible border in generated maskable PWA icons (#3300) 2026-10-30 15:33:33 -05:00
Graham Steffaniak df0a5ad10c fix sensitive data GHSA-892f-9h5p-mfc8 2026-10-30 15:22:46 -05:00
Graham Steffaniak a43d0ab2d3 updated changelog 2026-10-30 15:05:34 -05:00
Graham Steffaniak 9de09e5ec8 feat(search): sidebar File actions and close button in advanced search (#3303) 2026-10-30 15:02:51 -05:00
Graham Steffaniak 9d38c686a1 updated changelog 2026-10-30 15:01:15 -05:00
Graham Steffaniak 1cfca112e4 fix empty view after login if source named 'Files' (#3304) 2026-10-30 14:51:04 -05:00
Graham Steffaniak f540e22dc9 fix(quotas): complete delete accounting for bulk and nested resets (#3305) 2026-10-30 13:26:38 -05:00
Graham Steffaniak dade856ebe fix(move-copy): require choosing a destination when none can be inferred (advanced search) (#3276) 2026-10-30 13:19:15 -05:00
Graham SteffaniakandCursor 2f1da21253 fix(quotas): complete delete accounting for bulk and nested resets
Follow-up to #3235 / #3163: reset accounted folder quotas inside deleted
directories without index lookups, wire the same prepare/apply path into
bulk and public delete handlers, and dedupe scope quota selection.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-30 13:04:16 -05:00
Kevin B. de8d2acd43 fix empty view after login 2026-10-30 13:04:15 -05:00
Graham Steffaniak 6375be202e fix: release nested folder quotas when deleting their ancestor (#3235) 2026-10-30 12:59:12 -05:00
FandacqkaCorpandClaude Sonnet 5.5 816b99cee0 feat(search): add a close button to advanced search
On tool pages the top bar button only toggles the sidebar, so the only ways
out of advanced search were the sidebar or the browser back button.

Add an icon-only grey X at the top-left of the advanced search page (first
element of the tools wrapper); the top bar button is left as it is. It clears
the selection and goes back to the page the search was opened from, or to
/files when there is none (e.g. the URL was opened directly).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 16:14:12 +01:00
FandacqkaCorpandClaude Sonnet 5.5 bf349a7dcd feat(search): show the sidebar File actions in advanced search
The sidebar File actions button only exists in the listing, so in advanced
search the actions on the selected results (select all, download, copy, move)
were only reachable through the context menu, which is hard to find, especially
on touch devices.

Show the same sidebar button in advanced search. There it opens the context
menu on the current selection instead of the create-only menu, and it doesn't
depend on create permissions (the menu already checks permissions per
source). The menu now always offers Select multiple in advanced search, so
single results can be selected when "open with a single click" is on, and hides
the create (+) toggle, since results have no single folder to create in.

The button still honours the "hide sidebar file actions" preference and isn't
shown in shares.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 15:55:48 +01:00
Matt Van Horn db6504bf80 fix: prevent visible border in generated maskable PWA icons
Maskable icon generation now alpha-composites the resized artwork over
the opaque background using `OverlayCenter`. Added tests covering
transparent padding and opaque edges, verifying the output dimensions,
opacity, and background pixels outside the artwork.

Fixes #3187
2026-10-30 03:59:21 -07:00
FandacqkaCorpandClaude Sonnet 5.5 2019d573f8 fix(move-copy): require choosing a destination when there is no source to fall back to
In advanced search the open request is the list of results and has no source.
The Copy/Move prompt then showed "Choose a source" but, if the button was
pressed without choosing anything, silently sent the items to the root of the
first selected item's source.

Disable the Copy/Move button until a destination source is chosen in that
case (no destination yet and no source in the open request, outside shares),
and use the existing "Choose a source" text as its tooltip. Normal listings
and shares are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 11:05:37 +01:00
Matt Van Horn d716fc15ec test: share the nested quota delete fixture between API and WebDAV tests
TestResourceDeleteNestedQuota and TestWebDAVDeleteNestedQuota seeded the
same files, quotas and usage deltas. Move that setup into
seedNestedQuotaFixture and call it from both.
2026-10-30 00:28:00 -07:00
Graham Steffaniak 56e16608e4 fix(frontend): keep video loading spinner off when the fetch stalls mid-playback (#3289) 2026-10-29 19:19:47 -05:00
Graham Steffaniak c471fc135b fix(frontend): use native video fullscreen on iPhone (#3288) 2026-10-29 19:18:33 -05:00
Graham Steffaniak 5112cf87b1 fix(analytics): report freebsd instead of "other" in system.os (#3284) 2026-10-29 19:17:23 -05:00
Graham Steffaniak 3d7bac47ee fix(stream): play single-file share media on Windows (#3271) 2026-10-29 19:10:22 -05:00
vulragrag-star 2a38d2fa1e fix(users): enforce lockPassword on self-service password changes (#3283) 2026-10-29 19:01:34 -05:00
Matt Van Horn 82e110c08d test: cover scoped users in the API nested quota delete test
The API path is scope-relative while quota paths and the index are
source-relative. Run the test for a user scoped to /scoped as well as /,
which fails if the delete looks up the unscoped path.
2026-10-29 08:14:00 -07:00
Matt Van Horn f4c96078a1 Merge remote-tracking branch 'upstream/main' into fix/3163-release-nested-folder-quotas-on-delete
# Conflicts:
#	backend/internal/web/resource.go
#	backend/internal/web/webdav.go
#	backend/internal/web/webdav_test.go
2026-10-29 08:14:00 -07:00
975adedabb Address directory indexing bugs (#3292)
Co-authored-by: Lucas Siqueira <lucas.morais.siqueira@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-28 10:29:42 -05:00
Graham Steffaniak 14ec0eba0d Fix indexing regressions (#3260) 2026-10-27 19:41:12 -05:00
Lucas de Morais SiqueiraandGraham Steffaniak 307d5f6591 fix(indexing): skip unreadable directory entries instead of failing the listing (#3184)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-27 19:25:01 -05:00
Get The Fork Off ff9df43356 fix(frontend): keep video loading spinner off when the fetch stalls mid-playback 2026-10-27 20:50:30 -03:00
Get The Fork Off 802e91aa05 fix(frontend): use native video fullscreen on iPhone 2026-10-27 20:48:46 -03:00
NefelibateLGandZCode Scout 28ef44cdec fix(analytics): report freebsd instead of "other" in system.os
The analytics snapshot classified every GOOS outside {linux, windows,
darwin} as "other", so the shipped freebsd-amd64 release binary
reported "os": "other" in the analytics payload even though
backend/.goreleaser.yaml builds freebsd as a first-class target
(#3244).

Add freebsd to the mapSystemOS whitelist, matching the release build
matrix, and add a table-driven test covering all four shipped GOOS
values plus the unsupported and empty fallbacks.

Fixes #3244

Co-authored-by: ZCode Scout <noreply@zcode.local>
Signed-off-by: NefelibateLG <3453510037@qq.com>
2026-10-27 09:16:06 +08:00
sarankumar1325 0ee0f35301 fix(stream): play single-file share media on Windows
shareRelativeDisplayName ran filepath.Base on slash-separated paths. On
Windows filepath.Base("/") returns "\", which passed the "/" check, so a
single-file share streamed with file=/ got "\" as its name instead of the
shared file's name. IsMediaStreamFile then rejected it and the stream
endpoint answered 403 "stream endpoint supports audio and video only".

Use path.Base, since both the share-relative path and the share path are
slash-separated.

Fixes #3270
2026-10-25 11:30:00 +05:30
Graham Steffaniak cc99aa542e updated with symlink extraction change (#3265) 2026-10-23 15:36:09 -05:00
8514da4a9a fix(auth): API token permission handling for customized tokens (#3255)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
2026-10-23 14:48:18 -05:00
Graham SteffaniakandDevin 4625b89528 fix webp preview support (#3259)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 15:34:57 -05:00
Jay Hemnani b2a07196b7 fix(archive): extract tar hard links instead of dropping them (#3234) 2026-10-22 15:26:16 -05:00
Graham Steffaniak 0a8698ee4d updated changelog 2026-10-22 15:24:08 -05:00
ec8ad36c95 Feature/html preview navigation and media (#3131)
Co-authored-by: Graham Steffaniak <graham.steffaniak@autodesk.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-22 15:13:31 -05:00
vulragrag-star 2c0c1b4bb2 fix(frontend): show the share banner to public visitors (#3250) 2026-10-22 13:38:49 -05:00
Thorsten Liborius 6fa71bd13d fix(frontend): show correct relative expiration time for sharesFix/share expiry relative time (#3247) 2026-10-21 18:47:44 -05:00
Graham Steffaniakandcoderabbitai[bot] 06aa16e7d7 fix raw embedded image support - rtf rw2 raw rwl (#3243)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-10-20 17:43:38 -05:00
Graham Steffaniak b0f6d53f49 fix arm32 build in v2.1.3 (#3242) 2026-10-20 14:57:09 -05:00
Graham Steffaniak d27ab6a00b fix arm32 build in v2.1.3 2026-10-20 14:56:36 -05:00
Graham Steffaniak f0fb859479 updated package dependencies and replaced katex with temml (#3241) 2026-10-20 13:51:58 -05:00
Graham Steffaniak 4c8a913377 updated backage dependencies and replaced katex with temml 2026-10-20 13:31:43 -05:00
Graham Steffaniak c1684baffd Beta/v2.1.3 (#3238) 2026-10-20 12:47:40 -05:00
Matt Van Horn bb45ebcce1 fix: avoid shadowing err in nested quota delete tests 2026-10-20 10:41:27 -07:00
18138b1ab5 Fix CodeRabbit findings from PR #3238 (#3240)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
2026-10-20 12:31:48 -05:00
Matt Van Horn 26665efc13 fix: release nested folder quotas when deleting their ancestor
Adds shared preparation and application helpers for API and WebDAV
deletion accounting. Before deletion removes index metadata, each nested
accounted quota is measured independently, while covering, scope, and
share quotas use the deleted item's size. Adjustments are applied only
after deletion succeeds, and unknown sizes are omitted. API deletion
also derives the source index path from the item's real path.

Fixes #3163
2026-10-20 04:39:51 -07:00
Graham SteffaniakandDevin 98851e029f fix(access): match lowercase rule keys in path existence and rule updates (#3237)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-19 13:50:08 -05:00
4a1a3bbe09 fix(access): lowercase access-rule keys with migration (GHSA-4vrc-935w-xxq2) (#3232)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-10-19 13:50:08 -05:00
Graham Steffaniak a6d7f7c64c fix(auth): password-gate passkey enrollment changes (GHSA-r744-4vm3-2px3) (#3198) 2026-10-19 13:50:08 -05:00
Graham Steffaniak ada7a18620 Use statx reduce redundant calls (#3230) 2026-10-19 13:50:08 -05:00
Graham Steffaniak 4f4ea2370e updated linting and tests on macos (#3231) 2026-10-19 13:50:08 -05:00
Graham Steffaniak 52aa314e1d fix(quota): count upload usage against accounted limits (GHSA-m57f-66qj-8jxg) (#3201) 2026-10-19 13:50:08 -05:00
Kevin B.andGraham Steffaniak 1d6f67cc20 update frontend deps (v2.1.3) (#3228)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-19 13:50:08 -05:00
Graham Steffaniak d89b7e7deb fix(events): hold client locks during SSE sends (GHSA-p9wm-f5jj-75pj) (#3227) 2026-10-19 12:37:04 -05:00
Graham Steffaniak e26bd29848 fix(archive): enforce access on unarchive extraction (GHSA-p93p-v944-4p44) (#3203) 2026-10-19 12:37:04 -05:00
Graham Steffaniak 9c2b98655a fix: enforce readOnly on WebDAV, archive and OnlyOffice (GHSA-4v62-884x-2466) (#3202) 2026-10-19 12:37:04 -05:00
Paolo VallettaandGraham Steffaniak a379f047d1 fix(indexing): resume the persisted scan schedule on startup (#3225)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-19 12:37:04 -05:00
Kevin B.andGraham Steffaniak f4879be6bf fix path picker list on shares (#3223)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-19 12:23:45 -05:00
Kevin B.andGraham Steffaniak 7751da745d Hide activity, quota, access buttons in Info prompt in shares (#3224)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-19 12:23:17 -05:00
Graham Steffaniak 80cedcb4d5 fix(frontend): clear stale upload navigation approval when idle (#3222) 2026-10-18 13:48:38 -05:00
Graham Steffaniak ac19abf7d2 fix(frontend): restore upload guards while uploads are paused (#3220) 2026-10-18 13:48:38 -05:00
Graham Steffaniak 38e868e5af fix(frontend): resolve /files/ to default source for single-source installs (#3215) 2026-10-18 13:48:38 -05:00
anonymous-writerandGraham Steffaniak b3af5cc2d7 Fix upload session not released after successful single-file upload (#3186)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-18 13:48:38 -05:00
Paolo VallettaandGraham Steffaniak b399de3d92 fix(webdav): free the deleted bytes from the quota counters (#3160)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-18 13:45:50 -05:00
Graham Steffaniak 99f1a14e7b fix(shares): correct share path updates on folder move (GHSA-26fh-r34c-qxh6) (#3197) 2026-10-18 13:07:33 -05:00
Graham Steffaniak 84afd8b0b7 fix(webdav): require Create permission for new PUT files (GHSA-8655-2ph3-q7q7) (#3196) 2026-10-18 13:07:33 -05:00
Graham Steffaniak 78be5d8c0f fix(preview): enforce scoped symlink resolution (GHSA-g2q5-r6ch-v25f) (#3195) 2026-10-18 13:07:33 -05:00
Graham Steffaniak 5e29e1ee60 fix(auth): use rightmost X-Forwarded-For for client IP (GHSA-wjp8-g49g-xrhx) (#3191) 2026-10-18 13:07:33 -05:00
Graham Steffaniak 367a7f019c fix(tools): per-user duplicate finder cache (GHSA-4fhx-8pf6-5gmr) (#3190) 2026-10-18 13:07:33 -05:00
Graham Steffaniak fa9c094160 fix(view): block share view tokens on authenticated view route (GHSA-wxj5-g3qp-g732) (#3194) 2026-10-18 13:07:33 -05:00
Graham Steffaniak 36a9a6dc49 fix(events): prevent SSE EventRouter concurrent map panic (GHSA-p9wm-f5jj-75pj) (#3189) 2026-10-18 13:07:33 -05:00
Kevin B.andGraham Steffaniak 8fe6f1d5c3 Refactor Tooltip and generic to avoid v-html usage (GHSA-rmjp-pfrq-cc3v) (#3199)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-18 13:07:33 -05:00
JUSHUANGHUI LIandGraham Steffaniak 6dc3935c57 fix: preserve file permissions when saving edits (#3159)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-17 11:28:25 -05:00
Graham Steffaniak 5e73036657 fix(users): partial global permissions on API create (on top of #3171) (#3165) 2026-10-17 11:24:46 -05:00
Jay HemnaniandGraham Steffaniak e505900975 fix(activity): record uploads and deletes made through a share (#3166)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-17 11:24:46 -05:00
HawkForgeandGraham Steffaniak 26f320ee43 fix(settings): don't panic when config has no server map (#3176)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-16 16:23:41 -05:00
Kevin B. 184ae798c4 make listing highlight follow nextPrevious (#3156) 2026-10-15 15:26:59 -05:00
Graham Steffaniak 7ae8bab062 fix scope issues for archive extraction (#3177) 2026-10-15 15:26:59 -05:00
Graham Steffaniak 1af606e751 fix dev script 2026-10-15 15:26:59 -05:00
Graham Steffaniak 3683bc607c fixed markdown screenshots (#3171) 2026-10-15 15:26:59 -05:00
Som SamantrayandGraham Steffaniak 08d3e43c22 fix(auth): preserve admin source scope with createUserDir (#3125)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-15 15:26:59 -05:00
Kevin B.andGraham Steffaniak d76f9373c4 Fix missing creation time and tiny font on md viewer code blocks (#3157)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-15 15:00:55 -05:00
Kevin B.andGraham Steffaniak a72651e7aa Add alphabetical order + pinned indicators to file tree (#3154)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-15 15:00:28 -05:00
Kevin B.andGraham Steffaniak 73d0b20821 Fix: vite warnings and restricted origin (#3155)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-15 15:00:00 -05:00
Anatolii PrylutskyiandGraham Steffaniak 3e447307f7 fix(frontend): hide write actions in read-only sources (#3149)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-14 18:50:09 -05:00
vulragrag-starandGraham Steffaniak 264cc81c41 fix(share): honor userDefaults singleClick for anonymous shares (#3147)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-14 18:46:13 -05:00
Graham Steffaniak aa5b24119a updated changelog 2026-10-14 16:07:31 -05:00
Graham Steffaniak 52cf624711 add file modify logging (#3152) 2026-10-14 16:07:31 -05:00
Graham Steffaniak 5753f63970 updated webdav hardening (#3151) 2026-10-14 16:07:31 -05:00
Jay HemnaniandGraham Steffaniak 8f1cf54cb0 fix(users): keep requested permissions when creating a user (#3130)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-14 16:07:31 -05:00
Paolo VallettaandGraham Steffaniak a8cfcdd6eb feat(webdav): chunked uploads for large files (rclone --owncloud-chunk-size) (#3112)
Co-authored-by: Graham Steffaniak <42989099+gtsteffaniak@users.noreply.github.com>
2026-10-14 16:05:49 -05:00
161 changed files with 22475 additions and 18288 deletions

No files matched your search

+5
View File
@@ -12,6 +12,11 @@ All notable changes to this project will be documented in this file. For commit
- Transcoding for media playback with quality presets -- requires ffmpeg.
- Hardware Acceleration device selector (Software / detected GPU devices)
- More operations use native api operations rather than ffmpeg, improving performance in many cases. Added temporary config option to force ffmpeg to be used
- HLS keyframe indexes for MP4/MOV/M4V/MKV/WebM are read from the container index with go-media instead of demuxing every packet with ffprobe; ffprobe still runs when go-media cannot read the file or reports a keyframe gap that would rule out stream copy.
- HEIC/AVIF rotation (irot) for the ffmpeg decode fallback is read with go-media instead of a full `ffmpeg -f null` decode.
- AVIF images get previews through the HEIC preview path (go-media decode, ffmpeg fallback) when HEIC previews are enabled.
- WMV/ASF video thumbnails use go-media for WMV3; WMV1/WMV2 still use ffmpeg.
- ffmpeg and go-media share one media metadata cache keyed by path, size and modification time, so a probe, duration or subtitle result from either backend is reused by the other.
- Document enhancements
- Official Collabora support using WOPI config.
- Document previews use a pure-go in-process package (all versions get it now).
+31 -18
View File
@@ -1,20 +1,11 @@
# syntax=docker/dockerfile:1.4
# GPU-capable media image: Debian 13 Trixie GPU FFmpeg as runtime base
# (x86_64 only). Track G in gtsteffaniak/ffmpeg (docker/dockerfile.gpu,
# ARG DEBIAN_VERSION=trixie): mostly-static codecs + dynamic
# VA/Vulkan/OpenCL/NVIDIA userspace. Pi 4 (V4L2) stays on the static
# Alpine image (`Dockerfile`, not this file).
# NOTE: the golang:bookworm stage below is build-time only — the Go
# binary is built with CGO_ENABLED=0 (static), so it runs unchanged
# on the Trixie runtime. No structural change needed for the Trixie
# retarget of the FFmpeg GPU image (PR gtsteffaniak/ffmpeg#49).
# GPU-capable media image: mostly-static glibc FFmpeg + dynamic VA/Vulkan/OpenCL drivers (amd64).
# Codecs/TLS are linked into gtstef/ffmpeg:<ver> (full track, not *-static).
#
# Build after the ffmpeg GPU image is published and smoke-tested:
# docker build -f _docker/Dockerfile.media --build-arg VERSION=... --build-arg REVISION=... .
# Runtime flags: Intel/AMD need /dev/dri (see _docker/docker-compose.media.yaml);
# NVIDIA needs the host toolkit (`--gpus all` / `runtime: nvidia`).
ARG FFMPEG_GPU_IMAGE=gtstef/ffmpeg:9.0.2-gpu
# amd64: libva/Mesa/Vulkan/OpenCL userspace below; NVIDIA libs come from the host toolkit.
# arm64 (e.g. Raspberry Pi): h264_v4l2m2m uses kernel V4L2 only — no extra GPU .debs; pass
# /dev/video10 or /dev/video11 (and group video). Pi 5 VAAPI via /dev/dri is optional (mesa on arm64).
FROM gtstef/ffmpeg:9.0.2 AS ffmpeg
FROM golang:bookworm AS base
ARG VERSION
@@ -38,14 +29,36 @@ COPY ./frontend/ ./frontend/
WORKDIR /build/frontend
RUN npm run build:docker
FROM ${FFMPEG_GPU_IMAGE} AS runtime
FROM debian:bookworm-slim AS runtime
ENV DEBIAN_FRONTEND=noninteractive
ENV FILEBROWSER_FFMPEG_PATH="/usr/local/bin"
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
curl \
mailcap \
tzdata \
&& rm -rf /var/lib/apt/lists/* \
&& (dpkg --print-architecture | grep -q amd64 \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
libva2 \
libva-drm2 \
libdrm2 \
libvulkan1 \
mesa-va-drivers \
mesa-vulkan-drivers \
ocl-icd-libopencl1 \
intel-media-va-driver \
libmfx1 \
&& rm -rf /var/lib/apt/lists/* \
|| true)
COPY --from=ffmpeg [ "/ffmpeg", "/ffprobe", "/usr/local/bin/" ]
ENV FILEBROWSER_FFMPEG_PATH="/usr/local/bin/"
ENV FILEBROWSER_DATABASE_PATH="/home/filebrowser/data/database.sqlite"
ENV FILEBROWSER_CONFIG="/home/filebrowser/data/config.yaml"
ENV PATH="$PATH:/home/filebrowser"
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics
ENV NVIDIA_DRIVER_CAPABILITIES=compute,video,utility,graphics
RUN useradd -m -u 1000 -s /usr/sbin/nologin filebrowser
USER filebrowser
-38
View File
@@ -1,38 +0,0 @@
# Filebrowser media image (GPU)
`Dockerfile.media` builds a media image on top of the Debian 13 Trixie GPU
FFmpeg image (`gtstef/ffmpeg:<version>-gpu`, Track G in
`gtsteffaniak/ffmpeg`, `ARG DEBIAN_VERSION=trixie`). The
production Pi 4 setup stays on the static Alpine `Dockerfile` (V4L2).
## Prereq (sequencing)
The media image is built only **after** the FFmpeg GPU image is published
(`gtstef/ffmpeg:<version>-gpu`) and smoke-tested. The `FFMPEG_GPU_IMAGE`
build-arg pins the exact published tag; if the tag is not smoke-tested, do
not merge the media PR.
## Build
```bash
docker build -f _docker/Dockerfile.media \
--build-arg FFMPEG_GPU_IMAGE=gtstef/ffmpeg:9.0.2-gpu \
--build-arg VERSION=latest --build-arg REVISION=$(git rev-parse HEAD) \
-t filebrowser-media:latest .
```
## Smoke
```bash
docker run --rm filebrowser-media:latest ffmpeg -hwaccels | grep -E "vaapi|cuda|vulkan|opencl"
docker run --rm --device=/dev/dri/renderD128 filebrowser-media:latest \
-vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc=size=640x480:rate=1 \
-c:v h264_vaapi -t 1 -f null -
docker run --rm --gpus all filebrowser-media:latest \
-f lavfi -i testsrc=size=640x480:rate=1 -c:v h264_nvenc -t 1 -f null -
```
## Compose
See `docker-compose.media.yaml` (intel/AMD and NVIDIA snippets; Pi stays
static).
-67
View File
@@ -1,67 +0,0 @@
# Compose snippets for the GPU media image (`_docker/Dockerfile.media`).
#
# filebrowser-media runs on the Debian GPU FFmpeg image
# (gtstef/ffmpeg:<version>-gpu). Pi 4 production stays on the static
# Alpine `Dockerfile` (V4L2). Copy the service you need into your
# docker-compose.yaml.
#
# docker compose -f _docker/docker-compose.media.yaml run --rm filebrowser-media \
# -f lavfi -i testsrc=size=640x480:rate=1 -c:v h264_vulkan -t 1 -f null -
# (replace service flags with your image name)
services:
# Intel / AMD VAAPI (needs host /dev/dri)
# Find render group GID: getent group render
filebrowser-media-intel:
image: filebrowser-media:latest
build:
context: ../
dockerfile: ./_docker/Dockerfile.media
args:
VERSION: ${VERSION:-latest}
REVISION: ${REVISION:-local}
FFMPEG_GPU_IMAGE: ${FFMPEG_GPU_IMAGE:-gtstef/ffmpeg:9.0.2-gpu}
devices:
- /dev/dri:/dev/dri
group_add:
- "993" # render group GID placeholder — set to `getent group render`
environment:
# fallback when /dev/dri is not bind-mounted: LIBVA_DRIVER_NAME=radeonsi
LIBVA_DRIVER_NAME: ${LIBVA_DRIVER_NAME:-}
ports:
- "80:80"
# NVIDIA NVENC/NVDEC (needs NVIDIA driver + nvidia-container-toolkit on host)
# `runtime: nvidia` or `deploy.resources.reservations.devices` + NVIDIA_* env.
filebrowser-media-nvidia:
image: filebrowser-media:latest
build:
context: ../
dockerfile: ./_docker/Dockerfile.media
args:
VERSION: ${VERSION:-latest}
REVISION: ${REVISION:-local}
FFMPEG_GPU_IMAGE: ${FFMPEG_GPU_IMAGE:-gtstef/ffmpeg:9.0.2-gpu}
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
environment:
NVIDIA_VISIBLE_DEVICES: all
NVIDIA_DRIVER_CAPABILITIES: compute,utility,video,graphics
# runtime: nvidia (legacy alternative to deploy above)
ports:
- "80:80"
# Pi 4 V4L2 — NOT the GPU image. Production Pi stays on the static Alpine
# Dockerfile (h264_v4l2m2m via kernel V4L2, /dev/video10 or /dev/video11):
filebrowser-pi-static:
image: filebrowser-static:latest # placeholder for the existing Alpine build
devices:
- /dev/video10:/dev/video10
- /dev/video11:/dev/video11
ports:
- "80:80"
+1 -1
View File
@@ -22,7 +22,7 @@ require (
github.com/gtsteffaniak/go-cache v1.1.0
github.com/gtsteffaniak/go-ffmpeg v1.6.2
github.com/gtsteffaniak/go-logger v1.1.0
github.com/gtsteffaniak/go-media v1.2.2
github.com/gtsteffaniak/go-media v1.3.0
github.com/kovidgoyal/imaging v1.9.0
github.com/pquerna/otp v1.5.0
github.com/spf13/afero v1.15.0
+2 -2
View File
@@ -165,8 +165,8 @@ github.com/gtsteffaniak/go-ffmpeg v1.6.2 h1:gtvBg3G4PHO1Z/ZIPEK/XP4XT/g+l9S0Rpmm
github.com/gtsteffaniak/go-ffmpeg v1.6.2/go.mod h1:FefhoD+KZzHWqBaUpntcKp6lXtUzdFW00F74axnENUQ=
github.com/gtsteffaniak/go-logger v1.1.0 h1:baFds+iibuLLNpNLwygz32bDqdsunI6dGWcdxwMIVKI=
github.com/gtsteffaniak/go-logger v1.1.0/go.mod h1:rapapTju4D7bwt6h4JOdZNUXDjnp0WkTBd6TN5yWglI=
github.com/gtsteffaniak/go-media v1.2.2 h1:CgdXyqiZGa81vZUGL1AXGxQe+8owkUkX6roNACGQJ0w=
github.com/gtsteffaniak/go-media v1.2.2/go.mod h1:x8k0z759GiawTtKBTXDNXF/pT1UQDhR4e70/eypXpOk=
github.com/gtsteffaniak/go-media v1.3.0 h1:EOxoaOQq05V3OUPo5NndnPZ0/VYt4gFEwa6l7MdR6rk=
github.com/gtsteffaniak/go-media v1.3.0/go.mod h1:x8k0z759GiawTtKBTXDNXF/pT1UQDhR4e70/eypXpOk=
github.com/gtsteffaniak/go-vpx v1.0.1 h1:9lSI8yoz6omDChaeV0AJrhdfe2YTrWvJrDENI/bnkAM=
github.com/gtsteffaniak/go-vpx v1.0.1/go.mod h1:tRCdDZ15oZZTsiwmQAwyH56PTt1jn1C1O0P03oOkoJ8=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
@@ -9,7 +9,10 @@ import (
"io"
"os"
"path/filepath"
"sort"
"sync"
"sync/atomic"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/gtsteffaniak/go-logger/logger"
@@ -21,9 +24,28 @@ type Cache interface {
Set(key string, value []byte) error
}
const (
// evictTargetPercent is the size, as a percentage of the limit, that eviction shrinks the cache to.
// Stopping below the limit avoids running an eviction on every write once the cache is full.
evictTargetPercent = 90
// touchInterval is how old a file's modification time has to be before a read refreshes it. Reads
// refresh it so the files in use are the last ones evicted, but not on every read.
touchInterval = time.Hour
)
// FileCache struct for file-based caching
type FileCache struct {
dir string
// maxBytes is the size limit of the cache in bytes. 0 means no limit.
maxBytes int64
// size is the total size of the cached files in bytes. Only tracked when maxBytes > 0.
size atomic.Int64
// sizeMu keeps size consistent with the files on disk: stores and deletes hold it for reading while
// they change a file and its size, and eviction holds it for writing while it measures the directory,
// so a store can never fall between a measurement and the size being set from it.
sizeMu sync.RWMutex
// evicting is set while an eviction is running so only one runs at a time.
evicting atomic.Bool
// granular locks
scopedLocks struct {
sync.Mutex
@@ -33,6 +55,12 @@ type FileCache struct {
}
func NewFileCache(dir string) (*FileCache, error) {
return NewFileCacheWithLimit(dir, 0)
}
// NewFileCacheWithLimit is like NewFileCache but keeps the cache under maxBytes, deleting the least
// recently used files when it grows past it. 0 means no limit.
func NewFileCacheWithLimit(dir string, maxBytes int64) (*FileCache, error) {
cacheDir := filepath.Join(dir, "diskcache")
// Migrate existing cache files from old structure (if any)
@@ -44,7 +72,13 @@ func NewFileCache(dir string) (*FileCache, error) {
if err := os.MkdirAll(cacheDir, fileutils.PermDir); err != nil {
return nil, fmt.Errorf("failed to create cache directory: %w", err)
}
return &FileCache{dir: cacheDir}, nil
cache := &FileCache{dir: cacheDir}
if maxBytes > 0 {
cache.maxBytes = maxBytes
// measures the cache and trims it right away if it is already over the limit
cache.evictPass()
}
return cache, nil
}
// migrateOldCacheStructure moves cache files from old structure (dir/a/...) to new structure (dir/diskcache/a/...)
@@ -99,19 +133,50 @@ func (f *FileCache) Store(ctx context.Context, key string, value []byte) error {
return err
}
if f.maxBytes > 0 {
f.sizeMu.RLock()
defer f.sizeMu.RUnlock()
}
previousSize := f.fileSize(fileName)
if err := os.WriteFile(fileName, value, fileutils.PermFile); err != nil {
// a failed write can leave a truncated or partial file, so count what is actually there
f.sizeChanged(f.fileSize(fileName) - previousSize)
return err
}
f.sizeChanged(int64(len(value)) - previousSize)
return nil
}
// fileSize returns the size of a cached file, or 0 if it doesn't exist or the size isn't tracked.
func (f *FileCache) fileSize(fileName string) int64 {
if f.maxBytes <= 0 {
return 0
}
info, err := os.Stat(fileName)
if err != nil {
return 0
}
return info.Size()
}
// sizeChanged adds delta to the tracked size and starts an eviction if that put the cache over its limit.
func (f *FileCache) sizeChanged(delta int64) {
if f.maxBytes <= 0 {
return
}
if f.size.Add(delta) > f.maxBytes {
f.evictAsync()
}
}
func (f *FileCache) Load(ctx context.Context, key string) (value []byte, exist bool, err error) {
r, ok, err := f.open(key)
if err != nil || !ok {
return nil, ok, err
}
defer r.Close()
f.touch(r)
value, err = io.ReadAll(r)
if err != nil {
@@ -125,13 +190,119 @@ func (f *FileCache) Delete(ctx context.Context, key string) error {
mu.Lock()
defer mu.Unlock()
if f.maxBytes > 0 {
f.sizeMu.RLock()
defer f.sizeMu.RUnlock()
}
fileName := f.getFileName(key)
removedSize := f.fileSize(fileName)
if err := os.Remove(fileName); err != nil && !errors.Is(err, os.ErrNotExist) {
return err
}
if removedSize > 0 {
f.size.Add(-removedSize)
}
return nil
}
// touch refreshes the modification time of a file that was just read, which is what eviction orders by.
func (f *FileCache) touch(file *os.File) {
if f.maxBytes <= 0 {
return
}
info, err := file.Stat()
if err != nil || time.Since(info.ModTime()) < touchInterval {
return
}
now := time.Now()
_ = os.Chtimes(file.Name(), now, now)
}
type cachedFile struct {
path string
size int64
modTime time.Time
}
// scan lists the cached files and their total size.
func (f *FileCache) scan() ([]cachedFile, int64) {
var files []cachedFile
var total int64
_ = filepath.WalkDir(f.dir, func(path string, entry os.DirEntry, err error) error {
if err != nil || entry.IsDir() {
return nil
}
info, err := entry.Info()
if err != nil || !info.Mode().IsRegular() {
return nil
}
files = append(files, cachedFile{path: path, size: info.Size(), modTime: info.ModTime()})
total += info.Size()
return nil
})
return files, total
}
// evictAsync starts an eviction in the background unless one is already running, so the request that
// stored the file doesn't wait for it.
func (f *FileCache) evictAsync() {
if !f.evicting.CompareAndSwap(false, true) {
return
}
go func() {
for {
removed := f.evictPass()
f.evicting.Store(false)
if removed == 0 {
return // under the limit, or nothing could be removed: don't spin
}
// Files stored while the pass was running were counted but not necessarily seen by it, and a
// store that came in just before the flag was cleared saw it set and didn't start another
// pass, so check again before stopping.
if f.size.Load() <= f.maxBytes || !f.evicting.CompareAndSwap(false, true) {
return
}
}
}()
}
// evictPass measures the cache and, if it is over the limit, deletes the least recently used files until
// it is under evictTargetPercent of the limit. Deleting a file that is being read or written is harmless:
// readers keep their open file and a removed entry is simply rebuilt the next time it is requested.
func (f *FileCache) evictPass() (removed int) {
f.sizeMu.Lock()
files, total := f.scan()
f.size.Store(total)
f.sizeMu.Unlock()
if total <= f.maxBytes {
return 0
}
target := f.maxBytes / 100 * evictTargetPercent
sort.Slice(files, func(i, j int) bool { return files[i].modTime.Before(files[j].modTime) })
var removedBytes int64
for _, file := range files {
if f.size.Load() <= target {
break
}
// Hold the lock so no store or delete changes this file between measuring it and removing it, and
// subtract the size it has now: the listing is from the scan and the file may have been rewritten since.
f.sizeMu.Lock()
info, err := os.Stat(file.path)
if err == nil {
err = os.Remove(file.path)
}
if err == nil {
f.size.Add(-info.Size())
removed++
removedBytes += info.Size()
}
f.sizeMu.Unlock()
}
logger.Debugf("preview cache limit: removed %d files (%d MB), now %d MB of %d MB", removed, removedBytes>>20, f.size.Load()>>20, f.maxBytes>>20)
return removed
}
func (f *FileCache) open(key string) (*os.File, bool, error) {
fileName := f.getFileName(key)
file, err := os.Open(fileName)
@@ -2,9 +2,12 @@ package diskcache
import (
"context"
"fmt"
"os"
"path/filepath"
"sync"
"testing"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/stretchr/testify/require"
@@ -70,3 +73,168 @@ func fileExists(filename string) bool {
}
return !info.IsDir()
}
// storeAged stores a value and sets its modification time, so the eviction order is deterministic.
func storeAged(t *testing.T, cache *FileCache, key string, size int, age time.Duration) {
t.Helper()
require.NoError(t, cache.Store(context.Background(), key, make([]byte, size)))
when := time.Now().Add(-age)
require.NoError(t, os.Chtimes(cache.getFileName(key), when, when))
}
func waitFor(t *testing.T, condition func() bool) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for !condition() {
if time.Now().After(deadline) {
t.Fatal("condition not met in time")
}
time.Sleep(10 * time.Millisecond)
}
}
func newLimitedCache(t *testing.T, maxBytes int64) *FileCache {
t.Helper()
fileutils.SetFsPermissions(0644, 0755)
cache, err := NewFileCacheWithLimit(t.TempDir(), maxBytes)
require.NoError(t, err)
return cache
}
func TestFileCacheEvictsLeastRecentlyUsedOverLimit(t *testing.T) {
const kb = 1024
cache := newLimitedCache(t, 10*kb)
// 3 files of 3KB are under the 10KB limit
storeAged(t, cache, "oldest", 3*kb, 3*time.Hour)
storeAged(t, cache, "old", 3*kb, 2*time.Hour)
storeAged(t, cache, "recent", 3*kb, time.Hour)
require.True(t, fileExists(cache.getFileName("oldest")), "nothing is removed under the limit")
// the 4th file goes over the limit: the oldest ones are removed until it is under 90% of it
require.NoError(t, cache.Store(context.Background(), "newest", make([]byte, 3*kb)))
waitFor(t, func() bool { return !fileExists(cache.getFileName("oldest")) })
waitFor(t, func() bool { return !cache.evicting.Load() })
require.False(t, fileExists(cache.getFileName("oldest")))
require.False(t, fileExists(cache.getFileName("old")))
require.True(t, fileExists(cache.getFileName("recent")))
require.True(t, fileExists(cache.getFileName("newest")))
require.LessOrEqual(t, cache.size.Load(), int64(9*kb))
}
func TestFileCacheTrimsAnOversizedCacheAtStartup(t *testing.T) {
const kb = 1024
fileutils.SetFsPermissions(0644, 0755)
dir := t.TempDir()
unlimited, err := NewFileCache(dir)
require.NoError(t, err)
storeAged(t, &FileCache{dir: unlimited.dir}, "a", 4*kb, 4*time.Hour)
storeAged(t, &FileCache{dir: unlimited.dir}, "b", 4*kb, 3*time.Hour)
storeAged(t, &FileCache{dir: unlimited.dir}, "c", 4*kb, 2*time.Hour)
cache, err := NewFileCacheWithLimit(dir, 10*kb)
require.NoError(t, err)
// 12KB over a 10KB limit: only the oldest file is removed, which already brings it under 90% (9KB)
require.False(t, fileExists(cache.getFileName("a")))
require.True(t, fileExists(cache.getFileName("b")))
require.True(t, fileExists(cache.getFileName("c")))
require.Equal(t, int64(8*kb), cache.size.Load())
}
func TestFileCacheWithoutLimitKeepsEverything(t *testing.T) {
fileutils.SetFsPermissions(0644, 0755)
cache, err := NewFileCache(t.TempDir())
require.NoError(t, err)
for i := 0; i < 5; i++ {
storeAged(t, cache, fmt.Sprintf("key-%d", i), 1<<20, time.Duration(i)*time.Hour)
}
for i := 0; i < 5; i++ {
require.True(t, fileExists(cache.getFileName(fmt.Sprintf("key-%d", i))))
}
require.Zero(t, cache.size.Load(), "size is only tracked when there is a limit")
}
func TestFileCacheReadRefreshesModTime(t *testing.T) {
cache := newLimitedCache(t, 1<<20)
storeAged(t, cache, "used", 100, 3*time.Hour)
_, ok, err := cache.Load(context.Background(), "used")
require.NoError(t, err)
require.True(t, ok)
info, err := os.Stat(cache.getFileName("used"))
require.NoError(t, err)
require.WithinDuration(t, time.Now(), info.ModTime(), time.Minute, "a file that is read is the last to be evicted")
}
func TestFileCacheDeleteAndOverwriteKeepTheSizeAccurate(t *testing.T) {
const kb = 1024
cache := newLimitedCache(t, 1<<20)
ctx := context.Background()
require.NoError(t, cache.Store(ctx, "k", make([]byte, 5*kb)))
require.Equal(t, int64(5*kb), cache.size.Load())
require.NoError(t, cache.Store(ctx, "k", make([]byte, 2*kb)))
require.Equal(t, int64(2*kb), cache.size.Load(), "overwriting replaces the old size")
require.NoError(t, cache.Delete(ctx, "k"))
require.Zero(t, cache.size.Load())
}
// Stores that land while an eviction is measuring or deleting must not be lost from the tracked size,
// and the cache must end up under its limit even though more files kept arriving during the eviction.
func TestFileCacheStaysUnderTheLimitWithConcurrentStores(t *testing.T) {
const (
kb = 1024
limit = 100 * kb
workers = 16
stores = 40
)
cache := newLimitedCache(t, limit)
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func(w int) {
defer wg.Done()
for i := 0; i < stores; i++ {
_ = cache.Store(context.Background(), fmt.Sprintf("worker-%d-key-%d", w, i), make([]byte, 4*kb))
}
}(w)
}
wg.Wait()
waitFor(t, func() bool { return !cache.evicting.Load() })
_, onDisk := cache.scan()
require.LessOrEqual(t, onDisk, int64(limit), "the files left on disk are within the limit")
require.Equal(t, onDisk, cache.size.Load(), "the tracked size matches what is on disk")
}
// Files that are rewritten while an eviction is running must be subtracted at the size they have when
// they are removed, not at the size the scan saw, or the tracked size drifts from the disk.
func TestFileCacheKeepsTheSizeAccurateWhenFilesAreRewrittenDuringEviction(t *testing.T) {
const (
kb = 1024
limit = 60 * kb
workers = 8
rounds = 300
keys = 40
)
cache := newLimitedCache(t, limit)
var wg sync.WaitGroup
for w := 0; w < workers; w++ {
wg.Add(1)
go func(w int) {
defer wg.Done()
for i := 0; i < rounds; i++ {
// the same keys again and again, with a different size each time
key := fmt.Sprintf("key-%d", (w+i)%keys)
_ = cache.Store(context.Background(), key, make([]byte, (1+(w*7+i)%8)*kb))
}
}(w)
}
wg.Wait()
waitFor(t, func() bool { return !cache.evicting.Load() })
_, onDisk := cache.scan()
require.Equal(t, onDisk, cache.size.Load(), "the tracked size matches what is on disk")
require.LessOrEqual(t, onDisk, int64(limit))
}
+57 -27
View File
@@ -17,6 +17,7 @@ import (
"github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/gomedia"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
@@ -52,7 +53,7 @@ func addMetadataToChildren(response *iteminfo.ExtendedFileInfo, opts utils.FileO
logger.Debugf("failed to extract metadata for file: %s, error: %v", fileItem.Name, err)
}
} else {
if err := extractVideoMetadata(context.Background(), fileItem, fullPath, sharedFFmpegService); err != nil {
if err := extractVideoMetadata(context.Background(), fileItem, fullPath, opts.AlbumArt, sharedFFmpegService); err != nil {
logger.Debugf("failed to extract video metadata for file: %s, error: %v", fileItem.Name, err)
}
}
@@ -380,7 +381,7 @@ func processContent(info *iteminfo.ExtendedFileInfo, idx *indexing.Index, opts u
extItem := &iteminfo.ExtendedItemInfo{
ItemInfo: info.ItemInfo,
}
err := extractVideoMetadata(context.Background(), extItem, info.RealPath, nil)
err := extractVideoMetadata(context.Background(), extItem, info.RealPath, opts.AlbumArt, nil)
if err != nil {
logger.Debugf("failed to extract video metadata for file: %s (%s): %v", info.RealPath, info.Name, err)
} else {
@@ -393,7 +394,8 @@ func processContent(info *iteminfo.ExtendedFileInfo, idx *indexing.Index, opts u
info.GetSubtitles(parentInfo)
}
subtitles, ok := gomedia.EmbeddedSubtitles(info.RealPath, info.ModTime, func() ([]utils.SubtitleTrack, bool) {
return ffmpeg.DetectEmbeddedSubtitles(info.RealPath, info.ModTime), true
// Shadow run: bypass the shared cache so ffprobe is compared fresh.
return ffmpeg.DetectEmbeddedSubtitlesContext(mediacache.WithoutCache(context.Background()), info.RealPath), true
})
if !ok {
subtitles = ffmpeg.DetectEmbeddedSubtitles(info.RealPath, info.ModTime)
@@ -471,23 +473,11 @@ func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
// go-media fast path: tags, lyrics availability, and artwork. Unsupported or
// failed reads fall through to the existing dhowden/tag code below.
if info, ok := gomedia.Audio(realPath, getArt, func() (gomedia.AudioInfo, bool) { return legacyAudioInfo(realPath, getArt) }); ok {
item.Metadata = &iteminfo.MediaMetadata{
Title: info.Tags.Title,
Artist: info.Tags.Artist,
Album: info.Tags.Album,
Year: info.Tags.Year,
Genre: info.Tags.Genre,
Track: info.Tags.Track,
HasLyrics: info.HasLyrics,
}
item.Metadata = &iteminfo.MediaMetadata{HasLyrics: info.HasLyrics}
applyMediaTags(item.Metadata, info, realPath)
if getDuration {
audioDuration(ctx, item, realPath, ffmpegService)
}
if n := len(info.Artwork); n > 0 && n <= 5*1024*1024 {
item.Metadata.AlbumArt = info.Artwork
} else if n > 5*1024*1024 {
logger.Debugf("Skipping album art for %s: too large (%d bytes)", realPath, n)
}
return nil
}
@@ -541,8 +531,9 @@ func audioDuration(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPat
if service == nil {
service = ffmpeg.Get()
}
// The shadow run can outlive the request, so it uses a context that is not cancelled with it.
shadowCtx := context.WithoutCancel(ctx)
// The shadow run can outlive the request, so it uses a context that is not
// cancelled with it. It bypasses the shared cache so ffmpeg is compared fresh.
shadowCtx := mediacache.WithoutCache(context.WithoutCancel(ctx))
legacy := func() (float64, bool) {
d, err := service.GetMediaDuration(shadowCtx, realPath)
return d, err == nil
@@ -707,14 +698,15 @@ func legacyLyrics(realPath string) (gomedia.LyricsResult, bool) {
// extractVideoMetadata extracts duration and codec info from video files using ffprobe.
// If ffmpegService is nil, a new service will be created (for backward compatibility)
func extractVideoMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPath string, ffmpegService *ffmpeg.FFmpegService) error {
func extractVideoMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPath string, getArt bool, ffmpegService *ffmpeg.FFmpegService) error {
// go-media fast path. Works without ffmpeg installed.
probeService := ffmpegService
if probeService == nil {
probeService = ffmpeg.Get()
}
// The shadow run can outlive the request, so it uses a context that is not cancelled with it.
shadowCtx := context.WithoutCancel(ctx)
// The shadow run can outlive the request, so it uses a context that is not
// cancelled with it. It bypasses the shared cache so ffprobe is compared fresh.
shadowCtx := mediacache.WithoutCache(context.WithoutCancel(ctx))
legacyProbe := func() (gomedia.ProbeResult, bool) {
info, err := probeService.ProbeFile(shadowCtx, realPath)
if err != nil || info == nil {
@@ -759,12 +751,33 @@ func extractVideoMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
if info.FormatName != "" {
meta.Container = info.FormatName
}
if ok {
if tags, tagsOK := gomedia.VideoTags(realPath, getArt); tagsOK {
applyMediaTags(meta, tags, realPath)
}
}
if meta.Duration > 0 || meta.VideoCodec != "" || meta.AudioCodec != "" || meta.Container != "" {
item.Metadata = meta
}
return nil
}
// applyMediaTags copies go-media tags and embedded cover art into meta. Covers
// over 8MB are skipped.
func applyMediaTags(meta *iteminfo.MediaMetadata, info gomedia.AudioInfo, realPath string) {
meta.Title = info.Tags.Title
meta.Artist = info.Tags.Artist
meta.Album = info.Tags.Album
meta.Year = info.Tags.Year
meta.Genre = info.Tags.Genre
meta.Track = info.Tags.Track
if n := len(info.Artwork); n > 0 && n <= 8*1024*1024 {
meta.AlbumArt = info.Artwork
} else if n > 8*1024*1024 {
logger.Debugf("Skipping album art for %s: too large (%d bytes)", realPath, n)
}
}
// DeleteFiles deletes the file or directory at indexPath. boundIndexPath limits
// symlink resolution to the caller's scope ("" or "/" = source root only). The
// entry itself is removed: a symlink leaf is deleted, never followed, and an
@@ -777,6 +790,7 @@ func DeleteFiles(source, boundIndexPath, indexPath string, isDir bool) error {
if utils.CleanIndexPathSegment(indexPath) == "" {
return fmt.Errorf("refusing to delete source root directory: %s", indexPath)
}
deleteIndexPath := indexPath
realPath, err := index.GetRealPathForRemoval(boundIndexPath, indexPath)
if err != nil {
return err
@@ -795,9 +809,13 @@ func DeleteFiles(source, boundIndexPath, indexPath string, isDir bool) error {
return err
}
// Clear cache entries
indexing.RealPathCache.Delete(realPath)
indexing.IsDirCache.Delete(realPath + ":isdir")
if isDir {
indexing.ClearRealPathCaches()
}
index.InvalidateRealPathCache(deleteIndexPath, boundIndexPath)
if parent := utils.GetParentDirectoryPath(deleteIndexPath); parent != "" && parent != "/" {
index.InvalidateRealPathCache(parent, boundIndexPath)
}
// Remove metadata from index
deleteSuccess := index.DeleteMetadata(indexPath, isDir, isDir)
@@ -849,8 +867,7 @@ func RefreshIndex(source string, path string, isDir bool, recursive bool) error
if !Exists(realPath) {
// Directory no longer exists, remove it from the index
// This clears both Directories and DirectoriesLedger maps
indexing.RealPathCache.Delete(realPath)
indexing.IsDirCache.Delete(realPath + ":isdir")
idx.InvalidateRealPathCache(path)
idx.DeleteMetadata(path, true, false)
return nil
}
@@ -928,6 +945,9 @@ func moveResourceImpl(isSrcDir bool, sourceIndex, destIndex, realsrc, realdst st
if err != nil {
return err
}
if isSrcDir {
indexing.ClearRealPathCaches()
}
// Handle SOURCE cleanup (treat as deletion)
// Run async to avoid blocking the HTTP response
@@ -1088,6 +1108,11 @@ func WriteDirectory(boundIndexPath string, opts utils.FileOptions) error {
}
}
idx.InvalidateRealPathCache(opts.Path, boundIndexPath)
if parent := utils.GetParentDirectoryPath(opts.Path); parent != "" && parent != "/" {
idx.InvalidateRealPathCache(parent, boundIndexPath)
}
return nil
}
@@ -1163,6 +1188,11 @@ func writeFile(source, boundIndexPath, path string, in io.Reader) error {
}
}
idx.InvalidateRealPathCache(path, boundIndexPath)
if parent := utils.GetParentDirectoryPath(path); parent != "" && parent != "/" {
idx.InvalidateRealPathCache(parent, boundIndexPath)
}
return nil
}
@@ -0,0 +1,44 @@
package access
import (
"os"
"path/filepath"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
)
// PermittedAtRealPath reports whether username may access realPath after symlink
// resolution. DENY rules apply to the target index path, not only the link path.
func (s *Storage) PermittedAtRealPath(index *indexing.Index, username, realPath string, isDir bool) bool {
if s == nil || index == nil || username == "" || realPath == "" {
return false
}
lexical := realPath
if !strings.HasPrefix(realPath, index.Path) {
root, err := filepath.EvalSymlinks(index.Path)
if err != nil {
return false
}
rel, err := filepath.Rel(root, realPath)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return false
}
lexical = index.Path
if rel != "." {
lexical = filepath.Join(index.Path, rel)
}
}
indexPath := index.MakeIndexPath(lexical, isDir).String()
return s.Permitted(index.Path, utils.IndexPathFromNormalized(indexPath, true), username)
}
// IsDirAtRealPath reports whether an existing path is a directory; missing paths are treated as files.
func IsDirAtRealPath(realPath string) bool {
st, err := os.Stat(realPath)
if err != nil {
return false
}
return st.IsDir()
}
@@ -0,0 +1,140 @@
package access
import (
"os"
"path/filepath"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func setupSymlinkDenyFixture(t *testing.T) (sourceRoot string, idx *indexing.Index, store *Storage, username string) {
t.Helper()
sourceRoot = filepath.Join(t.TempDir(), "data")
aliceDir := filepath.Join(sourceRoot, "alice")
bobDir := filepath.Join(sourceRoot, "bob")
for _, dir := range []string{aliceDir, bobDir} {
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
}
if err := os.WriteFile(filepath.Join(bobDir, "secret.txt"), []byte("secret"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink("../bob/secret.txt", filepath.Join(aliceDir, "link.txt")); err != nil {
t.Skipf("symlinks required: %v", err)
}
if err := os.Symlink("../bob", filepath.Join(aliceDir, "bobdir")); err != nil {
t.Skipf("symlinks required: %v", err)
}
settings.Config.Server.SourceMap = map[string]*settings.Source{
sourceRoot: {Path: sourceRoot, Config: settings.SourceConfig{DenyByDefault: false}},
}
store = NewStorage(nil)
username = "scoped-user"
if err := store.DenyUser(sourceRoot, utils.IndexPathFromNormalized("/bob", true), username); err != nil {
t.Fatal(err)
}
idx = &indexing.Index{Source: settings.Source{Path: sourceRoot}}
return sourceRoot, idx, store, username
}
func TestPermittedAtRealPath_AllowsLinkPathDeniesResolvedTarget(t *testing.T) {
_, idx, store, user := setupSymlinkDenyFixture(t)
if !store.Permitted(idx.Path, utils.IndexPathFromNormalized("/alice/link.txt", false), user) {
t.Fatal("index path at the symlink should be allowed")
}
if store.Permitted(idx.Path, utils.IndexPathFromNormalized("/bob/secret.txt", false), user) {
t.Fatal("direct path under /bob should be denied")
}
realPath, _, err := idx.GetRealPathScoped("/", "/alice/link.txt")
if err != nil {
t.Fatal(err)
}
if store.PermittedAtRealPath(idx, user, realPath, false) {
t.Fatal("resolved target under denied /bob must not be permitted")
}
}
func TestPermittedAtRealPath_DeniesWriteThroughDirectorySymlink(t *testing.T) {
_, idx, store, user := setupSymlinkDenyFixture(t)
realPath, err := idx.GetRealPathForMutation("/", "/alice/bobdir/planted.txt")
if err != nil {
t.Fatal(err)
}
if store.PermittedAtRealPath(idx, user, realPath, false) {
t.Fatal("resolved write target under denied /bob must not be permitted")
}
}
func TestPermittedAtRealPath_DeniesCopySourceResolved(t *testing.T) {
_, idx, store, user := setupSymlinkDenyFixture(t)
realSrc, isDir, err := idx.GetRealPathScoped("/", "/alice/link.txt")
if err != nil {
t.Fatal(err)
}
if isDir {
t.Fatal("expected file")
}
if store.PermittedAtRealPath(idx, user, realSrc, isDir) {
t.Fatal("copy source resolved to denied target must not be permitted")
}
}
func TestPermittedAtRealPath_SymlinkedSourceRoot(t *testing.T) {
sourceRoot, _, _, user := setupSymlinkDenyFixture(t)
linkRoot := filepath.Join(filepath.Dir(sourceRoot), "root-link")
if err := os.Symlink(sourceRoot, linkRoot); err != nil {
t.Skipf("symlinks required: %v", err)
}
if _, err := filepath.EvalSymlinks(linkRoot); err != nil {
t.Fatal(err)
}
settings.Config.Server.SourceMap = map[string]*settings.Source{
linkRoot: {Path: linkRoot, Config: settings.SourceConfig{DenyByDefault: false}},
}
store := NewStorage(nil)
if err := store.DenyUser(linkRoot, utils.IndexPathFromNormalized("/bob", true), user); err != nil {
t.Fatal(err)
}
idx := &indexing.Index{Source: settings.Source{Path: linkRoot}}
realPath, _, err := idx.GetRealPathScoped("/", "/alice/link.txt")
if err != nil {
t.Fatal(err)
}
realPath, err = filepath.EvalSymlinks(realPath)
if err != nil {
t.Fatal(err)
}
if store.PermittedAtRealPath(idx, user, realPath, false) {
t.Fatal("resolved target under denied /bob must not be permitted with symlinked source root")
}
}
func TestIsDirAtRealPath(t *testing.T) {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "nested"), 0o755); err != nil {
t.Fatal(err)
}
file := filepath.Join(dir, "file.txt")
if err := os.WriteFile(file, []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if !IsDirAtRealPath(dir) {
t.Fatal("expected directory")
}
if IsDirAtRealPath(file) {
t.Fatal("expected file")
}
if IsDirAtRealPath(filepath.Join(dir, "missing")) {
t.Fatal("missing path should not be treated as directory")
}
}
+5 -4
View File
@@ -346,10 +346,11 @@ type NonAdminEditable struct {
}
type FileLoading struct {
MaxConcurrent int `json:"maxConcurrentUpload"`
UploadChunkSize int `json:"uploadChunkSizeMb"`
ClearAll bool `json:"clearAll"`
DownloadChunkSize int `json:"downloadChunkSizeMb"`
MaxConcurrent int `json:"maxConcurrentUpload"`
UploadChunkSize int `json:"uploadChunkSizeMb"`
ClearAll bool `json:"clearAll"`
DownloadChunkSize int `json:"downloadChunkSizeMb"`
UploadStallTimeoutSeconds int `json:"uploadStallTimeoutSeconds"` // seconds without progress before an upload is paused, 0 uses the default (10)
}
// SidebarLink represents a customizable link in the sidebar.
-14
View File
@@ -1,14 +0,0 @@
package ffmpeg
import (
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/go-cache/cache"
)
var (
MediaCache = cache.NewCache[[]utils.SubtitleTrack](24 * time.Hour) // subtitle track lists from ffprobe
SubtitleContentCache = cache.NewCache[string](24 * time.Hour) // extracted embedded subtitle content
MetadataCache = cache.NewCache[float64](24 * time.Hour) // media duration from ffprobe
)
@@ -9,13 +9,51 @@ import (
"os/exec"
"strconv"
"strings"
"sync/atomic"
"github.com/kovidgoyal/imaging"
)
// HEICDisplayRotationReader reads the display rotation that a HEIC/HEIF/AVIF
// file's item properties (irot) ask for, without decoding it. The rotation is
// in degrees counter-clockwise, normalized to (-180, 180], which is the value
// ffmpeg prints as "Display Matrix: rotation of ..." for the same file. ok is
// false when the reader could not tell; the ffmpeg probe is used instead.
type HEICDisplayRotationReader func(path string) (rotation float64, ok bool)
// heicDisplayRotationReader is the header reader set by
// SetHEICDisplayRotationReader, or nil.
var heicDisplayRotationReader atomic.Pointer[HEICDisplayRotationReader]
// SetHEICDisplayRotationReader installs the reader HEICDisplayRotation tries
// before running ffmpeg. The gomedia package installs go-media's reader when it
// is loaded; it cannot be called from here directly because gomedia imports
// this package. Passing nil removes the reader.
func SetHEICDisplayRotationReader(fn HEICDisplayRotationReader) {
if fn == nil {
heicDisplayRotationReader.Store(nil)
return
}
heicDisplayRotationReader.Store(&fn)
}
// HEICDisplayRotation returns the display rotation of heicPath and whether it is
// known. The installed header reader (go-media) answers first; the
// "ffmpeg -f null" stderr probe runs only when no reader is installed or the
// reader could not read the file.
func (s *Service) HEICDisplayRotation(ctx context.Context, heicPath string) (float64, bool) {
if p := heicDisplayRotationReader.Load(); p != nil {
if rotation, ok := (*p)(heicPath); ok {
return rotation, true
}
}
return s.GetHEICDisplayMatrixRotation(ctx, heicPath)
}
// GetHEICDisplayMatrixRotation returns the largest rotation (degrees) ffmpeg reports for a
// HEIC display matrix. The second return value is false when the probe could not run or
// parse stderr — callers must not treat that as “no rotation was applied”.
// HEICDisplayRotation is preferred; it only runs this probe as a fallback.
func (s *Service) GetHEICDisplayMatrixRotation(ctx context.Context, heicPath string) (float64, bool) {
if s == nil {
return 0, false
@@ -36,6 +74,8 @@ func (s *Service) GetHEICDisplayMatrixRotation(ctx context.Context, heicPath str
return rotation, ok
}
// parseDisplayMatrixRotation returns the largest-magnitude "Display Matrix"
// rotation in ffmpeg stderr, and whether any was found.
func parseDisplayMatrixRotation(stderr string) (float64, bool) {
var best float64
var found bool
@@ -65,6 +105,7 @@ func parseDisplayMatrixRotation(stderr string) (float64, bool) {
return best, found
}
// isNormalOrientation reports whether an EXIF orientation needs no change.
func isNormalOrientation(orientation string) bool {
switch orientation {
case "", "Horizontal (normal)", "Top-left", "1":
@@ -74,6 +115,8 @@ func isNormalOrientation(orientation string) bool {
}
}
// isPureRotationOrientation reports whether an EXIF orientation is a rotation
// without a mirror.
func isPureRotationOrientation(orientation string) bool {
switch orientation {
case "Rotate 90 CW", "Right-top", "6",
@@ -100,6 +143,8 @@ func orientationNeedsPostCorrection(orientation string, displayRotation float64,
return true
}
// applyOrientationToJPEG rotates or flips JPEG bytes for an EXIF orientation.
// The input is returned unchanged when it cannot be decoded or re-encoded.
func applyOrientationToJPEG(jpegBytes []byte, orientation string) []byte {
if len(jpegBytes) < 100 || isNormalOrientation(orientation) {
return jpegBytes
@@ -50,3 +50,35 @@ func TestOrientationNeedsPostCorrection(t *testing.T) {
}
}
}
func TestHEICDisplayRotationPrefersReader(t *testing.T) {
prev := heicDisplayRotationReader.Load()
t.Cleanup(func() { heicDisplayRotationReader.Store(prev) })
var calls int
SetHEICDisplayRotationReader(func(path string) (float64, bool) {
calls++
if path == "known.heic" {
return -90, true
}
return 0, false
})
var svc *Service // nil: the ffmpeg probe cannot run
got, ok := svc.HEICDisplayRotation(t.Context(), "known.heic")
if !ok || got != -90 {
t.Fatalf("HEICDisplayRotation(known) = %v, %v; want -90, true", got, ok)
}
// The reader cannot tell: the ffmpeg probe runs, and with no service it
// reports unknown rather than "no rotation".
if _, ok := svc.HEICDisplayRotation(t.Context(), "other.heic"); ok {
t.Fatal("HEICDisplayRotation(other) ok = true, want the ffmpeg probe's false")
}
if calls != 2 {
t.Fatalf("reader calls = %d, want 2", calls)
}
SetHEICDisplayRotationReader(nil)
if _, ok := svc.HEICDisplayRotation(t.Context(), "known.heic"); ok {
t.Fatal("with no reader, HEICDisplayRotation must use the ffmpeg probe")
}
}
+5 -3
View File
@@ -81,14 +81,16 @@ func (s *Service) ConvertHEICToJPEG(ctx context.Context, heicPath string, target
return s.CorrectHEICJPEGOrientation(ctx, heicPath, jpegBytes), nil
}
// CorrectHEICJPEGOrientation applies irot / display-matrix correction after a full-resolution
// HEIC decode when pixels are still in coded orientation (go-media and ffmpeg filter-free paths).
// CorrectHEICJPEGOrientation applies EXIF orientation after a filter-free ffmpeg HEIC decode
// when the pixels are still in coded orientation. The display rotation (irot) comes from
// HEICDisplayRotation: go-media reads it from the item properties, and the ffmpeg stderr probe
// is only a fallback.
func (s *Service) CorrectHEICJPEGOrientation(ctx context.Context, heicPath string, jpegBytes []byte) []byte {
orientation, err := s.GetHEICOrientation(heicPath)
if err != nil {
orientation = "Horizontal (normal)"
}
displayRotation, displayKnown := s.GetHEICDisplayMatrixRotation(ctx, heicPath)
displayRotation, displayKnown := s.HEICDisplayRotation(ctx, heicPath)
if orientationNeedsPostCorrection(orientation, displayRotation, displayKnown) {
return applyOrientationToJPEG(jpegBytes, orientation)
}
@@ -0,0 +1,115 @@
package ffmpeg
import (
"context"
"os"
"os/exec"
"path/filepath"
"sync"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
)
const cacheFixtureMP4 = "../../../frontend/tests/playwright-files/myfolder/testdata/sample-640x480.mp4"
var (
testServiceOnce sync.Once
testServiceErr error
)
// initTestService starts the ffmpeg service once, without hardware probes, or
// skips.
func initTestService(t *testing.T) *Service {
t.Helper()
if testing.Short() {
t.Skip("starts ffmpeg")
}
if _, err := exec.LookPath("ffmpeg"); err != nil {
t.Skip("ffmpeg not installed")
}
if _, err := exec.LookPath("ffprobe"); err != nil {
t.Skip("ffprobe not installed")
}
testServiceOnce.Do(func() {
testServiceErr = Initialize(context.Background(), InitOptions{SkipHWTests: true, CacheDir: os.TempDir()})
})
if testServiceErr != nil {
t.Skipf("ffmpeg init failed: %v", testServiceErr)
}
svc := Get()
if svc == nil {
t.Skip("ffmpeg service unavailable")
}
return svc
}
// copyFixture copies the MP4 fixture so the test owns its cache key.
func copyFixture(t *testing.T) (string, mediacache.Key) {
t.Helper()
data, err := os.ReadFile(cacheFixtureMP4)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "sample.mp4")
if err = os.WriteFile(path, data, 0o644); err != nil {
t.Fatal(err)
}
key, err := mediacache.Stat(path)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
mediacache.Probes.Delete(key.String())
mediacache.Durations.Delete(key.String())
})
return path, key
}
// TestProbeFileUsesSharedCache checks that ffprobe results go into the shared
// cache, cached values are reused, and WithoutCache bypasses both.
func TestProbeFileUsesSharedCache(t *testing.T) {
svc := initTestService(t)
path, key := copyFixture(t)
got, err := svc.ProbeFile(context.Background(), path)
if err != nil {
t.Fatal(err)
}
cached, ok := mediacache.Probes.Get(key.String())
if !ok || cached != *got {
t.Fatalf("ProbeFile must store its result in the shared cache: cached=%+v ok=%v got=%+v", cached, ok, *got)
}
sentinel := FileProbeInfo{Duration: 1, VideoCodec: "sentinel"}
mediacache.Probes.Set(key.String(), sentinel)
got, err = svc.ProbeFile(context.Background(), path)
if err != nil || *got != sentinel {
t.Fatalf("ProbeFile must reuse a cached value (e.g. from go-media): got=%+v err=%v", got, err)
}
fresh, err := svc.ProbeFile(mediacache.WithoutCache(context.Background()), path)
if err != nil || *fresh == sentinel {
t.Fatalf("WithoutCache must probe fresh: got=%+v err=%v", fresh, err)
}
if cached, _ := mediacache.Probes.Get(key.String()); cached != sentinel {
t.Fatal("WithoutCache must not overwrite the shared cache")
}
}
// TestGetMediaDurationUsesSharedCache checks that cached durations are reused
// and WithoutCache probes fresh.
func TestGetMediaDurationUsesSharedCache(t *testing.T) {
svc := initTestService(t)
path, key := copyFixture(t)
mediacache.Durations.Set(key.String(), 4242)
d, err := svc.GetMediaDuration(context.Background(), path)
if err != nil || d != 4242 {
t.Fatalf("GetMediaDuration must reuse a cached value: d=%v err=%v", d, err)
}
d, err = svc.GetMediaDuration(mediacache.WithoutCache(context.Background()), path)
if err != nil || d == 4242 || d <= 0 {
t.Fatalf("WithoutCache must probe fresh: d=%v err=%v", d, err)
}
}
+32 -16
View File
@@ -8,6 +8,7 @@ import (
"strings"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
)
@@ -49,24 +50,29 @@ func resolveLocalMediaPath(mediaPath string) (string, os.FileInfo, error) {
return clean, info, nil
}
// FileProbeInfo holds ffprobe results for a local media file.
type FileProbeInfo struct {
Duration float64
VideoCodec string
AudioCodec string
FormatName string
}
// FileProbeInfo holds probe results for a local media file. It is the shared
// cache type, so go-media and ffprobe results are interchangeable.
type FileProbeInfo = mediacache.ProbeInfo
// ProbeFile extracts duration and codec info from a media file.
// ProbeFile extracts duration and codec info from a media file. Results are
// stored in the shared media cache, which the go-media probe also reads, unless
// ctx is mediacache.WithoutCache.
func (s *Service) ProbeFile(ctx context.Context, mediaPath string) (*FileProbeInfo, error) {
if s == nil || s.inner == nil {
return nil, fmt.Errorf("ffmpeg service not available")
}
localPath, _, err := resolveLocalMediaPath(mediaPath)
localPath, fileInfo, err := resolveLocalMediaPath(mediaPath)
if err != nil {
return nil, err
}
useCache := !mediacache.Bypassed(ctx)
cacheKey := mediacache.FromFileInfo(localPath, fileInfo).String()
if useCache {
if cached, ok := mediacache.Probes.Get(cacheKey); ok {
return &cached, nil
}
}
info, err := s.inner.ProbeStream(ctx, goffmpeg.ProbeStreamOptions{
URL: localPath,
@@ -77,15 +83,20 @@ func (s *Service) ProbeFile(ctx context.Context, mediaPath string) (*FileProbeIn
return nil, err
}
return &FileProbeInfo{
res := FileProbeInfo{
Duration: info.Duration,
VideoCodec: info.VideoCodec,
AudioCodec: info.AudioCodec,
FormatName: info.FormatName,
}, nil
}
if useCache {
mediacache.Probes.Set(cacheKey, res)
}
return &res, nil
}
// GetMediaDuration extracts the duration of a media file in seconds.
// GetMediaDuration extracts the duration of a media file in seconds. Results
// are stored in the shared media cache unless ctx is mediacache.WithoutCache.
func (s *Service) GetMediaDuration(ctx context.Context, mediaPath string) (float64, error) {
if s == nil || s.inner == nil {
return 0, fmt.Errorf("ffmpeg service not available")
@@ -95,9 +106,12 @@ func (s *Service) GetMediaDuration(ctx context.Context, mediaPath string) (float
if err != nil {
return 0, err
}
cacheKey := fmt.Sprintf("%s:%d", localPath, fileInfo.ModTime().Unix())
if duration, found := MetadataCache.Get(cacheKey); found {
return duration, nil
useCache := !mediacache.Bypassed(ctx)
cacheKey := mediacache.FromFileInfo(localPath, fileInfo).String()
if useCache {
if duration, found := mediacache.Durations.Get(cacheKey); found {
return duration, nil
}
}
duration, err := s.inner.GetMediaDuration(ctx, localPath)
@@ -105,6 +119,8 @@ func (s *Service) GetMediaDuration(ctx context.Context, mediaPath string) (float
return 0, err
}
MetadataCache.Set(cacheKey, duration)
if useCache {
mediacache.Durations.Set(cacheKey, duration)
}
return duration, nil
}
+12 -5
View File
@@ -9,6 +9,7 @@ import (
"sync"
"sync/atomic"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
"github.com/gtsteffaniak/go-ffmpeg/ops"
@@ -460,7 +461,8 @@ func (s *Service) VideoPreview(ctx context.Context, w io.Writer, opts ops.Previe
return s.inner.CaptureFrame(ctx, w, frame)
}
// ExtractSubtitle returns embedded subtitle content as WebVTT.
// ExtractSubtitle returns embedded subtitle content as WebVTT. Results are
// stored in the shared media cache unless ctx is mediacache.WithoutCache.
func (s *Service) ExtractSubtitle(ctx context.Context, videoPath string, streamIndex int) (string, error) {
if s == nil || s.inner == nil {
return "", fmt.Errorf("ffmpeg service not available")
@@ -470,9 +472,12 @@ func (s *Service) ExtractSubtitle(ctx context.Context, videoPath string, streamI
if err != nil {
return "", fmt.Errorf("failed to stat file: %w", err)
}
cacheKey := fmt.Sprintf("subtitle_content:%s:%d:%d", videoPath, streamIndex, fileInfo.ModTime().Unix())
if content, ok := SubtitleContentCache.Get(cacheKey); ok {
return content, nil
useCache := !mediacache.Bypassed(ctx)
cacheKey := mediacache.FromFileInfo(videoPath, fileInfo).Stream(streamIndex)
if useCache {
if content, ok := mediacache.SubtitleContent.Get(cacheKey); ok {
return content, nil
}
}
content, err := s.inner.ExtractSubtitle(ctx, videoPath, streamIndex)
@@ -480,7 +485,9 @@ func (s *Service) ExtractSubtitle(ctx context.Context, videoPath string, streamI
return "", err
}
SubtitleContentCache.Set(cacheKey, content)
if useCache {
mediacache.SubtitleContent.Set(cacheKey, content)
}
return content, nil
}
+22 -6
View File
@@ -7,16 +7,30 @@ import (
"strings"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-logger/logger"
)
// DetectEmbeddedSubtitles detects embedded subtitle streams.
// DetectEmbeddedSubtitles detects embedded subtitle streams. Results are stored
// in the shared media cache, keyed by the file's current size and modification
// time; modtime is kept for API compatibility and is not used for the key.
func DetectEmbeddedSubtitles(videoPath string, modtime time.Time) []utils.SubtitleTrack {
key := "embedded_subtitles:" + videoPath + ":" + modtime.Format(time.RFC3339)
if cached, ok := MediaCache.Get(key); ok {
return cached
_ = modtime
return DetectEmbeddedSubtitlesContext(context.Background(), videoPath)
}
// DetectEmbeddedSubtitlesContext is DetectEmbeddedSubtitles with a context. The
// shared media cache is skipped when ctx is mediacache.WithoutCache.
func DetectEmbeddedSubtitlesContext(ctx context.Context, videoPath string) []utils.SubtitleTrack {
useCache := !mediacache.Bypassed(ctx)
key, statErr := mediacache.Stat(videoPath)
useCache = useCache && statErr == nil
if useCache {
if cached, ok := mediacache.SubtitleTracks.Get(key.String()); ok {
return cached
}
}
svc := Get()
@@ -24,14 +38,16 @@ func DetectEmbeddedSubtitles(videoPath string, modtime time.Time) []utils.Subtit
return nil
}
tracks, err := svc.inner.DetectSubtitles(context.Background(), videoPath)
tracks, err := svc.inner.DetectSubtitles(ctx, videoPath)
if err != nil {
logger.Debug("ffprobe failed for file: " + videoPath + ", error: " + err.Error())
return nil
}
subtitles := mapSubtitleTracks(tracks)
MediaCache.Set(key, subtitles)
if useCache {
mediacache.SubtitleTracks.Set(key.String(), subtitles)
}
return subtitles
}
@@ -0,0 +1,73 @@
package gomedia
import (
"os"
"os/exec"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
)
const fixtureWMV2 = fixtureDir + "/file_example_WMV_480_1_2MB.wmv"
// TestContainerGates pins which extensions go-media handles for probes and
// for thumbnails.
func TestContainerGates(t *testing.T) {
cases := []struct {
name string
probe, thumbnail bool
}{
{"a.mp4", true, true},
{"a.M4V", true, true},
{"a.mov", true, true},
{"a.mkv", true, true},
{"a.webm", true, true},
{"a.wmv", false, true},
{"a.ASF", false, true},
{"a.avi", false, false},
{"a.ts", false, false},
{"a.m2ts", false, false},
{"a.mts", false, false},
{"a.mpg", false, false},
{"a.mpeg", false, false},
{"a.vob", false, false},
{"a.flv", false, false},
}
for _, c := range cases {
require.Equal(t, c.probe, supportsVideoContainer(c.name), "probe gate for %s", c.name)
require.Equal(t, c.thumbnail, supportsVideoThumbnail(c.name), "thumbnail gate for %s", c.name)
}
for _, name := range []string{"a.heic", "a.HEIF", "a.heics", "a.avif"} {
require.True(t, supportsHEIFImage(name), name)
}
require.False(t, supportsHEIFImage("a.jpg"))
}
// TestWMV2FallsBackToFFmpeg checks that a WMV2 file is not probed by go-media
// and that its thumbnail falls back to ffmpeg (go-media has no WMV1/WMV2
// decoder).
func TestWMV2FallsBackToFFmpeg(t *testing.T) {
withGate(t, false)
_, ok := Probe(fixtureWMV2, nil)
require.False(t, ok, "WMV probes stay on ffprobe")
_, ok = VideoThumbnail(fixtureWMV2, 320, 240, 80, 10, nil)
require.False(t, ok, "WMV2 thumbnails must fall back to ffmpeg")
}
// TestWMV1FallsBackToFFmpeg is TestWMV2FallsBackToFFmpeg for a WMV1 stream
// encoded on the fly.
func TestWMV1FallsBackToFFmpeg(t *testing.T) {
withGate(t, false)
requireTool(t, "ffmpeg")
path := filepath.Join(t.TempDir(), "wmv1.wmv")
out, err := exec.Command("ffmpeg", "-v", "error", "-y", "-f", "lavfi", "-i", "testsrc2=size=160x120:rate=25:duration=2",
"-c:v", "wmv1", path).CombinedOutput()
if err != nil {
t.Skipf("ffmpeg cannot encode wmv1: %v: %s", err, out)
}
_, err = os.Stat(path)
require.NoError(t, err)
_, ok := VideoThumbnail(path, 160, 120, 80, 10, nil)
require.False(t, ok, "WMV1 thumbnails must fall back to ffmpeg")
}
+146 -75
View File
@@ -20,6 +20,7 @@ import (
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-cache/cache"
@@ -28,20 +29,27 @@ import (
"golang.org/x/sync/singleflight"
)
var (
subtitleListCache = cache.NewCache[[]utils.SubtitleTrack](24 * time.Hour)
subtitleContentCache = cache.NewCache[string](24 * time.Hour)
// Probe, duration, and subtitle results are cached in the mediacache package,
// which the ffmpeg code path shares. A directory listing probes every video
// child, and the preview flow probes the focused file again; the cache keeps
// each request from re-reading the same containers. Keys include size and
// modification time, so an edited file is read again.
// probeCache caches successful probe results by file identity (path, modtime,
// size). A directory listing probes every video child, and the preview flow
// probes the focused file again; without this each request re-read the same
// containers. Keying on modtime+size means an edited file is re-probed.
probeCache = cache.NewCache[ProbeResult](24 * time.Hour)
// probeGroup coalesces concurrent probes of the same file so a burst of
// requests for one container reads it once.
var probeGroup singleflight.Group
// probeGroup coalesces concurrent probes of the same file so a burst of
// requests for one container reads it once.
probeGroup singleflight.Group
)
// tagsCache caches the embedded part of a tag read by file identity. A listing
// reads tags for every media child on each uncached request. Sidecar lyrics
// are not cached: adding an .lrc next to a file does not change its modtime.
var tagsCache = cache.NewCache[AudioInfo](24 * time.Hour)
func init() {
// The ffmpeg HEIC decode path corrects orientation after decoding and needs
// the file's irot. internal/ffmpeg cannot import this package, so go-media's
// header reader is installed here.
ffmpeg.SetHEICDisplayRotationReader(HEICDisplayRotation)
}
// Enabled reports whether the go-media fast path may run. The admin
// "Always use ffmpeg" setting turns it off.
@@ -60,9 +68,15 @@ func videoDimensions(info *media.ProbeInfo) (w, h int, ok bool) {
}
// videoContainerExts are the containers go-media parses for video probes,
// thumbnails, and embedded subtitles. Other video containers (AVI, WMV, TS, ...)
// are not go-media targets, so they go straight to ffmpeg with no file open and
// no log line. go-media reads MOV as MP4.
// thumbnails, embedded subtitles, and keyframe indexes. go-media reads MOV as
// MP4.
//
// go-media also demuxes AVI, ASF/WMV, MPEG-TS/M2TS, and MPEG-PS (VOB), but their
// probes are left to ffmpeg on purpose: go-media's results for those containers
// do not yet match ffprobe (TS/M2TS and PS list no audio streams, PS durations
// are missing or wrong, AVI misnames MPEG-4 Part 2 and AAC, ASF reports WMA Pro
// as wmav2). Those files go straight to ffmpeg with no file open and no log
// line. See videoThumbnailExts for the thumbnail exception.
var videoContainerExts = map[string]bool{
".mp4": true,
".m4v": true,
@@ -71,11 +85,31 @@ var videoContainerExts = map[string]bool{
".webm": true,
}
// videoThumbnailExts are the containers go-media decodes video thumbnails from:
// videoContainerExts plus ASF/WMV, whose WMV3 keyframes go-media decodes
// bit-exact with ffmpeg. WMV1, WMV2, MS-MPEG4, and WVC1 Advanced Profile return
// an unsupported error and fall back to ffmpeg. AVI, TS/M2TS, and PS stay on
// ffmpeg: go-media's TS and PS thumbnails scan the whole file for the duration
// (several times slower than ffmpeg on large files), and AVI seeks by a
// duration that can be wrong.
var videoThumbnailExts = map[string]bool{
".mp4": true,
".m4v": true,
".mov": true,
".mkv": true,
".webm": true,
".wmv": true,
".asf": true,
}
// heifImageExts are still-image containers go-media decodes for HEIC preview.
// AVIF is a HEIF container with an AV1 image item; go-media decodes it in pure
// Go and applies its irot/imir, like HEIC.
var heifImageExts = map[string]bool{
".heic": true,
".heif": true,
".heics": true,
".avif": true,
}
const heicDecodeMaxEdge = 16384
@@ -93,15 +127,27 @@ var cameraRawExts = map[string]bool{
".dc2": true, ".x3f": true, ".iiq": true, ".nkc": true, ".r3d": true,
}
// supportsVideoContainer reports whether go-media targets path's container.
// supportsVideoContainer reports whether go-media targets path's container for
// probes, embedded subtitles, and keyframe indexes.
func supportsVideoContainer(path string) bool {
return videoContainerExts[strings.ToLower(filepath.Ext(path))]
}
// supportsVideoThumbnail reports whether go-media targets path's container for
// video thumbnails.
func supportsVideoThumbnail(path string) bool {
return videoThumbnailExts[strings.ToLower(filepath.Ext(path))]
}
// supportsHEIFImage reports whether path is a HEIF-family still image
// (HEIC/HEIF/AVIF) go-media decodes.
func supportsHEIFImage(path string) bool {
return heifImageExts[strings.ToLower(filepath.Ext(path))]
}
// supportsCameraRAW reports whether path is a camera RAW format go-media
// extracts a preview from.
func supportsCameraRAW(path string) bool {
return cameraRawExts[strings.ToLower(filepath.Ext(path))]
}
@@ -133,16 +179,26 @@ func logUnsupported(op, path, reason string) {
// openFile opens path for go-media, which reads through io.ReaderAt.
func openFile(path string) (*os.File, int64, error) {
f, err := os.Open(path)
f, info, err := openFileInfo(path)
if err != nil {
return nil, 0, err
}
return f, info.Size(), nil
}
// openFileInfo opens path and returns its FileInfo, which also gives the
// shared cache key.
func openFileInfo(path string) (*os.File, os.FileInfo, error) {
f, err := os.Open(path)
if err != nil {
return nil, nil, err
}
info, err := f.Stat()
if err != nil {
_ = f.Close()
return nil, 0, err
return nil, nil, err
}
return f, info.Size(), nil
return f, info, nil
}
// ffprobeFormatNames maps go-media containers to the format_name strings ffprobe
@@ -179,30 +235,25 @@ func probeFast(path string) (ProbeResult, bool) {
return ProbeResult{}, false
}
f, size, err := openFile(path)
f, info, err := openFileInfo(path)
if err != nil {
logFallback("probe", path, err)
return ProbeResult{}, false
}
defer f.Close()
info, err := f.Stat()
if err != nil {
logFallback("probe", path, err)
return ProbeResult{}, false
}
cacheKey := probeCacheKey(path, info.ModTime().UnixNano(), size)
if res, ok := probeCache.Get(cacheKey); ok {
cacheKey := mediacache.FromFileInfo(path, info).String()
if res, ok := mediacache.Probes.Get(cacheKey); ok {
return res, true
}
// Coalesce concurrent probes of the same file into one container read.
v, err, _ := probeGroup.Do(cacheKey, func() (any, error) {
res, ok := parseProbe(path, f, size)
res, ok := parseProbe(path, f, info.Size())
if !ok {
return nil, errProbeUnsupported
}
probeCache.Set(cacheKey, res)
mediacache.Probes.Set(cacheKey, res)
return res, nil
})
if err != nil {
@@ -219,12 +270,6 @@ func probeFast(path string) (ProbeResult, bool) {
// no detail because parseProbe already logged the specific reason.
var errProbeUnsupported = errors.New("probe unsupported")
// probeCacheKey identifies a file version for caching: path plus modtime and size.
// An edited file gets a new key, so stale metadata is never served.
func probeCacheKey(path string, modTimeNanos, size int64) string {
return fmt.Sprintf("probe:%s:%d:%d", path, modTimeNanos, size)
}
// parseProbe reads a container and builds a ProbeResult. It reports false when
// go-media should defer to ffmpeg, having logged why.
func parseProbe(path string, f *os.File, size int64) (ProbeResult, bool) {
@@ -280,14 +325,18 @@ func durationFast(path string) (float64, bool) {
if !Enabled() {
return 0, false
}
f, size, err := openFile(path)
f, info, err := openFileInfo(path)
if err != nil {
logFallback("duration", path, err)
return 0, false
}
defer f.Close()
res, err := media.Duration(f, size)
cacheKey := mediacache.FromFileInfo(path, info).String()
if d, ok := mediacache.Durations.Get(cacheKey); ok {
return d, true
}
res, err := media.Duration(f, info.Size())
if err != nil {
logFallback("duration", path, err)
return 0, false
@@ -296,7 +345,9 @@ func durationFast(path string) (float64, bool) {
// Unknown duration is not an error; ffmpeg may still find one.
return 0, false
}
return res.Duration.Seconds(), true
d := res.Duration.Seconds()
mediacache.Durations.Set(cacheKey, d)
return d, true
}
// AudioInfo is the tag, lyrics availability, and cover art FileBrowser reads
@@ -311,38 +362,60 @@ type AudioInfo struct {
// is true. A missing cover is not an error.
// legacy is the dhowden tag read, run only in shadow mode.
func Audio(path string, withArt bool, legacy Legacy[AudioInfo]) (AudioInfo, bool) {
return timed("tags", path, legacy, audioDiffer(withArt), func() (AudioInfo, bool) { return audioFast(path, withArt) })
return timed("tags", path, legacy, audioDiffer(withArt), func() (AudioInfo, bool) { return audioFast(path, withArt, true) })
}
func audioFast(path string, withArt bool) (AudioInfo, bool) {
func audioFast(path string, withArt, withSidecar bool) (AudioInfo, bool) {
if !Enabled() {
return AudioInfo{}, false
}
f, size, err := openFile(path)
f, st, err := openFileInfo(path)
if err != nil {
logFallback("tags", path, err)
return AudioInfo{}, false
}
defer f.Close()
// One container parse gives tags, the embedded lyrics flag, and cover bytes.
// Separate ReadTags/HasLyrics/EmbeddedArtwork calls each re-parsed the file.
audio, err := media.InspectAudio(f, size, media.AudioOptions{WithArtwork: withArt})
if err != nil {
logFallback("tags", path, err)
return AudioInfo{}, false
size := st.Size()
cacheKey := tagsCacheKey(path, st.ModTime().UnixNano(), size)
var info AudioInfo
cached := false
if !withArt {
info, cached = tagsCache.Get(cacheKey)
}
info := AudioInfo{Tags: audio.Tags, HasLyrics: audio.HasEmbeddedLyrics}
if !info.HasLyrics {
if !cached {
// One container parse gives tags, the embedded lyrics flag, and cover bytes.
// Separate ReadTags/HasLyrics/EmbeddedArtwork calls each re-parsed the file.
audio, err := media.InspectAudio(f, size, media.AudioOptions{WithArtwork: withArt})
if err != nil {
logFallback("tags", path, err)
return AudioInfo{}, false
}
info = AudioInfo{Tags: audio.Tags, HasLyrics: audio.HasEmbeddedLyrics}
stored := info
stored.Tags.Raw = nil
tagsCache.Set(cacheKey, stored)
if withArt {
info.Artwork = audio.Artwork
}
}
if withSidecar && !info.HasLyrics {
// Sidecar lyrics are only checked when the file has no embedded lyrics.
_, _, info.HasLyrics = media.LyricsSidecar(path)
}
if withArt {
info.Artwork = audio.Artwork
}
return info, true
}
// tagsCacheKey identifies a file version for the tag cache: path plus modtime and size.
func tagsCacheKey(path string, modTimeNanos, size int64) string {
return fmt.Sprintf("tags:%s:%d:%d", path, modTimeNanos, size)
}
// VideoTags reads embedded tags and cover from a video container
func VideoTags(path string, withArt bool) (AudioInfo, bool) {
return timed("tags", path, nil, nil, func() (AudioInfo, bool) { return audioFast(path, withArt, false) })
}
// Lyrics returns lyrics from a sidecar file, or else from embedded tags. The
// format is "embedded" for tags or the sidecar extension. An empty result with
// ok=true means go-media checked and found no lyrics.
@@ -384,7 +457,9 @@ func lyricsFast(path string) (text, format string, ok bool) {
// EmbeddedSubtitles lists embedded subtitle tracks. go-media supports MKV, WebM,
// and MP4 only. Other containers are reported as unsupported, not as "no
// subtitles", because go-media returns an empty list for them.
// subtitles", because go-media returns an empty list for them. modtime is kept
// for API compatibility; the cache key comes from the file's current size and
// modification time.
// legacy is the ffmpeg subtitle detection, run only in shadow mode.
func EmbeddedSubtitles(path string, modtime time.Time, legacy Legacy[[]utils.SubtitleTrack]) ([]utils.SubtitleTrack, bool) {
return timed("subtitle list", path, legacy, subtitleListDiff, func() ([]utils.SubtitleTrack, bool) {
@@ -399,19 +474,20 @@ func embeddedSubtitlesFast(path string, modtime time.Time) ([]utils.SubtitleTrac
if !supportsVideoContainer(path) {
return nil, false
}
key := fmt.Sprintf("embedded_subtitles:%s:%d", path, modtime.Unix())
if cached, ok := subtitleListCache.Get(key); ok {
return cached, true
}
f, size, err := openFile(path)
_ = modtime
f, st, err := openFileInfo(path)
if err != nil {
logFallback("subtitle list", path, err)
return nil, false
}
defer f.Close()
info, err := media.Probe(f, size)
key := mediacache.FromFileInfo(path, st).String()
if cached, ok := mediacache.SubtitleTracks.Get(key); ok {
return cached, true
}
info, err := media.Probe(f, st.Size())
if err != nil {
logFallback("subtitle list", path, err)
return nil, false
@@ -433,7 +509,7 @@ func embeddedSubtitlesFast(path string, modtime time.Time) ([]utils.SubtitleTrac
})
}
tracks := ffmpeg.BuildSubtitleTracks(streams)
subtitleListCache.Set(key, tracks)
mediacache.SubtitleTracks.Set(key, tracks)
return tracks, true
}
@@ -453,24 +529,19 @@ func extractSubtitleFast(path string, streamIndex int) (string, bool) {
if !supportsVideoContainer(path) {
return "", false
}
st, err := os.Stat(path)
if err != nil {
logFallback("subtitle extract", path, err)
return "", false
}
key := fmt.Sprintf("subtitle_content:%s:%d:%d", path, streamIndex, st.ModTime().Unix())
if content, ok := subtitleContentCache.Get(key); ok {
return content, true
}
f, size, err := openFile(path)
f, st, err := openFileInfo(path)
if err != nil {
logFallback("subtitle extract", path, err)
return "", false
}
defer f.Close()
vtt, err := media.ExtractEmbeddedSubtitle(f, size, streamIndex)
key := mediacache.FromFileInfo(path, st).Stream(streamIndex)
if content, ok := mediacache.SubtitleContent.Get(key); ok {
return content, true
}
vtt, err := media.ExtractEmbeddedSubtitle(f, st.Size(), streamIndex)
if err != nil {
logFallback("subtitle extract", path, err)
return "", false
@@ -479,7 +550,7 @@ func extractSubtitleFast(path string, streamIndex int) (string, bool) {
logFallback("subtitle extract", path, errors.New("empty subtitle output"))
return "", false
}
subtitleContentCache.Set(key, vtt)
mediacache.SubtitleContent.Set(key, vtt)
return vtt, true
}
@@ -501,7 +572,7 @@ func videoThumbnailFast(path string, width, height, quality int, seekPercent flo
if !Enabled() {
return nil, false
}
if !supportsVideoContainer(path) {
if !supportsVideoThumbnail(path) {
return nil, false
}
f, size, err := openFile(path)
+66 -13
View File
@@ -11,6 +11,8 @@ import (
"testing"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/stretchr/testify/require"
)
@@ -131,27 +133,78 @@ func TestProbeUsesCache(t *testing.T) {
require.NoError(t, err)
require.NoError(t, os.WriteFile(path, data, 0o644))
st, err := os.Stat(path)
key, err := mediacache.Stat(path)
require.NoError(t, err)
key := probeCacheKey(path, st.ModTime().UnixNano(), st.Size())
sentinel := ProbeResult{Duration: 123, VideoCodec: "sentinel", AudioCodec: "sentinel", FormatName: "sentinel"}
probeCache.Set(key, sentinel)
t.Cleanup(func() { probeCache.Delete(key) })
// The ffmpeg probe stores into the same shared cache, so this also stands in
// for a value ffprobe produced earlier.
mediacache.Probes.Set(key.String(), sentinel)
t.Cleanup(func() { mediacache.Probes.Delete(key.String()) })
got, ok := Probe(path, nil)
require.True(t, ok)
require.Equal(t, sentinel, got, "probe must return the cached value for an unchanged file")
// An edit (new size and mtime) must miss the cached value.
require.NoError(t, os.WriteFile(path, append(data, 0), 0o644))
future := time.Now().Add(time.Hour)
require.NoError(t, os.Chtimes(path, future, future))
got, ok = Probe(path, nil)
require.True(t, ok)
require.NotEqual(t, sentinel, got, "an edited file must be probed again")
edited, err := mediacache.Stat(path)
require.NoError(t, err)
t.Cleanup(func() { mediacache.Probes.Delete(edited.String()) })
}
// TestProbeCacheKeyTracksFileIdentity checks that path, modtime, and size all
// participate in the cache key, so an edited file is never served stale metadata.
func TestProbeCacheKeyTracksFileIdentity(t *testing.T) {
base := probeCacheKey("f.mp4", 100, 200)
require.Equal(t, base, probeCacheKey("f.mp4", 100, 200), "identical identity must hash the same")
require.NotEqual(t, base, probeCacheKey("f.mp4", 101, 200), "modtime change must change the key")
require.NotEqual(t, base, probeCacheKey("f.mp4", 100, 201), "size change must change the key")
require.NotEqual(t, base, probeCacheKey("g.mp4", 100, 200), "path change must change the key")
require.NotEqual(t, base, probeCacheKey("f.mp4", 100, 201), "key must not collide across fields")
// TestDurationUsesSharedCache checks that Duration reads and fills the shared
// cache that ffmpeg's GetMediaDuration also uses.
func TestDurationUsesSharedCache(t *testing.T) {
withGate(t, false)
path := filepath.Join(t.TempDir(), "sample.mp4")
data, err := os.ReadFile(fixtureMP4)
require.NoError(t, err)
require.NoError(t, os.WriteFile(path, data, 0o644))
key, err := mediacache.Stat(path)
require.NoError(t, err)
t.Cleanup(func() { mediacache.Durations.Delete(key.String()) })
d, ok := Duration(path, nil)
require.True(t, ok)
cached, found := mediacache.Durations.Get(key.String())
require.True(t, found, "go-media duration must be stored in the shared cache")
require.Equal(t, d, cached)
mediacache.Durations.Set(key.String(), 4242)
d, ok = Duration(path, nil)
require.True(t, ok)
require.Equal(t, 4242.0, d, "a cached duration (from either backend) must be reused")
}
// TestSubtitleCachesUseFileIdentity checks the subtitle list cache is keyed by
// the file's real size and mtime, not by the caller's modtime argument.
func TestSubtitleCachesUseFileIdentity(t *testing.T) {
withGate(t, false)
path := filepath.Join(t.TempDir(), "sample.mp4")
data, err := os.ReadFile(fixtureMP4)
require.NoError(t, err)
require.NoError(t, os.WriteFile(path, data, 0o644))
key, err := mediacache.Stat(path)
require.NoError(t, err)
sentinel := []utils.SubtitleTrack{{Name: "sentinel"}}
mediacache.SubtitleTracks.Set(key.String(), sentinel)
t.Cleanup(func() { mediacache.SubtitleTracks.Delete(key.String()) })
// The modtime argument is ignored for the key.
got, ok := EmbeddedSubtitles(path, time.Unix(1, 0), nil)
require.True(t, ok)
require.Equal(t, sentinel, got)
mediacache.SubtitleContent.Set(key.Stream(7), "WEBVTT\n\nsentinel")
t.Cleanup(func() { mediacache.SubtitleContent.Delete(key.Stream(7)) })
vtt, ok := ExtractSubtitle(path, 7, nil)
require.True(t, ok)
require.Equal(t, "WEBVTT\n\nsentinel", vtt)
}
func TestProbeWebMMatchesFFprobe(t *testing.T) {
+101
View File
@@ -0,0 +1,101 @@
package gomedia
import (
"context"
"fmt"
"math"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/go-media/pkg/media"
)
// heicRotationShadowTimeout bounds the dev-mode ffmpeg comparison probe.
const heicRotationShadowTimeout = time.Minute
// HEICDisplayRotation returns the display rotation that a HEIC/HEIF/AVIF
// file's irot property asks for, read from the item properties without
// decoding. The value is degrees counter-clockwise in (-180, 180], the same
// number ffmpeg prints as "Display Matrix: rotation of ..." for the file. A
// file with no irot returns 0 and ok=true. ok is false when go-media is
// disabled or cannot read the item properties; the caller then runs the ffmpeg
// probe.
//
// It is installed as ffmpeg's HEIC display rotation reader. In shadow mode the
// ffmpeg stderr probe runs for comparison.
func HEICDisplayRotation(path string) (float64, bool) {
return timed("heic rotation", path, heicRotationLegacy(path), heicRotationDiff, func() (float64, bool) {
return heicDisplayRotationFast(path)
})
}
// heicRotationLegacy returns the ffmpeg stderr probe for shadow mode, or nil
// when ffmpeg is not available.
func heicRotationLegacy(path string) Legacy[float64] {
if !ShadowEnabled() {
return nil
}
svc := ffmpeg.Get()
if svc == nil {
return nil
}
return func() (float64, bool) {
ctx, cancel := context.WithTimeout(context.Background(), heicRotationShadowTimeout)
defer cancel()
return svc.GetHEICDisplayMatrixRotation(ctx, path)
}
}
// heicDisplayRotationFast reads the primary item's irot with go-media and
// converts it to ffmpeg's display rotation.
func heicDisplayRotationFast(path string) (float64, bool) {
if !Enabled() {
return 0, false
}
// No extension check: the resolved path can have another name (symlink
// target). go-media sniffs the content and reports other files unsupported.
f, size, err := openFile(path)
if err != nil {
logFallback("heic rotation", path, err)
return 0, false
}
defer f.Close()
t, err := media.HEIFTransform(f, size)
if err != nil {
logFallback("heic rotation", path, err)
return 0, false
}
return displayRotation(t), true
}
// displayRotation converts go-media's counter-clockwise irot angle to ffmpeg's
// display matrix convention: counter-clockwise degrees normalized to
// (-180, 180]. ffmpeg builds the matrix with av_display_rotation_set(-irot)
// and prints av_display_rotation_get, so irot 270 is reported as -90. A mirror
// does not change the rotation value.
func displayRotation(t media.ImageTransform) float64 {
r := ((t.Rotation % 360) + 360) % 360
if r > 180 {
r -= 360
}
return float64(r)
}
// heicRotationDiff compares go-media's display rotation with ffmpeg's. 180 and
// -180 are the same rotation.
func heicRotationDiff(fast, legacy float64) []string {
norm := func(v float64) float64 {
v = math.Mod(v, 360)
if v <= -180 {
v += 360
} else if v > 180 {
v -= 360
}
return v
}
if math.Abs(norm(fast)-norm(legacy)) > 0.5 {
return []string{fmt.Sprintf("display rotation go-media=%.2f ffmpeg=%.2f", fast, legacy)}
}
return nil
}
@@ -0,0 +1,82 @@
package gomedia
import (
"math"
"os"
"path/filepath"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/go-media/pkg/media"
"github.com/stretchr/testify/require"
)
// fixtureAVIFRotated has irot=1 (90° counter-clockwise) and imir axis 0.
const fixtureAVIFRotated = "testdata/irot1_imir0.avif"
// TestDisplayRotation checks the irot to ffmpeg display rotation mapping.
func TestDisplayRotation(t *testing.T) {
cases := []struct {
rotation int
want float64
}{
{0, 0}, {90, 90}, {180, 180}, {270, -90}, {-90, -90}, {360, 0}, {450, 90},
}
for _, c := range cases {
require.Equal(t, c.want, displayRotation(media.ImageTransform{Rotation: c.rotation}), "rotation %d", c.rotation)
}
require.Equal(t, 90.0, displayRotation(media.ImageTransform{Rotation: 90, Mirror: true}), "a mirror does not change the rotation")
}
// TestHEICDisplayRotationReadsIrot checks the rotation read from a fixture
// with irot=1.
func TestHEICDisplayRotationReadsIrot(t *testing.T) {
withGate(t, false)
got, ok := HEICDisplayRotation(fixtureAVIFRotated)
require.True(t, ok)
require.Equal(t, 90.0, got)
}
// TestHEICDisplayRotationFallsBack checks that non-HEIF input, broken input,
// and the Always use ffmpeg setting leave the answer to ffmpeg.
func TestHEICDisplayRotationFallsBack(t *testing.T) {
withGate(t, false)
_, ok := HEICDisplayRotation(fixtureMP4)
require.False(t, ok, "a non-HEIF file must fall back to the ffmpeg probe")
junk := filepath.Join(t.TempDir(), "junk.heic")
require.NoError(t, os.WriteFile(junk, []byte("not heif"), 0o644))
_, ok = HEICDisplayRotation(junk)
require.False(t, ok)
withGate(t, true)
_, ok = HEICDisplayRotation(fixtureAVIFRotated)
require.False(t, ok, "the Always use ffmpeg setting must skip go-media")
}
// TestHEICDisplayRotationInstalled checks that loading this package installs
// go-media as the ffmpeg package's rotation reader.
func TestHEICDisplayRotationInstalled(t *testing.T) {
withGate(t, false)
var svc *ffmpeg.Service // nil: the ffmpeg probe cannot run, so a result proves the reader answered
got, ok := svc.HEICDisplayRotation(t.Context(), fixtureAVIFRotated)
require.True(t, ok)
require.Equal(t, 90.0, got)
}
// TestHEICRotationDiff checks the shadow comparison of display rotations.
func TestHEICRotationDiff(t *testing.T) {
require.Empty(t, heicRotationDiff(-90, -90))
require.Empty(t, heicRotationDiff(180, -180))
require.Empty(t, heicRotationDiff(0, math.Copysign(0, -1)))
require.Len(t, heicRotationDiff(90, -90), 1)
}
// TestAVIFDecodesThroughHEICPath checks that an AVIF file is accepted by the
// HEIC decode path.
func TestAVIFDecodesThroughHEICPath(t *testing.T) {
withGate(t, false)
data, ok := HEICJPEG(fixtureAVIFRotated, 80, nil)
require.True(t, ok)
require.NotEmpty(t, data)
}
+145
View File
@@ -0,0 +1,145 @@
package gomedia
import (
"context"
"fmt"
"io"
"math"
"sort"
"github.com/gtsteffaniak/go-media/pkg/media"
)
// keyframeTimeDedupeSec matches go-ffmpeg's ffprobe keyframe probe, which
// drops keyframes within 1 ms of the previous one.
const keyframeTimeDedupeSec = 0.001
// keyframeShadowTolerance is how far apart two backends' keyframe times can be
// before the shadow comparison reports them. ffprobe prints six decimals.
const keyframeShadowTolerance = 0.002
// KeyframeTimes returns the presentation times, in seconds, of the first video
// stream's keyframes, for HLS segment planning. The result is sorted, has no
// negative times, and drops keyframes within 1 ms of the previous one, the same
// as go-ffmpeg's ffprobe probe. go-media reads only the container index (MP4
// stss/stts/ctts, Matroska Cues or block headers), so it is much faster than
// demuxing every packet with ffprobe.
//
// ok is false when go-media is disabled, the container is not one go-media
// indexes, or the read fails; the caller then runs ffprobe. ctx cancels the
// file reads. legacy is the ffprobe probe, run only in shadow mode.
//
// go-media can report fewer keyframes than ffprobe (ffprobe's parsers also flag
// IDR frames the container index omits, and some Matroska muxers index only
// part of the keyframes in Cues). It never reports a keyframe ffprobe would not.
func KeyframeTimes(ctx context.Context, path string, legacy Legacy[[]float64]) ([]float64, bool) {
return timed("keyframe index", path, legacy, keyframeDiff, func() ([]float64, bool) {
return keyframeTimesFast(ctx, path, 0)
})
}
// KeyframeTimesWithin is KeyframeTimes limited to keyframes within windowSec
// seconds of the first keyframe. It is the pre-flight check that can rule out
// stream copy without reading the whole index (Matroska files without Cues).
// It has no shadow comparison.
func KeyframeTimesWithin(ctx context.Context, path string, windowSec float64) ([]float64, bool) {
return timed("keyframe preflight", path, nil, nil, func() ([]float64, bool) {
return keyframeTimesFast(ctx, path, windowSec)
})
}
// keyframeTimesFast reads the keyframe index with go-media. windowSec > 0
// limits it to keyframes within that many seconds of the first one.
func keyframeTimesFast(ctx context.Context, path string, windowSec float64) ([]float64, bool) {
if !Enabled() {
return nil, false
}
if !supportsVideoContainer(path) {
return nil, false
}
if err := ctx.Err(); err != nil {
return nil, false
}
f, size, err := openFile(path)
if err != nil {
logFallback("keyframe index", path, err)
return nil, false
}
defer f.Close()
r := ctxReaderAt{ctx: ctx, r: f}
var times []float64
if windowSec > 0 {
times, err = media.KeyframeTimesWithin(r, size, windowSec)
} else {
times, err = media.KeyframeTimes(r, size)
}
if err != nil {
if ctx.Err() == nil {
logFallback("keyframe index", path, err)
}
return nil, false
}
times = dedupeKeyframeTimes(times)
if len(times) == 0 {
logUnsupported("keyframe index", path, "no keyframes in container index")
return nil, false
}
return times, true
}
// dedupeKeyframeTimes sorts times, drops negative values, and drops times
// within keyframeTimeDedupeSec of the previous kept time, as go-ffmpeg does for
// ffprobe output.
func dedupeKeyframeTimes(times []float64) []float64 {
sorted := append([]float64(nil), times...)
sort.Float64s(sorted)
out := make([]float64, 0, len(sorted))
for _, t := range sorted {
if t < 0 || math.IsNaN(t) || math.IsInf(t, 0) {
continue
}
if len(out) == 0 || t > out[len(out)-1]+keyframeTimeDedupeSec {
out = append(out, t)
}
}
return out
}
// ctxReaderAt fails reads once ctx is done, so a cancelled request or the plan
// timeout stops a long index walk on a slow share.
type ctxReaderAt struct {
ctx context.Context
r io.ReaderAt
}
// ReadAt reads from the wrapped reader unless the context is done.
func (c ctxReaderAt) ReadAt(p []byte, off int64) (int, error) {
if err := c.ctx.Err(); err != nil {
return 0, err
}
return c.r.ReadAt(p, off)
}
// keyframeDiff compares go-media's keyframe index with ffprobe's. go-media
// reporting a subset of ffprobe's keyframes is expected (parser-flagged IDR
// frames, partial Cues) and is logged at info level.
func keyframeDiff(fast, legacy []float64) []string {
missing := 0
j := 0
for _, t := range fast {
for j < len(legacy) && legacy[j] < t-keyframeShadowTolerance {
j++
missing++
}
if j >= len(legacy) || math.Abs(legacy[j]-t) > keyframeShadowTolerance {
return []string{fmt.Sprintf("keyframe at %.6fs from go-media not in ffmpeg index (go-media=%d ffmpeg=%d keyframes)", t, len(fast), len(legacy))}
}
j++
}
missing += len(legacy) - j
if missing > 0 {
return []string{fmt.Sprintf("%sgo-media found %d of %d ffmpeg keyframes (keyframes outside the container index)", expectedDiffPrefix, len(fast), len(legacy))}
}
return nil
}
+176
View File
@@ -0,0 +1,176 @@
package gomedia
import (
"context"
"math"
"os"
"os/exec"
"path/filepath"
"sort"
"strconv"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
// ffprobeKeyframes runs the same packet probe as go-ffmpeg's
// ProbeVideoKeyframeTimes: key packets of v:0, pts_time falling back to
// dts_time, sorted and de-duplicated within 1 ms.
func ffprobeKeyframes(t *testing.T, path string) []float64 {
t.Helper()
requireTool(t, "ffprobe")
out, err := exec.Command("ffprobe", "-v", "error", "-select_streams", "v:0",
"-show_entries", "packet=pts_time,dts_time,flags", "-of", "csv=p=0", path).Output()
require.NoError(t, err)
var times []float64
for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {
f := strings.Split(strings.TrimSpace(line), ",")
if len(f) < 3 || !strings.Contains(f[2], "K") {
continue
}
v, perr := strconv.ParseFloat(f[0], 64)
if perr != nil {
v, perr = strconv.ParseFloat(f[1], 64)
}
if perr == nil {
times = append(times, v)
}
}
sort.Float64s(times)
return dedupeKeyframeTimes(times)
}
// encodeKeyframeClip makes a short clip with a fixed GOP. B-frames give MP4
// files a ctts table and an edit list, the hard case for the timestamp model.
func encodeKeyframeClip(t *testing.T, name string, args ...string) string {
t.Helper()
requireTool(t, "ffmpeg")
path := filepath.Join(t.TempDir(), name)
full := []string{"-v", "error", "-y", "-f", "lavfi", "-i", "testsrc2=size=160x120:rate=25:duration=6"}
full = append(full, args...)
full = append(full, path)
if out, err := exec.Command("ffmpeg", full...).CombinedOutput(); err != nil {
t.Skipf("ffmpeg cannot encode %s: %v: %s", name, err, out)
}
return path
}
// requireSameKeyframes fails unless got matches want within the shadow
// tolerance.
func requireSameKeyframes(t *testing.T, want, got []float64) {
t.Helper()
require.Equal(t, len(want), len(got), "keyframe count: ffprobe=%v go-media=%v", want, got)
for i := range want {
require.InDelta(t, want[i], got[i], keyframeShadowTolerance, "keyframe %d: ffprobe=%v go-media=%v", i, want, got)
}
}
// TestKeyframeTimesMatchFFprobe compares go-media keyframe indexes with
// ffprobe for fixtures and encoded clips, including B-frame MP4/MOV.
func TestKeyframeTimesMatchFFprobe(t *testing.T) {
if testing.Short() {
t.Skip("encodes clips with ffmpeg")
}
withGate(t, false)
x264 := []string{"-c:v", "libx264", "-g", "50", "-keyint_min", "50", "-sc_threshold", "0", "-bf", "2", "-pix_fmt", "yuv420p"}
cases := map[string]string{
"fixture.mp4": fixtureMP4,
"fixture.webm": fixtureWebM,
"bframes.mp4": encodeKeyframeClip(t, "bframes.mp4", x264...),
"bframes.mov": encodeKeyframeClip(t, "bframes.mov", x264...),
"bframes.mkv": encodeKeyframeClip(t, "bframes.mkv", x264...),
"mpeg4.m4v": encodeKeyframeClip(t, "mpeg4.m4v", "-c:v", "mpeg4", "-g", "30", "-f", "mp4"),
}
for name, path := range cases {
t.Run(name, func(t *testing.T) {
want := ffprobeKeyframes(t, path)
got, ok := KeyframeTimes(context.Background(), path, nil)
require.True(t, ok)
requireSameKeyframes(t, want, got)
within, ok := KeyframeTimesWithin(context.Background(), path, 2.5)
require.True(t, ok)
require.NotEmpty(t, within)
require.LessOrEqual(t, len(within), len(got))
for i := range within {
require.Equal(t, got[i], within[i])
require.LessOrEqual(t, within[i]-got[0], 2.5+1e-9)
}
})
}
}
// TestKeyframeTimesFallsBack checks the cases that must leave the index to
// ffprobe.
func TestKeyframeTimesFallsBack(t *testing.T) {
withGate(t, false)
dir := t.TempDir()
// Not an indexed container: no file open, ffprobe answers.
ts := filepath.Join(dir, "clip.ts")
require.NoError(t, os.WriteFile(ts, []byte{0x47}, 0o644))
_, ok := KeyframeTimes(context.Background(), ts, nil)
require.False(t, ok)
// Indexed extension but not a real container.
junk := filepath.Join(dir, "junk.mp4")
require.NoError(t, os.WriteFile(junk, []byte("not a video"), 0o644))
_, ok = KeyframeTimes(context.Background(), junk, nil)
require.False(t, ok)
// Missing file.
_, ok = KeyframeTimes(context.Background(), filepath.Join(dir, "missing.mp4"), nil)
require.False(t, ok)
// Cancelled context.
ctx, cancel := context.WithCancel(context.Background())
cancel()
_, ok = KeyframeTimes(ctx, fixtureMP4, nil)
require.False(t, ok)
}
// TestKeyframeTimesGate checks that the Always use ffmpeg setting disables
// the go-media keyframe index.
func TestKeyframeTimesGate(t *testing.T) {
withGate(t, true)
_, ok := KeyframeTimes(context.Background(), fixtureMP4, nil)
require.False(t, ok, "KeyframeTimes must skip go-media when the gate is on")
_, ok = KeyframeTimesWithin(context.Background(), fixtureMP4, 10)
require.False(t, ok)
}
// TestDedupeKeyframeTimes checks sorting, filtering, and the 1 ms dedupe.
func TestDedupeKeyframeTimes(t *testing.T) {
got := dedupeKeyframeTimes([]float64{2, -1, 0, 0.0005, 1, 1.001, 1.0021, math.NaN(), math.Inf(1)})
require.Equal(t, []float64{0, 1, 1.0021, 2}, got)
require.Empty(t, dedupeKeyframeTimes(nil))
}
// TestCtxReaderAt checks that reads stop once the context is cancelled.
func TestCtxReaderAt(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
r := ctxReaderAt{ctx: ctx, r: strings.NewReader("abcdef")}
buf := make([]byte, 3)
n, err := r.ReadAt(buf, 1)
require.NoError(t, err)
require.Equal(t, "bcd", string(buf[:n]))
cancel()
_, err = r.ReadAt(buf, 0)
require.ErrorIs(t, err, context.Canceled)
}
// TestKeyframeDiff checks the shadow comparison: a subset is expected, an
// extra keyframe is a real difference.
func TestKeyframeDiff(t *testing.T) {
require.Empty(t, keyframeDiff([]float64{0, 2, 4}, []float64{0, 2.001, 4}))
subset := keyframeDiff([]float64{0, 4}, []float64{0, 2, 4, 6})
require.Len(t, subset, 1)
require.True(t, strings.HasPrefix(subset[0], expectedDiffPrefix), subset[0])
require.Contains(t, subset[0], "2 of 4")
extra := keyframeDiff([]float64{0, 1, 2}, []float64{0, 2})
require.Len(t, extra, 1)
require.False(t, strings.HasPrefix(extra[0], expectedDiffPrefix), extra[0])
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.7 KiB

+109
View File
@@ -0,0 +1,109 @@
// Package mediacache holds the media metadata caches shared by the ffmpeg and
// go-media code paths. Both backends read and write the same entries, so a
// value found by one is reused by the other instead of being probed again.
//
// Every entry is keyed by file identity: path, size, and modification time in
// nanoseconds. An edited file gets a new key, so a stale value is never served
// for it; the old entry ages out after TTL.
package mediacache
import (
"context"
"fmt"
"io/fs"
"os"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/go-cache/cache"
)
// TTL is how long a cached media value is kept after it is stored.
const TTL = 24 * time.Hour
// ProbeInfo is the container summary both backends produce: ffprobe through
// go-ffmpeg, and go-media. FormatName uses ffprobe's format_name strings.
type ProbeInfo struct {
Duration float64
VideoCodec string
AudioCodec string
FormatName string
}
var (
// Probes caches container probes (duration, first video and audio codec,
// format name).
Probes = cache.NewCache[ProbeInfo](TTL)
// Durations caches media length in seconds.
Durations = cache.NewCache[float64](TTL)
// SubtitleTracks caches the embedded subtitle track list of a video.
SubtitleTracks = cache.NewCache[[]utils.SubtitleTrack](TTL)
// SubtitleContent caches one embedded subtitle stream converted to WebVTT.
// Use Key.Stream for its key.
SubtitleContent = cache.NewCache[string](TTL)
)
// Key identifies one version of a file. The zero Key is never produced for a
// real file, because Path is always set.
type Key struct {
Path string
Size int64
ModTime int64 // modification time in Unix nanoseconds
}
// FromFileInfo returns the Key for path using an already-read FileInfo, so
// callers that stat or open the file anyway do not stat it twice.
func FromFileInfo(path string, info fs.FileInfo) Key {
return Key{Path: path, Size: info.Size(), ModTime: info.ModTime().UnixNano()}
}
// Stat returns the Key for path. It follows symlinks, like os.Stat.
func Stat(path string) (Key, error) {
info, err := os.Stat(path)
if err != nil {
return Key{}, err
}
return FromFileInfo(path, info), nil
}
// String returns the cache key for k. The path is quoted so a path containing
// the separator cannot collide with another file's key.
func (k Key) String() string {
return fmt.Sprintf("%q:%d:%d", k.Path, k.Size, k.ModTime)
}
// Stream returns the cache key for one stream of k, used by SubtitleContent.
func (k Key) Stream(index int) string {
return fmt.Sprintf("%s:%d", k.String(), index)
}
// ClearAll drops every cached media value. It is used when the backend that
// produced the values is switched (the admin "Always use ffmpeg" setting), so
// the newly selected backend answers fresh.
func ClearAll() {
Probes.ClearAll()
Durations.ClearAll()
SubtitleTracks.ClearAll()
SubtitleContent.ClearAll()
}
// bypassKey is the context key set by WithoutCache.
type bypassKey struct{}
// WithoutCache returns a context under which the ffmpeg code paths neither read
// nor write the shared caches. Dev-mode shadow runs use it, so the ffmpeg result
// they compare against go-media is computed fresh rather than read back from the
// value go-media just stored.
func WithoutCache(ctx context.Context) context.Context {
return context.WithValue(ctx, bypassKey{}, true)
}
// Bypassed reports whether ctx was returned by WithoutCache. A nil context is
// not bypassed.
func Bypassed(ctx context.Context) bool {
if ctx == nil {
return false
}
v, _ := ctx.Value(bypassKey{}).(bool)
return v
}
@@ -0,0 +1,157 @@
package mediacache
import (
"context"
"io/fs"
"os"
"path/filepath"
"testing"
"time"
)
// TestStatKeyChangesWithSizeAndModTime checks on disk that a size change
// or an mtime change gives a new key. The mtimes differ by whole seconds so
// the check holds on filesystems with coarse timestamps; the 1 ns case is
// covered by TestFromFileInfoNanosecondModTime.
func TestStatKeyChangesWithSizeAndModTime(t *testing.T) {
path := filepath.Join(t.TempDir(), "a.mp4")
if err := os.WriteFile(path, []byte("one"), 0o600); err != nil {
t.Fatal(err)
}
k1, err := Stat(path)
if err != nil {
t.Fatal(err)
}
if k1.Path != path || k1.Size != 3 || k1.ModTime == 0 {
t.Fatalf("unexpected key %+v", k1)
}
// Same size, different mtime.
mt := time.Unix(1_700_000_000, 0)
if err = os.Chtimes(path, mt, mt); err != nil {
t.Fatal(err)
}
k2, err := Stat(path)
if err != nil {
t.Fatal(err)
}
if k2.String() == k1.String() {
t.Fatalf("mtime change kept key %s", k2)
}
mt2 := mt.Add(time.Second)
if err = os.Chtimes(path, mt2, mt2); err != nil {
t.Fatal(err)
}
k3, _ := Stat(path)
if k3.String() == k2.String() {
t.Fatalf("1s mtime change kept key %s", k3)
}
// Same mtime, different size.
if err = os.WriteFile(path, []byte("three"), 0o600); err != nil {
t.Fatal(err)
}
if err = os.Chtimes(path, mt2, mt2); err != nil {
t.Fatal(err)
}
k4, _ := Stat(path)
if k4.String() == k3.String() {
t.Fatalf("size change kept key %s", k4)
}
}
// stubFileInfo is an fs.FileInfo with a fixed size and mtime.
type stubFileInfo struct {
size int64
modTime time.Time
}
// Name implements fs.FileInfo.
func (s stubFileInfo) Name() string { return "a.mp4" }
// Size implements fs.FileInfo.
func (s stubFileInfo) Size() int64 { return s.size }
// Mode implements fs.FileInfo.
func (s stubFileInfo) Mode() fs.FileMode { return 0o600 }
// ModTime implements fs.FileInfo.
func (s stubFileInfo) ModTime() time.Time { return s.modTime }
// IsDir implements fs.FileInfo.
func (s stubFileInfo) IsDir() bool { return false }
// Sys implements fs.FileInfo.
func (s stubFileInfo) Sys() any { return nil }
// TestFromFileInfoNanosecondModTime checks that mtimes 1 ns apart give
// different keys, independent of what a filesystem can store.
func TestFromFileInfoNanosecondModTime(t *testing.T) {
a := FromFileInfo("/m/a.mp4", stubFileInfo{size: 3, modTime: time.Unix(1_700_000_000, 123)})
b := FromFileInfo("/m/a.mp4", stubFileInfo{size: 3, modTime: time.Unix(1_700_000_000, 124)})
if a.String() == b.String() {
t.Fatalf("1ns mtime change kept key %s", a)
}
if a.ModTime != time.Unix(1_700_000_000, 123).UnixNano() {
t.Fatalf("ModTime = %d, want nanoseconds", a.ModTime)
}
}
// TestStatMissingFile checks that Stat reports a missing file.
func TestStatMissingFile(t *testing.T) {
if _, err := Stat(filepath.Join(t.TempDir(), "missing")); err == nil {
t.Fatal("expected an error for a missing file")
}
}
// TestKeyStringDoesNotCollide checks that keys of different files, and of
// different streams of one file, never collide.
func TestKeyStringDoesNotCollide(t *testing.T) {
a := Key{Path: "a:1", Size: 2, ModTime: 3}
b := Key{Path: "a", Size: 1, ModTime: 2}
if a.String() == b.String() {
t.Fatalf("keys collide: %s", a)
}
if a.Stream(1) == a.Stream(2) || a.Stream(1) == a.String() {
t.Fatalf("stream keys collide: %s %s", a.Stream(1), a.Stream(2))
}
}
// TestClearAll checks that ClearAll empties every shared cache.
func TestClearAll(t *testing.T) {
k := Key{Path: "/clear", Size: 1, ModTime: 1}
Probes.Set(k.String(), ProbeInfo{Duration: 1})
Durations.Set(k.String(), 1)
SubtitleContent.Set(k.Stream(0), "x")
ClearAll()
if _, ok := Probes.Get(k.String()); ok {
t.Fatal("probe survived ClearAll")
}
if _, ok := Durations.Get(k.String()); ok {
t.Fatal("duration survived ClearAll")
}
if _, ok := SubtitleContent.Get(k.Stream(0)); ok {
t.Fatal("subtitle content survived ClearAll")
}
}
// TestWithoutCache checks the cache bypass marker and that derived contexts
// keep it.
func TestWithoutCache(t *testing.T) {
if Bypassed(context.Background()) {
t.Fatal("background context must not bypass")
}
//nolint:staticcheck // nil is part of the documented contract.
if Bypassed(nil) {
t.Fatal("nil context must not bypass")
}
ctx := WithoutCache(context.Background())
if !Bypassed(ctx) {
t.Fatal("WithoutCache context must bypass")
}
child, cancel := context.WithCancel(ctx)
defer cancel()
if !Bypassed(child) {
t.Fatal("bypass must survive derived contexts")
}
}
+121
View File
@@ -0,0 +1,121 @@
package preview
import (
"fmt"
"image"
"image/color"
"runtime"
"testing"
"github.com/kovidgoyal/imaging"
"github.com/stretchr/testify/require"
)
// patternYCbCr is a 4:2:0 image (what a camera JPEG decodes to) with a pattern that differs in
// every row and column, so a crop off by one pixel shows up in the resized output.
func patternYCbCr(w, h int) *image.YCbCr {
return patternYCbCrRatio(w, h, image.YCbCrSubsampleRatio420)
}
// patternYCbCrRatio is patternYCbCr with the given chroma subsampling.
func patternYCbCrRatio(w, h int, r image.YCbCrSubsampleRatio) *image.YCbCr {
img := image.NewYCbCr(image.Rect(0, 0, w, h), r)
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Y[img.YOffset(x, y)] = uint8((x*7 + y*13) % 251)
}
}
for i := range img.Cb {
img.Cb[i] = uint8((i*3 + i/img.CStride) % 256)
img.Cr[i] = uint8((i + 5*(i/img.CStride)) % 256)
}
return img
}
// patternNRGBA is an NRGBA image whose pixels differ in every row and column.
func patternNRGBA(w, h int) *image.NRGBA {
img := image.NewNRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.SetNRGBA(x, y, color.NRGBA{uint8(x % 256), uint8(y % 256), uint8((x + y) % 256), 255})
}
}
return img
}
// requireSamePixels fails unless want and got are the same size and every pixel is equal,
// comparing from each image's own origin.
func requireSamePixels(t *testing.T, want, got image.Image) {
t.Helper()
require.Equal(t, want.Bounds().Size(), got.Bounds().Size())
wb, gb := want.Bounds().Min, got.Bounds().Min
for y := 0; y < want.Bounds().Dy(); y++ {
for x := 0; x < want.Bounds().Dx(); x++ {
wr, wg, wbl, wa := want.At(wb.X+x, wb.Y+y).RGBA()
gr, gg, gbl, ga := got.At(gb.X+x, gb.Y+y).RGBA()
if wr != gr || wg != gg || wbl != gbl || wa != ga {
t.Fatalf("pixel (%d,%d) differs: imaging.Fill %v, fillCenter %v", x, y,
[4]uint32{wr, wg, wbl, wa}, [4]uint32{gr, gg, gbl, ga})
}
}
}
}
// fillCenter must produce exactly what imaging.Fill(…, imaging.Center, …) produces: the same
// pixels and the same image type, so the encoded preview is the same bytes.
func TestFillCenter_MatchesImagingFill(t *testing.T) {
sources := map[string]image.Image{
"ycbcr landscape": patternYCbCr(1200, 801),
"ycbcr portrait": patternYCbCr(601, 1000),
"nrgba square": patternNRGBA(700, 700),
"nrgba wide": patternNRGBA(1500, 300),
"small (falls back to imaging.Fill)": patternNRGBA(90, 60),
"exact target size": patternNRGBA(256, 256),
"offset origin": patternNRGBA(900, 700).SubImage(image.Rect(37, 11, 837, 611)),
}
// Every subsampling ratio, in shapes whose centered crop has an x (or y) offset and reaches
// the source's last row (or column) -- where a SubImage's buffer ends short of a full stride.
for _, r := range []image.YCbCrSubsampleRatio{
image.YCbCrSubsampleRatio444, image.YCbCrSubsampleRatio422, image.YCbCrSubsampleRatio420,
image.YCbCrSubsampleRatio440, image.YCbCrSubsampleRatio411, image.YCbCrSubsampleRatio410,
} {
for _, sz := range [][2]int{{600, 264}, {264, 600}, {601, 263}} {
sources[fmt.Sprintf("ycbcr %v %dx%d", r, sz[0], sz[1])] = patternYCbCrRatio(sz[0], sz[1], r)
}
}
sources["nycbcra 444 600x264"] = &image.NYCbCrA{YCbCr: *patternYCbCrRatio(600, 264, image.YCbCrSubsampleRatio444), A: make([]uint8, 600*264), AStride: 600}
// 264x264 is the crop of the 600x264 and 264x600 sources itself: nothing to scale.
targets := [][2]int{{256, 256}, {320, 180}, {180, 320}, {264, 264}}
filters := map[string]imaging.ResampleFilter{
"nearest": imaging.NearestNeighbor, "box": imaging.Box, "catmullrom": imaging.CatmullRom,
}
for name, src := range sources {
for _, tg := range targets {
for fname, f := range filters {
t.Run(fmt.Sprintf("%s/%dx%d/%s", name, tg[0], tg[1], fname), func(t *testing.T) {
want := imaging.Fill(src, tg[0], tg[1], imaging.Center, f)
got := fillCenter(src, tg[0], tg[1], f)
require.Equal(t, fmt.Sprintf("%T", want), fmt.Sprintf("%T", got), "image type")
requireSamePixels(t, want, got)
})
}
}
}
}
// A square thumbnail of a large photo must not allocate a full-resolution copy of it.
func TestFillCenter_NoFullSizeCopy(t *testing.T) {
const w, h = 4000, 3000
src := patternYCbCr(w, h)
fullCopy := uint64(h * h * 4) // the square crop imaging.Fill materializes as NRGBA
var before, after runtime.MemStats
runtime.ReadMemStats(&before)
out := fillCenter(src, 256, 256, imaging.Box)
runtime.ReadMemStats(&after)
require.Equal(t, image.Pt(256, 256), out.Bounds().Size())
allocated := after.TotalAlloc - before.TotalAlloc
require.Less(t, allocated, fullCopy/4,
"fillCenter allocated %d bytes; a copy of the crop would be %d", allocated, fullCopy)
}
+72 -2
View File
@@ -7,9 +7,11 @@ import (
"image"
"image/jpeg"
"io"
"math"
exif "github.com/dsoprea/go-exif/v3"
exifcommon "github.com/dsoprea/go-exif/v3/common"
"github.com/gtsteffaniak/filebrowser/backend/internal/embeddedpreview"
"github.com/kovidgoyal/imaging"
)
@@ -203,6 +205,50 @@ func (s *Service) ResizeWithSize(ctx context.Context, in io.Reader, out io.Write
return err
}
// fillCenter is imaging.Fill(img, width, height, imaging.Center, filter) without a full-size copy
// of the source. For sources of 100px or more, imaging.Fill first crops into a new NRGBA image
// (4 bytes per pixel of the cropped area, about 144 MB for a 48 MP photo) and only then resizes.
// Here the crop is a SubImage view into the decoded pixels, so only the resize allocates. The
// crop rectangle is computed as imaging's cropAndResize computes it, so the output is the same.
//
// Two cases still go through imaging.Fill: a crop that is already the target size (nothing to
// scale, and Fill's small copy is what keeps the output type the same), and 4:4:4 YCbCr sources,
// whose scanner in imaging v1.9.0 advances one full parent stride past the end of a SubImage's
// last row and panics when the crop has an x offset and reaches the bottom row.
func fillCenter(img image.Image, width, height int, filter imaging.ResampleFilter) image.Image {
b := img.Bounds()
sub, ok := img.(interface {
SubImage(image.Rectangle) image.Image
})
if !ok || is444(img) || width <= 0 || height <= 0 || b.Dx() < 100 || b.Dy() < 100 || (b.Dx() == width && b.Dy() == height) {
return imaging.Fill(img, width, height, imaging.Center, filter)
}
cropW, cropH := b.Dx(), b.Dy()
if float64(b.Dx())/float64(b.Dy()) < float64(width)/float64(height) {
cropH = int(math.Max(1, float64(b.Dx())*float64(height)/float64(width)) + 0.5)
} else {
cropW = int(math.Max(1, float64(b.Dy())*float64(width)/float64(height)) + 0.5)
}
if cropW == width && cropH == height {
return imaging.Fill(img, width, height, imaging.Center, filter)
}
x0 := b.Min.X + (b.Dx()-cropW)/2
y0 := b.Min.Y + (b.Dy()-cropH)/2
crop := image.Rect(x0, y0, x0+cropW, y0+cropH).Intersect(b)
return imaging.Resize(sub.SubImage(crop), width, height, filter)
}
// is444 reports whether img is a YCbCr image without chroma subsampling.
func is444(img image.Image) bool {
switch p := img.(type) {
case *image.YCbCr:
return p.SubsampleRatio == image.YCbCrSubsampleRatio444
case *image.NYCbCrA:
return p.SubsampleRatio == image.YCbCrSubsampleRatio444
}
return false
}
func (s *Service) resizeWithSize(in io.Reader, out io.Writer, fileSize int64, opts ResizeOptions) error {
// Set defaults
if opts.ResizeMode == 0 {
@@ -280,7 +326,7 @@ func (s *Service) resizeWithSize(in io.Reader, out io.Writer, fileSize int64, op
resampleFilter := opts.Quality.resampleFilter()
switch opts.ResizeMode {
case ResizeModeFill:
img = imaging.Fill(img, opts.Width, opts.Height, imaging.Center, resampleFilter)
img = fillCenter(img, opts.Width, opts.Height, resampleFilter)
case ResizeModeFit:
fallthrough
default:
@@ -406,8 +452,32 @@ func getEmbeddedThumbnail(in io.Reader) ([]byte, io.Reader, error) {
return nil, wrappedReader, exif.ErrNoThumbnail
}
// Read IFD1's Orientation first; fall back to IFD0's Orientation if not specified (per MWG guidelines)
orientStr := ""
tags1, err1 := ifd.FindTagWithName("Orientation")
if err1 == nil && len(tags1) > 0 {
orientStr, _ = tags1[0].FormatFirst()
}
if orientStr == "" {
tags0, err0 := index.RootIfd.FindTagWithName("Orientation")
if err0 == nil && len(tags0) > 0 {
orientStr, _ = tags0[0].FormatFirst()
}
}
thm, err := ifd.Thumbnail()
return thm, wrappedReader, err
if err != nil {
return nil, wrappedReader, err
}
if orientStr != "" && orientStr != "1" {
if !embeddedpreview.Usable(thm) {
return nil, wrappedReader, fmt.Errorf("embedded thumbnail exceeds decode limits")
}
thm = applyOrientationToPreviewBytes(thm, orientStr)
}
return thm, wrappedReader, nil
}
// CreateThumbnail decodes an image and creates a fixed-size thumbnail.
+38
View File
@@ -549,3 +549,41 @@ func TestReadImageFileDimensions(t *testing.T) {
require.Greater(t, width, 0)
require.Greater(t, height, 0)
}
func TestApplyOrientationToPreviewBytes(t *testing.T) {
t.Parallel()
// Generate a 100x60 test JPEG image in memory (landscape)
img := image.NewRGBA(image.Rect(0, 0, 100, 60))
var buf bytes.Buffer
err := jpeg.Encode(&buf, img, nil)
require.NoError(t, err)
rawBytes := buf.Bytes()
tests := []struct {
orientation string
wantWidth int
wantHeight int
}{
{orientation: "1", wantWidth: 100, wantHeight: 60},
{orientation: "Horizontal (normal)", wantWidth: 100, wantHeight: 60},
{orientation: "8", wantWidth: 60, wantHeight: 100},
{orientation: "Rotate 270 CW", wantWidth: 60, wantHeight: 100},
{orientation: "6", wantWidth: 60, wantHeight: 100},
{orientation: "Rotate 90 CW", wantWidth: 60, wantHeight: 100},
{orientation: "3", wantWidth: 100, wantHeight: 60},
{orientation: "Rotate 180", wantWidth: 100, wantHeight: 60},
}
for _, tt := range tests {
t.Run(tt.orientation, func(t *testing.T) {
rotated := applyOrientationToPreviewBytes(rawBytes, tt.orientation)
require.NotEmpty(t, rotated)
cfg, _, decodeErr := image.DecodeConfig(bytes.NewReader(rotated))
require.NoError(t, decodeErr)
require.Equal(t, tt.wantWidth, cfg.Width)
require.Equal(t, tt.wantHeight, cfg.Height)
})
}
}
+7 -4
View File
@@ -64,7 +64,7 @@ func NewPreviewGenerator(concurrencyLimit int, cacheDir string) *Service {
var fileCache diskcache.Interface
// Use file cache if cacheDir is specified
var err error
fileCache, err = diskcache.NewFileCache(actualCacheDir)
fileCache, err = diskcache.NewFileCacheWithLimit(actualCacheDir, max(settings.Config.Server.CacheDirMaxSizeMB, 0)*1024*1024)
if err != nil {
logger.Error("The cache dir could not be created. Make sure the user that you executed the program with has access to create directories in the local path. See ")
logger.Fatalf("failed to create file cache path, which is now require to run the server: %v", err)
@@ -272,8 +272,9 @@ func determinePreviewType(file iteminfo.ExtendedFileInfo) filePreviewType {
return previewTypeOffice
}
// Check HEIC with conversion enabled
if strings.HasPrefix(file.Type, "image/heic") &&
// Check HEIC with conversion enabled. AVIF is a HEIF container too and uses
// the same path (go-media decode, ffmpeg fallback) under the HEIC setting.
if (strings.HasPrefix(file.Type, "image/heic") || file.Type == "image/avif") &&
settings.CanConvertImage(string(settings.HEICImagePreview)) {
return previewTypeHEIC
}
@@ -503,7 +504,9 @@ func GeneratePreviewWithMD5(ctx context.Context, file iteminfo.ExtendedFileInfo,
}
// Enforce file size limit for image preview generation to prevent memory exhaustion
if strings.HasPrefix(file.Type, "image") && file.Size > iteminfo.LargeFileSizeThreshold {
// Camera RAW is exempt: its preview is the embedded JPEG and the RAW data is never decoded.
if strings.HasPrefix(file.Type, "image") && file.Size > iteminfo.LargeFileSizeThreshold &&
!iteminfo.IsRawImage(strings.ToLower(filepath.Ext(file.Name))) {
message := fmt.Sprintf("Image file too large for preview: %s (size: %d bytes, limit: %d bytes)",
file.Name, file.Size, iteminfo.LargeFileSizeThreshold)
logger.Warning(message)
@@ -0,0 +1,49 @@
package preview
import (
"testing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
// withHEICConversion sets ffmpeg availability and the HEIC preview setting for
// one test.
func withHEICConversion(t *testing.T, enabled bool) {
t.Helper()
prevAvail := settings.Env.FFmpegAvailable
prevPreview := settings.Config.Integrations.Media.Convert.ImagePreview
t.Cleanup(func() {
settings.Env.FFmpegAvailable = prevAvail
settings.Config.Integrations.Media.Convert.ImagePreview = prevPreview
})
settings.Env.FFmpegAvailable = true
settings.Config.Integrations.Media.Convert.ImagePreview = map[settings.ImagePreviewType]*bool{
settings.HEICImagePreview: &enabled,
}
}
// previewFile returns a file info with only a name and MIME type set.
func previewFile(name, mimeType string) iteminfo.ExtendedFileInfo {
var f iteminfo.ExtendedFileInfo
f.Name = name
f.Type = mimeType
return f
}
// TestDeterminePreviewTypeAVIF checks that AVIF uses the HEIF decode path when
// HEIC conversion is enabled, and the plain image path otherwise.
func TestDeterminePreviewTypeAVIF(t *testing.T) {
withHEICConversion(t, true)
if got := determinePreviewType(previewFile("a.avif", "image/avif")); got != previewTypeHEIC {
t.Fatalf("avif with HEIC conversion = %v, want previewTypeHEIC", got)
}
if got := determinePreviewType(previewFile("a.heic", "image/heic")); got != previewTypeHEIC {
t.Fatalf("heic with HEIC conversion = %v, want previewTypeHEIC", got)
}
withHEICConversion(t, false)
if got := determinePreviewType(previewFile("a.avif", "image/avif")); got != previewTypeImage {
t.Fatalf("avif without HEIC conversion = %v, want previewTypeImage", got)
}
}
+5 -2
View File
@@ -28,8 +28,11 @@ func (s *Service) GenerateVideoPreview(ctx context.Context, videoPath, previewSi
return nil, err
}
// go-media fast path: fit within the box, no crop. Falls back to ffmpeg when
// unsupported (e.g. HEVC, VP9, AV1) or failed.
// go-media fast path: fit within the box, no crop. go-media decodes H.264,
// HEVC, VP8, VP9, AV1, MPEG-4 Part 2, and WMV3 keyframes in pure Go. It falls
// back to ffmpeg when the container is not a go-media target (AVI, MPEG-TS,
// MPEG-PS, ...), the codec or profile is unsupported (WMV1/WMV2, VC-1
// Advanced, AV1 above 4K, ...), or the decode fails.
// In shadow mode (dev only) the ffmpeg preview also runs, for comparison.
// The shadow run can outlive the request, so it uses a context that is not cancelled with it.
shadowCtx := context.WithoutCancel(ctx)
+9
View File
@@ -19,6 +19,15 @@ func AccessPermitted(sourcePath string, indexPath utils.IndexPath, username stri
return accessDb.Permitted(sourcePath, indexPath, username)
}
// AccessPermittedAtRealPath reports whether username may access realPath after symlink
// resolution on the given index. When accessDb is not initialized, access is allowed.
func AccessPermittedAtRealPath(idx *indexing.Index, username, realPath string, isDir bool) bool {
if accessDb == nil {
return true
}
return accessDb.PermittedAtRealPath(idx, username, realPath, isDir)
}
func AllowUser(sourcePath string, indexPath utils.IndexPath, username string) error {
return accessDb.AllowUser(sourcePath, indexPath, username)
}
+12
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"reflect"
"slices"
"strings"
"sync"
@@ -297,6 +298,17 @@ func ApplyUserDefaults(u *users.User) {
applyToolAccessDefaults(u)
}
// PrefillUserProfileDefaults seeds u with the effective profile defaults so a JSON decode
// overlays only the keys the request sent (encoding/json leaves absent keys untouched).
// Permissions are left zero so omitted permissions stay distinguishable; CreateUser merges them.
func PrefillUserProfileDefaults(u *users.User) {
settings.ApplyFullProfileFromDefaults(u, EffectiveUserDefaults())
// json.Unmarshal reuses an existing slice's backing array, so clone it or a request
// would overwrite the shared in-memory defaults.
u.NewFileTemplate = slices.Clone(u.NewFileTemplate)
u.Permissions = users.Permissions{}
}
func applyUserSettingsDefaults(u *users.User) {
settings.ApplyUserDefaultsFrom(u, EffectiveUserDefaults())
}
+19
View File
@@ -202,6 +202,18 @@ func UserForShareOwner(link share.Share) (users.User, error) {
// may apply default-enabled sources. In-memory/cache users always keep FrontendScopes nil; GET handlers use
// PrepForFrontend to repopulate FrontendScopes from BackendScopes for responses.
func CreateUser(user *users.User, plaintextPassword string) error {
return createUser(user, plaintextPassword, false)
}
// CreateUserKeepingProfile is CreateUser for callers whose user was seeded with
// PrefillUserProfileDefaults before decoding the request: the profile fields already hold
// defaults overlaid with the request, so they are not replaced by userDefaults again.
// Enforced defaults still win.
func CreateUserKeepingProfile(user *users.User, plaintextPassword string) error {
return createUser(user, plaintextPassword, true)
}
func createUser(user *users.User, plaintextPassword string, keepProfile bool) error {
if err := users.ValidateUsername(user.Username); err != nil {
return err
}
@@ -256,12 +268,19 @@ func CreateUser(user *users.User, plaintextPassword string) error {
requestedScopePermissions[backendScopeKeyFor(scope)] = scope.Permissions
}
}
var requestedProfile settings.UserProfile
if keepProfile {
requestedProfile = settings.ProfileFromUser(user)
}
// Apply profile/scope defaults using the final account permissions (admin flag drives scope defaults).
if permissionsRequested {
settings.ApplyUserDefaultsFromKeepingAccountPermissions(user, EffectiveUserDefaults())
} else {
applyUserSettingsDefaults(user)
}
if keepProfile {
settings.ExpandProfileIntoUser(user, requestedProfile)
}
if preserveRequirePasswordChange {
user.RequirePasswordChange = true
}
+1 -1
View File
@@ -133,7 +133,7 @@ func (e *MediaEngine) BuildPlan(ctx context.Context, realPath string, profile Pr
copyImpossible := false
plan.segmentSec = defaults.SegmentDurationSec
if plan.remux || plan.videoCopy {
rawKeyframes, kfErr := probeKeyframesCached(ctx, svc, realPath, stat.Size(), stat.ModTime().UnixNano(), plan.segmentSec, true)
rawKeyframes, kfErr := probeKeyframesCached(ctx, svc, realPath, stat.Size(), stat.ModTime().UnixNano(), plan.segmentSec, mediaSpan{durationSec: info.Duration, startTimeSec: info.StartTime}, true)
if kfErr != nil {
copyImpossible = true
logger.Warningf("transcode: keyframe probe failed on %s (%v) — forcing video transcode for HLS", realPath, kfErr)
+79 -1
View File
@@ -12,6 +12,7 @@ import (
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/gomedia"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-logger/logger"
)
@@ -55,6 +56,9 @@ type keyframeCache struct {
var defaultKeyframeCache = &keyframeCache{entries: map[keyframeKey]*keyframeEntry{}}
// get returns the keyframe timeline for key from memory, then disk, then
// probe. Concurrent callers for one key wait on a single probe; failures are
// not cached.
func (c *keyframeCache) get(ctx context.Context, key keyframeKey, probe func(context.Context) ([]float64, error)) ([]float64, error) {
c.mu.Lock()
e := c.entries[key]
@@ -105,12 +109,15 @@ func keyframeDiskCacheEnabled() bool {
return settings.Config.Server.CacheDir != ""
}
// keyframeDiskPath returns the disk cache file for key's path.
func keyframeDiskPath(key keyframeKey) string {
sum := sha256.Sum256([]byte(key.path))
name := hex.EncodeToString(sum[:])[:32] + ".json"
return filepath.Join(settings.TranscodeKeyframeCacheDir(), name)
}
// readKeyframeDiskCache returns the persisted timeline for key when it exists
// and matches the file's size, mtime, and the cache version.
func readKeyframeDiskCache(key keyframeKey) ([]float64, bool) {
if !keyframeDiskCacheEnabled() {
return nil, false
@@ -132,6 +139,8 @@ func readKeyframeDiskCache(key keyframeKey) ([]float64, bool) {
return stamp.Times, true
}
// writeKeyframeDiskCache persists a timeline for key atomically. Errors are
// ignored; the cache is an optimization.
func writeKeyframeDiskCache(key keyframeKey, times []float64) {
if len(times) == 0 || !keyframeDiskCacheEnabled() {
return
@@ -219,19 +228,88 @@ func probeKeyframesWithPreflight(ctx context.Context, svc *ffmpeg.Service, realP
)
}
// mediaSpan is the ffprobe format duration and start time of a file, the
// values the HLS planner turns into its media timeline. The zero value means
// unknown.
type mediaSpan struct {
durationSec float64
startTimeSec float64
}
// trailingGapExceedsCopyCap reports whether the stretch from the last
// keyframe to the end of the media is longer than the copy cap, measured on
// the same normalized timeline the planner builds: the final segment runs
// from the last keyframe to the media end, so such a gap forces a transcode
// just like a gap between two keyframes. An unknown span reports false.
func trailingGapExceedsCopyCap(keyframes []float64, span mediaSpan, segmentSec float64) bool {
if len(keyframes) == 0 || span.durationSec <= 0 {
return false
}
kf, mediaDur := ffmpeg.PrepareHLSMediaTimeline(keyframes, span.durationSec, span.startTimeSec)
if len(kf) == 0 || mediaDur <= 0 {
return false
}
return mediaDur-kf[len(kf)-1] > maxCopySegmentDurationSec(segmentSec)
}
// keyframeProbeTimeout bounds a single full-file keyframe probe. The default
// demux has no deadline of its own, so a slow network share would otherwise
// stall plan building (and the client's session POST) indefinitely.
const keyframeProbeTimeout = 2 * time.Minute
func probeKeyframesCached(ctx context.Context, svc *ffmpeg.Service, realPath string, size, modTime int64, segmentSec float64, preflight bool) ([]float64, error) {
// probeKeyframesCached returns the keyframe timeline of a file version from
// the memory or disk cache, or probes it: go-media reads the container index
// first, and ffprobe runs only when go-media cannot answer or reports a gap
// that would rule out stream copy. span is the file's duration and start
// time, used to check the gap after the last keyframe; the zero value skips
// that check.
func probeKeyframesCached(ctx context.Context, svc *ffmpeg.Service, realPath string, size, modTime int64, segmentSec float64, span mediaSpan, preflight bool) ([]float64, error) {
key := keyframeKey{path: realPath, size: size, modTime: modTime}
return defaultKeyframeCache.get(ctx, key, func(ctx context.Context) ([]float64, error) {
probeCtx, cancel := context.WithTimeout(ctx, keyframeProbeTimeout)
defer cancel()
if times, ok := gomediaKeyframes(probeCtx, svc, realPath, segmentSec, span, preflight); ok {
return times, nil
}
if preflight {
return probeKeyframesWithPreflight(probeCtx, svc, realPath, segmentSec)
}
return svc.ProbeVideoKeyframeTimes(probeCtx, realPath)
})
}
// gomediaKeyframes reads the keyframe timeline from the container index with
// go-media. It reports ok=false when go-media cannot answer, and also when
// preflight is set and the timeline has a gap past the copy cap, including
// the gap from the last keyframe to the end of span: go-media can miss
// keyframes that ffprobe's parsers flag (IDR frames not in the index,
// partial Matroska Cues) but never adds one, so only ffprobe may decide that
// stream copy is impossible. A bounded go-media read runs first, so a file
// whose head already shows such a gap does not pay for a full index walk.
func gomediaKeyframes(ctx context.Context, svc *ffmpeg.Service, realPath string, segmentSec float64, span mediaSpan, preflight bool) ([]float64, bool) {
if preflight {
if head, ok := gomedia.KeyframeTimesWithin(ctx, realPath, keyframeBoundedPreflightWindowSec); ok && keyframesExceedCopyCap(head, segmentSec) {
logger.Debugf("transcode: go-media keyframe pre-flight shows a gap past the copy cap on %s; confirming with ffprobe", realPath)
return nil, false
}
}
legacy := func() ([]float64, bool) {
shadowCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), keyframeProbeTimeout)
defer cancel()
times, err := svc.ProbeVideoKeyframeTimes(shadowCtx, realPath)
return times, err == nil
}
times, ok := gomedia.KeyframeTimes(ctx, realPath, legacy)
if !ok {
return nil, false
}
if preflight && keyframesExceedCopyCap(times, segmentSec) {
logger.Debugf("transcode: go-media keyframe index shows a gap past the copy cap on %s; confirming with ffprobe", realPath)
return nil, false
}
if preflight && trailingGapExceedsCopyCap(times, span, segmentSec) {
logger.Debugf("transcode: go-media keyframe index ends more than the copy cap before the end of %s; confirming with ffprobe", realPath)
return nil, false
}
return times, true
}
@@ -177,3 +177,33 @@ func TestKeyframeCacheDoesNotCacheFailures(t *testing.T) {
t.Fatalf("retry after failure: %v, %v", got, err)
}
}
// TestTrailingGapExceedsCopyCap checks the gap from the last keyframe to the
// end of the media, on the planner's start-time-normalized timeline.
func TestTrailingGapExceedsCopyCap(t *testing.T) {
every2 := func(from, to float64) []float64 {
var out []float64
for v := from; v < to; v += 2 {
out = append(out, v)
}
return out
}
cases := []struct {
name string
kf []float64
span mediaSpan
want bool
}{
{"partial index", []float64{0, 2}, mediaSpan{durationSec: 60}, true},
{"full index", every2(0, 60), mediaSpan{durationSec: 60}, false},
{"unknown span", []float64{0, 2}, mediaSpan{}, false},
{"start time, partial", []float64{10, 12}, mediaSpan{durationSec: 70, startTimeSec: 10}, true},
{"start time, full", every2(10, 70), mediaSpan{durationSec: 70, startTimeSec: 10}, false},
{"no keyframes", nil, mediaSpan{durationSec: 60}, false},
}
for _, c := range cases {
if got := trailingGapExceedsCopyCap(c.kf, c.span, 4); got != c.want {
t.Errorf("%s: got %v, want %v", c.name, got, c.want)
}
}
}
@@ -0,0 +1,111 @@
package transcoding
import (
"context"
"os"
"os/exec"
"path/filepath"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
const keyframeFixtureMP4 = "../../../frontend/tests/playwright-files/myfolder/testdata/sample-640x480.mp4"
// TestProbeKeyframesCachedUsesGoMedia checks that an MP4 keyframe timeline is
// built without ffprobe: with no ffmpeg service, only go-media can answer.
func TestProbeKeyframesCachedUsesGoMedia(t *testing.T) {
if settings.AlwaysUseFfmpegEnabled() {
t.Skip("go-media disabled by settings")
}
prevCacheDir := settings.Config.Server.CacheDir
settings.Config.Server.CacheDir = "" // keep the disk cache out of the test
t.Cleanup(func() { settings.Config.Server.CacheDir = prevCacheDir })
path := filepath.Join(t.TempDir(), "sample.mp4")
data, err := os.ReadFile(keyframeFixtureMP4)
if err != nil {
t.Fatal(err)
}
if err = os.WriteFile(path, data, 0o644); err != nil {
t.Fatal(err)
}
st, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
times, err := probeKeyframesCached(context.Background(), nil, path, st.Size(), st.ModTime().UnixNano(), 4, mediaSpan{}, true)
if err != nil {
t.Fatalf("probeKeyframesCached: %v", err)
}
if len(times) == 0 || times[0] < 0 {
t.Fatalf("unexpected keyframes %v", times)
}
for i := 1; i < len(times); i++ {
if times[i] <= times[i-1] {
t.Fatalf("keyframes not strictly increasing: %v", times)
}
}
}
// TestGomediaKeyframesDefersCopyDecision checks that a go-media timeline with a
// gap past the copy cap is not used for the pre-flight decision (ffprobe must
// confirm it), while a plain probe still returns it.
func TestGomediaKeyframesDefersCopyDecision(t *testing.T) {
if testing.Short() {
t.Skip("encodes a clip with ffmpeg")
}
if settings.AlwaysUseFfmpegEnabled() {
t.Skip("go-media disabled by settings")
}
if _, err := exec.LookPath("ffmpeg"); err != nil {
t.Skip("ffmpeg not installed")
}
path := filepath.Join(t.TempDir(), "longgop.mp4")
out, err := exec.Command("ffmpeg", "-v", "error", "-y", "-f", "lavfi", "-i", "testsrc2=size=128x96:rate=25:duration=40",
"-c:v", "libx264", "-g", "750", "-keyint_min", "750", "-sc_threshold", "0", "-pix_fmt", "yuv420p", path).CombinedOutput()
if err != nil {
t.Skipf("ffmpeg cannot encode libx264: %v: %s", err, out)
}
if times, ok := gomediaKeyframes(context.Background(), nil, path, 4, mediaSpan{}, true); ok {
t.Fatalf("pre-flight must defer a 30s GOP to ffprobe, got %v", times)
}
times, ok := gomediaKeyframes(context.Background(), nil, path, 4, mediaSpan{}, false)
if !ok {
t.Fatal("a plain probe must return the go-media timeline")
}
if len(times) != 2 || times[0] != 0 || times[1] < 29.9 || times[1] > 30.1 {
t.Fatalf("unexpected keyframes %v, want [0 30]", times)
}
}
// TestGomediaKeyframesDefersTrailingGap checks that a go-media index whose
// last keyframe is more than the copy cap before the end of the media is not
// used for the pre-flight decision, so ffprobe can confirm it, while the
// same index is used when it reaches the end.
func TestGomediaKeyframesDefersTrailingGap(t *testing.T) {
if testing.Short() {
t.Skip("encodes a clip with ffmpeg")
}
if settings.AlwaysUseFfmpegEnabled() {
t.Skip("go-media disabled by settings")
}
if _, err := exec.LookPath("ffmpeg"); err != nil {
t.Skip("ffmpeg not installed")
}
path := filepath.Join(t.TempDir(), "short.mp4")
out, err := exec.Command("ffmpeg", "-v", "error", "-y", "-f", "lavfi", "-i", "testsrc2=size=128x96:rate=25:duration=4",
"-c:v", "libx264", "-g", "50", "-keyint_min", "50", "-sc_threshold", "0", "-pix_fmt", "yuv420p", path).CombinedOutput()
if err != nil {
t.Skipf("ffmpeg cannot encode libx264: %v: %s", err, out)
}
// The index ends at 2 s; a 60 s span leaves a 58 s final segment.
if times, ok := gomediaKeyframes(context.Background(), nil, path, 4, mediaSpan{durationSec: 60}, true); ok {
t.Fatalf("pre-flight must defer a 58s trailing gap to ffprobe, got %v", times)
}
times, ok := gomediaKeyframes(context.Background(), nil, path, 4, mediaSpan{durationSec: 4}, true)
if !ok || len(times) != 2 {
t.Fatalf("index reaching the end must be used, got %v, %v", times, ok)
}
}
+8 -2
View File
@@ -15,12 +15,18 @@ type SubtitleTrack struct {
Embedded bool `json:"embedded"` // true for embedded subtitles, false for external files
}
// ErrSubtitleSidecarNotRegular is returned when an external subtitle path is not a regular file (e.g. symlink).
var ErrSubtitleSidecarNotRegular = errors.New("subtitle sidecar is not a regular file")
func GetSubtitleSidecarContent(subtitlePath string) (string, error) {
// validate the size is not too large
info, err := os.Stat(subtitlePath)
// Sidecars sit beside the media file; do not follow symlink leaves (see lyricsSidecar).
info, err := os.Lstat(subtitlePath)
if err != nil {
return "", err
}
if !info.Mode().IsRegular() {
return "", ErrSubtitleSidecarNotRegular
}
if info.Size() > 1024*1024*50 { // 50MB
return "", errors.New("subtitle file is too large")
}
+1
View File
@@ -22,6 +22,7 @@ import (
func setupAccessHTTPTest(t *testing.T) {
t.Helper()
lockWebGlobals(t)
dbPath := filepath.Join(t.TempDir(), "access-http-test.sqlite")
if _, err := state.Initialize(dbPath); err != nil {
t.Fatal(err)
+11 -8
View File
@@ -279,19 +279,21 @@ func archiveCreateHandler(w http.ResponseWriter, r *http.Request, d *Context) (i
if fullDestParent != fullDestFile && !state.AccessPermitted(idxTo.Path, utils.IndexPathFromNormalized(fullDestParent, true), d.User.Username) {
return http.StatusForbidden, fmt.Errorf("access denied to destination parent of %s", req.Destination)
}
parentReal, err := idxTo.GetRealPathForMutation(userScopeTo, fullDestParent)
if err != nil {
return http.StatusBadRequest, fmt.Errorf("destination parent path invalid: %v", err)
}
if err = os.MkdirAll(parentReal, fileutils.EffectiveDirPerm()); err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot create destination parent directory: %v", err)
}
destRel := strings.TrimLeft(filepath.ToSlash(req.Destination), "/")
fileName := path.Base(destRel)
if fileName == "." || fileName == "/" {
return http.StatusBadRequest, fmt.Errorf("invalid destination file name")
}
destFileReal := filepath.Join(parentReal, fileName)
destFileReal, err := idxTo.GetRealPathForMutation(userScopeTo, fullDestFile)
if err != nil {
return http.StatusBadRequest, fmt.Errorf("destination path invalid: %v", err)
}
if !state.AccessPermittedAtRealPath(idxTo, d.User.Username, destFileReal, false) {
return http.StatusForbidden, fmt.Errorf("access denied to destination %s", req.Destination)
}
if err = os.MkdirAll(filepath.Dir(destFileReal), fileutils.EffectiveDirPerm()); err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot create destination parent directory: %v", err)
}
format := strings.ToLower(strings.TrimSpace(req.Format))
if format == "" {
@@ -1270,6 +1272,7 @@ func extractArchivedSymlink(destRoot, destPath, target string, modTime time.Time
logger.Debugf("unarchive: symlink: %v", err)
return fmt.Errorf("failed to create symlink from archive entry")
}
indexing.ClearRealPathCaches()
return applyArchivedTimesAndPerm(destPath, fs.ModeSymlink, modTime)
}
@@ -0,0 +1,86 @@
package web
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func TestArchiveCreate_DeniesResolvedSymlinkDestination(t *testing.T) {
_, sourceRoot, sourceName, username := setupExtractAccessPolicyTest(t)
if err := state.DenyUser(sourceRoot, utils.IndexPathFromNormalized("/denied", true), username); err != nil {
t.Fatal(err)
}
allowedDir := filepath.Join(sourceRoot, "allowed")
deniedDir := filepath.Join(sourceRoot, "denied")
for _, dir := range []string{allowedDir, deniedDir} {
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
}
victim := filepath.Join(deniedDir, "victim")
if err := os.WriteFile(victim, []byte("keep"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(sourceRoot, "a.txt"), []byte("payload"), 0o644); err != nil {
t.Fatal(err)
}
link := filepath.Join(allowedDir, "report.zip")
if err := os.Symlink(filepath.Join("..", "denied", "victim"), link); err != nil {
t.Fatal(err)
}
user, err := state.GetUserByUsername(username)
if err != nil {
t.Fatal(err)
}
settings.Config.Server.SourceMap[sourceRoot].Config.DenyByDefault = false
body := `{"fromSource":"` + sourceName + `","paths":["/a.txt"],"destination":"/allowed/report.zip","format":"zip"}`
req := httptest.NewRequest(http.MethodPost, "/api/resources/archive", strings.NewReader(body))
rec := httptest.NewRecorder()
status, handlerErr := archiveCreateHandler(rec, req, &Context{User: &user})
if status != http.StatusForbidden || handlerErr == nil {
t.Fatalf("status=%d err=%v", status, handlerErr)
}
got, err := os.ReadFile(victim)
if err != nil || string(got) != "keep" {
t.Fatalf("victim content=%q err=%v", got, err)
}
}
func TestArchiveCreate_AllowsNonSymlinkDestination(t *testing.T) {
_, sourceRoot, sourceName, username := setupExtractAccessPolicyTest(t)
dest := filepath.Join(sourceRoot, "out.zip")
if err := os.WriteFile(filepath.Join(sourceRoot, "a.txt"), []byte("payload"), 0o644); err != nil {
t.Fatal(err)
}
user, err := state.GetUserByUsername(username)
if err != nil {
t.Fatal(err)
}
body := `{"fromSource":"` + sourceName + `","paths":["/a.txt"],"destination":"/out.zip","format":"zip"}`
req := httptest.NewRequest(http.MethodPost, "/api/resources/archive", strings.NewReader(body))
rec := httptest.NewRecorder()
status, handlerErr := archiveCreateHandler(rec, req, &Context{User: &user})
if status != http.StatusOK || handlerErr != nil {
t.Fatalf("status=%d err=%v body=%s", status, handlerErr, rec.Body.String())
}
var resp map[string]string
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(dest); err != nil {
t.Fatalf("expected archive at %s: %v", dest, err)
}
}
@@ -23,6 +23,7 @@ import (
func setupExtractAccessPolicyTest(t *testing.T) (destDir, sourceRoot, sourceName, username string) {
t.Helper()
lockWebGlobals(t)
tempDir := t.TempDir()
dbPath := filepath.Join(tempDir, "access.sqlite")
if _, err := state.Initialize(dbPath); err != nil {
+21
View File
@@ -3,11 +3,16 @@ package web
import (
"archive/tar"
"archive/zip"
"errors"
"io"
"os"
"path/filepath"
"strings"
"testing"
commonerrors "github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func TestNormalizeArchiveEntryName(t *testing.T) {
@@ -557,3 +562,19 @@ func TestExtractZip_symlinkDestination(t *testing.T) {
t.Fatalf("victim.txt should be unchanged: %q, %v", victimData, err)
}
}
func TestArchiveCreateDestination_RejectsEscapingSymlinkLeaf(t *testing.T) {
t.Parallel()
root := t.TempDir()
outside := filepath.Join(filepath.Dir(root), "outside-archive")
if err := os.MkdirAll(outside, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(root, "report.zip")); err != nil {
t.Fatal(err)
}
idx := &indexing.Index{Source: settings.Source{Path: root}}
if _, err := idx.GetRealPathForMutation("", "/report.zip"); !errors.Is(err, commonerrors.ErrPathEscapesScope) {
t.Fatalf("expected ErrPathEscapesScope, got %v", err)
}
}
@@ -25,6 +25,7 @@ import (
func setupShareArchiveDownloadTest(t *testing.T) (sourceRoot string, sourceName string, owner *users.User) {
t.Helper()
lockWebGlobals(t)
dbPath := filepath.Join(t.TempDir(), "share-dl-test.sqlite")
if _, err := state.Initialize(dbPath); err != nil {
t.Fatal(err)
+12 -6
View File
@@ -187,10 +187,10 @@ func fileWatchHandler(w http.ResponseWriter, r *http.Request, d *Context) (int,
return http.StatusForbidden, fmt.Errorf("access denied to file")
}
// Get real file path
realPath, _, err := idx.GetRealPath(scopePath)
// Get real file path (scoped) and enforce ACL on the resolved target
realPath, _, err := idx.GetRealPathScoped(userScope, scopePath)
if err != nil {
return http.StatusNotFound, fmt.Errorf("file not found: %v", err)
return realPathErrStatus(err), err
}
// Get file/directory info
@@ -198,6 +198,9 @@ func fileWatchHandler(w http.ResponseWriter, r *http.Request, d *Context) (int,
if err != nil {
return http.StatusNotFound, fmt.Errorf("path not found: %v", err)
}
if !state.AccessPermittedAtRealPath(idx, d.User.Username, realPath, info.IsDir()) {
return http.StatusForbidden, fmt.Errorf("access denied to file")
}
// Get MIME type from the index if available
mimeType := "application/octet-stream"
@@ -335,10 +338,10 @@ func fileWatchSSEHandler(w http.ResponseWriter, r *http.Request, d *Context) (in
return http.StatusForbidden, fmt.Errorf("access denied to file")
}
// Get real file path
realPath, _, err := idx.GetRealPath(scopePath)
// Get real file path (scoped) and enforce ACL on the resolved target
realPath, _, err := idx.GetRealPathScoped(userScope, scopePath)
if err != nil {
return http.StatusNotFound, fmt.Errorf("path not found: %v", err)
return realPathErrStatus(err), err
}
// Get file/directory info
@@ -346,6 +349,9 @@ func fileWatchSSEHandler(w http.ResponseWriter, r *http.Request, d *Context) (in
if err != nil {
return http.StatusNotFound, fmt.Errorf("path not found: %v", err)
}
if !state.AccessPermittedAtRealPath(idx, d.User.Username, realPath, info.IsDir()) {
return http.StatusForbidden, fmt.Errorf("access denied to file")
}
// Set up SSE headers
w.Header().Set("Content-Type", "text/event-stream")
+34
View File
@@ -22,3 +22,37 @@ func TestErrToStatusWrapsUnsupportedPreviewFormat(t *testing.T) {
t.Fatalf("ErrToStatus() = %d, want %d", status, http.StatusUnsupportedMediaType)
}
}
func TestIsViewerOriginalPreview(t *testing.T) {
tests := []struct {
name string
size string
file string
fileType string
docs bool // the server can render documents to an image
want bool
}{
{"Fuji RAW viewer", "original", "DSCF0308.RAF", "image/x-fuji-raf", false, true},
{"Canon RAW viewer", "original", "IMG_0001.CR3", "image/x-canon-cr3", false, true},
{"Adobe DNG viewer", "original", "a.dng", "image/x-adobe-dng", false, true},
{"HEIC viewer", "original", "a.heic", "image/heic", false, true},
{"HEIF viewer", "original", "a.heif", "image/heif", false, true},
{"rendered document viewer", "original", "a.docx", "application/vnd.openxmlformats-officedocument.wordprocessingml.document", true, true},
{"documents need the server to be able to render them", "original", "a.docx", "application/vnd.openxmlformats-officedocument.wordprocessingml.document", false, false},
{"RAW thumbnail stays disabled", "small", "DSCF0308.RAF", "image/x-fuji-raf", true, false},
{"RAW large stays disabled", "large", "DSCF0308.RAF", "image/x-fuji-raf", true, false},
{"no size stays disabled", "", "DSCF0308.RAF", "image/x-fuji-raf", true, false},
{"video poster stays disabled", "original", "a.mov", "video/quicktime", true, false},
{"audio cover stays disabled", "original", "a.mp3", "audio/mpeg", true, false},
{"JPEG is opened directly, not through previews", "original", "a.jpg", "image/jpeg", true, false},
{"PDF is opened directly, not through previews", "original", "a.pdf", "application/pdf", true, false},
{"text stays disabled", "original", "a.txt", "text/plain", true, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isViewerOriginalPreview(tt.size, tt.file, tt.fileType, tt.docs); got != tt.want {
t.Fatalf("isViewerOriginalPreview(%q, %q, %q, %v) = %v, want %v", tt.size, tt.file, tt.fileType, tt.docs, got, tt.want)
}
})
}
}
+60 -2
View File
@@ -3,6 +3,7 @@ package web
import (
"bytes"
"context"
"errors"
"fmt"
"net/http"
"path/filepath"
@@ -12,7 +13,10 @@ import (
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/gomedia"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
@@ -106,9 +110,34 @@ func subtitlesHandler(w http.ResponseWriter, r *http.Request, d *Context) (int,
var content string
if !embedded {
subtitlePath := filepath.Join(filepath.Dir(fileInfo.RealPath), filepath.Base(track.Name))
var subtitlePath string
if d.Share.Hash != "" {
if d.Share.Path == "" {
return http.StatusForbidden, fmt.Errorf("share has no path scope")
}
idx := indexing.GetIndex(source)
if idx == nil {
return http.StatusInternalServerError, fmt.Errorf("source %s is not available", source)
}
bound, sidecarIndexPath, scopeErr := shareSubtitleSidecarIndexPaths(d, source, path, track.Name)
if scopeErr != nil {
return http.StatusForbidden, scopeErr
}
subtitlePath, _, err = idx.GetRealPathScoped(bound, sidecarIndexPath)
if err != nil {
return realPathErrStatus(err), err
}
if !state.AccessPermittedAtRealPath(idx, d.ShareUser.Username, subtitlePath, false) {
return http.StatusForbidden, fmt.Errorf("access denied")
}
} else {
subtitlePath = filepath.Join(filepath.Dir(fileInfo.RealPath), filepath.Base(track.Name))
}
content, err = utils.GetSubtitleSidecarContent(subtitlePath)
if err != nil {
if errors.Is(err, utils.ErrSubtitleSidecarNotRegular) {
return http.StatusForbidden, err
}
return http.StatusInternalServerError, fmt.Errorf("failed to get subtitle sidecar content: %v", err)
}
} else {
@@ -123,7 +152,8 @@ func subtitlesHandler(w http.ResponseWriter, r *http.Request, d *Context) (int,
}
// Shadow ffmpeg runs after the fast path; bound it so a detached request cannot
// leave extraction running indefinitely (sync fallback still uses r.Context()).
legacyCtx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), time.Minute)
// WithoutCache: compare against a fresh extraction, not the value go-media stored.
legacyCtx, cancel := context.WithTimeout(mediacache.WithoutCache(context.WithoutCancel(r.Context())), time.Minute)
defer cancel()
vtt, extractErr := svc.ExtractSubtitle(legacyCtx, fileInfo.RealPath, *track.Index)
return vtt, extractErr == nil
@@ -160,6 +190,34 @@ func findSubtitleTrack(subtitles []utils.SubtitleTrack, name string, embedded bo
return nil
}
// shareSubtitleSidecarIndexPaths maps middleware IndexPath (user-scope-relative) and Share.Path
// to scoped index paths for sidecar containment (see publicDeleteHandler).
func shareSubtitleSidecarIndexPaths(d *Context, sourceName, videoIndexPath, sidecarFileName string) (bound, sidecarIndexPath string, err error) {
userScope, scopeErr := d.ShareUser.GetScopeForSourceName(sourceName)
if scopeErr != nil || userScope == "" {
return "", "", fmt.Errorf("user has no access to source: %s", sourceName)
}
if d.Share.Path == "" {
return "", "", fmt.Errorf("share has no path scope")
}
shareRel := strings.TrimPrefix(d.Share.Path, userScope)
bound = utils.JoinScopedIndexPath(userScope, shareRel)
videoRel := strings.Trim(strings.TrimSpace(videoIndexPath), "/")
fullVideo := utils.JoinScopedIndexPath(userScope, videoRel)
if strings.TrimSuffix(bound, "/") == strings.TrimSuffix(fullVideo, "/") {
parent := utils.GetParentDirectoryPath(strings.TrimSuffix(bound, "/"))
if parent == "" {
parent = "/"
}
bound = parent
}
sidecarIndexPath = utils.JoinPathAsUnix(
utils.GetParentDirectoryPath(strings.TrimSuffix(fullVideo, "/")),
filepath.Base(sidecarFileName),
)
return bound, sidecarIndexPath, nil
}
// metadataHandler returns the same directory resource shape as GET /api/resources with metadata enabled,
// for client-side patching after a fast listing load.
// @Summary Directory with media metadata
@@ -0,0 +1,333 @@
package web
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/app"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/share"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func setupSubtitlesHTTPTest(t *testing.T) (downloadsPath string, admin *users.User) {
t.Helper()
lockWebGlobals(t)
downloadsPath = filepath.Join(t.TempDir(), "downloads")
dbPath := filepath.Join(t.TempDir(), "subtitles-http-test.sqlite")
if _, err := state.Initialize(dbPath); err != nil {
t.Fatal(err)
}
app.MustWireServices(state.Default())
t.Cleanup(func() {
state.Close()
})
settings.Config.Server.SourceMap = map[string]*settings.Source{
downloadsPath: {
Path: downloadsPath,
Name: "Downloads",
},
}
settings.Config.Server.NameToSource = map[string]*settings.Source{
"Downloads": settings.Config.Server.SourceMap[downloadsPath],
}
settings.InitializeUserResolvers()
adminUser := &users.User{
ID: 1,
FrontendUser: users.FrontendUser{
Username: "admin",
Permissions: users.Permissions{Admin: true},
},
BackendScopes: []users.BackendScope{
{Path: downloadsPath, Scope: "/"},
},
BackendSourcePermissions: map[string]users.SourceFilePermissions{
downloadsPath: {
View: true,
Download: true,
Modify: true,
Create: true,
Delete: true,
},
},
Version: users.SourcePermissionsMigrationVersion,
}
if err := state.CreateUser(adminUser, ""); err != nil {
t.Fatal(err)
}
adminUser.Permissions = users.Permissions{Admin: true}
if err := state.UpdateUser(adminUser, "", "permissions"); err != nil {
t.Fatal(err)
}
if err := state.UpdateUser(adminUser, "", "backendScopes"); err != nil {
t.Fatal(err)
}
if err := state.UpdateUser(adminUser, "", "backendSourcePermissions"); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(downloadsPath, 0755); err != nil {
t.Fatal(err)
}
indexing.SetTestIndex("Downloads", downloadsPath)
t.Cleanup(func() {
indexing.ClearTestIndices()
})
loaded, err := state.GetUserByUsername("admin")
if err != nil {
t.Fatal(err)
}
return downloadsPath, &loaded
}
func mockVideoWithExternalSubtitle(t *testing.T, trackName, videoRealPath string) {
t.Helper()
prev := files.FileInfoFasterFunc
t.Cleanup(func() { files.FileInfoFasterFunc = prev })
files.FileInfoFasterFunc = func(opts utils.FileOptions, user *users.User) (*iteminfo.ExtendedFileInfo, error) {
return &iteminfo.ExtendedFileInfo{
FileInfo: iteminfo.FileInfo{
Path: opts.Path,
ItemInfo: iteminfo.ItemInfo{
Type: "video/mp4",
},
},
Subtitles: []utils.SubtitleTrack{
{Name: trackName, Embedded: false},
},
RealPath: videoRealPath,
}, nil
}
}
func TestSubtitlesHandler_RejectsEscapingSidecarSymlink(t *testing.T) {
downloadsPath, adminUser := setupSubtitlesHTTPTest(t)
videoPath := filepath.Join(downloadsPath, "clip.mov")
mockVideoWithExternalSubtitle(t, "clip.srt", videoPath)
outside := filepath.Join(filepath.Dir(downloadsPath), "outside-secret")
if err := os.MkdirAll(outside, 0755); err != nil {
t.Fatal(err)
}
secret := filepath.Join(outside, "secret.srt")
if err := os.WriteFile(secret, []byte("SECRET_OUTSIDE_SUBTITLE"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(videoPath, []byte("video"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(secret, filepath.Join(downloadsPath, "clip.srt")); err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodGet, "/api/media/subtitles?source=Downloads&path=/clip.mov&name=clip.srt&embedded=false", nil)
rec := httptest.NewRecorder()
status, handlerErr := subtitlesHandler(rec, req, &Context{User: adminUser})
if status != http.StatusForbidden {
t.Fatalf("status=%d err=%v body=%q", status, handlerErr, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "SECRET_OUTSIDE_SUBTITLE") {
t.Fatalf("response leaked outside subtitle content")
}
}
func TestSubtitlesHandler_ReturnsInScopeSidecar(t *testing.T) {
downloadsPath, adminUser := setupSubtitlesHTTPTest(t)
videoPath := filepath.Join(downloadsPath, "clip.mov")
mockVideoWithExternalSubtitle(t, "clip.srt", videoPath)
srtBody := "1\n00:00:01,000 --> 00:00:02,000\nHello in scope"
if err := os.WriteFile(videoPath, []byte("video"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(downloadsPath, "clip.srt"), []byte(srtBody), 0644); err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodGet, "/api/media/subtitles?source=Downloads&path=/clip.mov&name=clip.srt&embedded=false", nil)
rec := httptest.NewRecorder()
status, handlerErr := subtitlesHandler(rec, req, &Context{User: adminUser})
if status != http.StatusOK || handlerErr != nil {
t.Fatalf("status=%d err=%v body=%q", status, handlerErr, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "Hello in scope") {
t.Fatalf("body=%q want in-scope subtitle text", rec.Body.String())
}
}
func TestSubtitlesHandler_ShareScopedOwnerLoadsSidecar(t *testing.T) {
sourceRoot, adminUser := setupSubtitlesHTTPTest(t)
aliceRoot := filepath.Join(sourceRoot, "users", "alice", "shared")
if err := os.MkdirAll(aliceRoot, 0755); err != nil {
t.Fatal(err)
}
srtBody := "1\n00:00:01,000 --> 00:00:02,000\nScoped share OK"
videoPath := filepath.Join(aliceRoot, "clip.mov")
if err := os.WriteFile(videoPath, []byte("video"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(aliceRoot, "clip.srt"), []byte(srtBody), 0644); err != nil {
t.Fatal(err)
}
mockVideoWithExternalSubtitle(t, "clip.srt", videoPath)
adminUser.BackendScopes = []users.BackendScope{{Path: sourceRoot, Scope: "/users/alice"}}
req := httptest.NewRequest(http.MethodGet, "/public/api/media/subtitles?hash=h&path=clip.mov&name=clip.srt&embedded=false", nil)
rec := httptest.NewRecorder()
status, handlerErr := subtitlesHandler(rec, req, &Context{
User: adminUser,
ShareUser: adminUser,
IndexPath: "/shared/clip.mov/",
Share: share.Share{
ShareColumns: share.ShareColumns{
Hash: "h",
Path: "/users/alice/shared",
},
SourcePath: sourceRoot,
},
})
if status != http.StatusOK || handlerErr != nil {
t.Fatalf("status=%d err=%v body=%q", status, handlerErr, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "Scoped share OK") {
t.Fatalf("body=%q", rec.Body.String())
}
}
func TestSubtitlesHandler_ShareSingleFileLoadsSidecar(t *testing.T) {
sourceRoot, adminUser := setupSubtitlesHTTPTest(t)
srtBody := "single-file share subtitle"
videoPath := filepath.Join(sourceRoot, "clip.mov")
if err := os.WriteFile(videoPath, []byte("video"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(sourceRoot, "clip.srt"), []byte(srtBody), 0644); err != nil {
t.Fatal(err)
}
mockVideoWithExternalSubtitle(t, "clip.srt", videoPath)
req := httptest.NewRequest(http.MethodGet, "/public/api/media/subtitles?hash=h&path=/&name=clip.srt&embedded=false", nil)
rec := httptest.NewRecorder()
status, handlerErr := subtitlesHandler(rec, req, &Context{
User: adminUser,
ShareUser: adminUser,
IndexPath: "/clip.mov/",
Share: share.Share{
ShareColumns: share.ShareColumns{
Hash: "h",
Path: "/clip.mov",
},
SourcePath: sourceRoot,
},
})
if status != http.StatusOK || handlerErr != nil {
t.Fatalf("status=%d err=%v body=%q", status, handlerErr, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "single-file share subtitle") {
t.Fatalf("body=%q", rec.Body.String())
}
}
func TestSubtitlesHandler_ShareDeniesSidecarSymlinkToDeniedTarget(t *testing.T) {
sourceRoot, adminUser := setupSubtitlesHTTPTest(t)
sharedDir := filepath.Join(sourceRoot, "shared")
privateDir := filepath.Join(sourceRoot, "private")
for _, dir := range []string{sharedDir, privateDir} {
if err := os.MkdirAll(dir, 0755); err != nil {
t.Fatal(err)
}
}
secret := filepath.Join(privateDir, "clip.srt")
if err := os.WriteFile(secret, []byte("SECRET_DENIED_SUBTITLE"), 0644); err != nil {
t.Fatal(err)
}
videoPath := filepath.Join(sharedDir, "clip.mov")
if err := os.WriteFile(videoPath, []byte("video"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(filepath.Join("..", "private", "clip.srt"), filepath.Join(sharedDir, "clip.srt")); err != nil {
t.Fatal(err)
}
if err := state.DenyUser(sourceRoot, utils.IndexPathFromNormalized("/private", true), "admin"); err != nil {
t.Fatal(err)
}
mockVideoWithExternalSubtitle(t, "clip.srt", videoPath)
adminUser.BackendScopes = []users.BackendScope{{Path: sourceRoot, Scope: "/"}}
req := httptest.NewRequest(http.MethodGet, "/public/api/media/subtitles?hash=h&path=clip.mov&name=clip.srt&embedded=false", nil)
rec := httptest.NewRecorder()
status, handlerErr := subtitlesHandler(rec, req, &Context{
User: adminUser,
ShareUser: adminUser,
IndexPath: "/shared/clip.mov/",
Share: share.Share{
ShareColumns: share.ShareColumns{
Hash: "h",
Path: "/shared",
},
SourcePath: sourceRoot,
},
})
if status != http.StatusForbidden {
t.Fatalf("status=%d err=%v body=%q", status, handlerErr, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "SECRET_DENIED_SUBTITLE") {
t.Fatalf("response leaked denied subtitle")
}
}
func TestSubtitlesHandler_ShareRejectsSidecarPastEscapingDirSymlink(t *testing.T) {
sourceRoot, adminUser := setupSubtitlesHTTPTest(t)
videoIndexPath := "/shared/link/clip.mov/"
mockVideoWithExternalSubtitle(t, "clip.srt", filepath.Join(sourceRoot, "shared", "link", "clip.mov"))
sharedDir := filepath.Join(sourceRoot, "shared")
if err := os.MkdirAll(sharedDir, 0755); err != nil {
t.Fatal(err)
}
outside := filepath.Join(filepath.Dir(sourceRoot), "outside-share")
if err := os.MkdirAll(outside, 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(outside, "clip.srt"), []byte("SECRET_PAST_DIR_SYMLINK"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(sharedDir, "link")); err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodGet, "/public/api/media/subtitles?hash=h&path=link/clip.mov&name=clip.srt&embedded=false", nil)
rec := httptest.NewRecorder()
status, handlerErr := subtitlesHandler(rec, req, &Context{
User: adminUser,
ShareUser: adminUser,
IndexPath: videoIndexPath,
Share: share.Share{
ShareColumns: share.ShareColumns{
Hash: "h",
Path: "/shared",
},
SourcePath: sourceRoot,
},
})
if status != http.StatusForbidden {
t.Fatalf("status=%d err=%v body=%q", status, handlerErr, rec.Body.String())
}
if strings.Contains(rec.Body.String(), "SECRET_PAST_DIR_SYMLINK") {
t.Fatalf("response leaked subtitle past directory symlink")
}
}
+4 -1
View File
@@ -122,7 +122,10 @@ func withHashFileHelper(fn handleFunc) handleFunc {
pathWithoutUserScope = pathWithoutUserScope + "/"
}
data.IndexPath = pathWithoutUserScope
// skip file fetch for certain apis
// Skip FileInfoFaster prefetch for performance; handlers must apply equivalent
// checks (access rules + GetRealPathScoped with d.Share.Path where bytes are read).
// TODO: align all skipped public-share routes on one bound (Share.Path) or stop
// skipping FileInfoFaster here for media/metadata/items.
if (r.Method == "POST" && strings.Contains(r.URL.Path, "/resources")) ||
(r.Method == "POST" && strings.Contains(r.URL.Path, "/resources/view-token")) ||
(r.Method == "GET" && strings.Contains(r.URL.Path, "/resources/items")) ||
+85
View File
@@ -5,7 +5,9 @@ import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
@@ -22,6 +24,7 @@ import (
)
func setupTestEnv(t *testing.T) {
lockWebGlobals(t)
utils.BcryptCost = bcrypt.MinCost
dbPath := filepath.Join(t.TempDir(), "test.sqlite")
@@ -535,6 +538,88 @@ func newHTTPRequest(t *testing.T, hash string, requestModifiers ...func(*http.Re
return r
}
func TestPublicGetResourceChecksum(t *testing.T) {
setupTestEnv(t)
filePath := filepath.Join(t.TempDir(), "a.txt")
if err := os.WriteFile(filePath, []byte("hello\n"), 0o600); err != nil {
t.Fatal(err)
}
originalFileInfoFaster := FileInfoFasterFunc
t.Cleanup(func() { FileInfoFasterFunc = originalFileInfoFaster })
FileInfoFasterFunc = func(opts utils.FileOptions, user *users.User) (*iteminfo.ExtendedFileInfo, error) {
return &iteminfo.ExtendedFileInfo{
FileInfo: iteminfo.FileInfo{
Path: opts.Path,
ItemInfo: iteminfo.ItemInfo{
Name: "a.txt",
Type: "text/plain",
},
},
RealPath: filePath,
}, nil
}
newOwner := &users.User{
FrontendUser: users.FrontendUser{
Username: "checksum_owner",
Permissions: users.Permissions{Admin: false},
},
BackendScopes: []users.BackendScope{{Path: "/srv", Scope: "/"}},
}
if err := state.CreateUser(newOwner, ""); err != nil {
t.Fatal("failed to save user:", err)
}
ownerValue, err := state.GetUserByUsername("checksum_owner")
if err != nil {
t.Fatal(err)
}
owner := &ownerValue
if err := state.CreateShare(&share.Share{
ShareSettings: share.ShareSettings{ShareLimits: share.ShareLimits{SourceName: "srv"}},
ShareColumns: share.ShareColumns{Hash: "checksum_hash", Path: "/"},
SourcePath: "/srv",
UserID: owner.ID,
}); err != nil {
t.Fatal("failed to create share:", err)
}
originalAuthKey := settings.Config.Auth.Key
settings.Config.Auth.Key = "key"
t.Cleanup(func() { settings.Config.Auth.Key = originalAuthKey })
t.Run("anonymous", func(t *testing.T) {
recorder := httptest.NewRecorder()
req := newTestRequest(t, "checksum_hash", "", "", nil)
q := req.URL.Query()
q.Set("checksum", "md5")
req.URL.RawQuery = q.Encode()
withHashFile(publicGetResourceHandler)(recorder, req)
if recorder.Code != http.StatusUnauthorized {
t.Fatalf("expected status %d, got %d body=%s", http.StatusUnauthorized, recorder.Code, recorder.Body.String())
}
})
t.Run("authenticated", func(t *testing.T) {
token := testSessionToken(t, owner, time.Hour)
recorder := httptest.NewRecorder()
req := newTestRequest(t, "checksum_hash", "", "", map[string]string{
"Authorization": "Bearer " + token,
})
q := req.URL.Query()
q.Set("checksum", "md5")
req.URL.RawQuery = q.Encode()
withHashFile(publicGetResourceHandler)(recorder, req)
if recorder.Code != http.StatusOK {
t.Fatalf("expected status %d, got %d body=%s", http.StatusOK, recorder.Code, recorder.Body.String())
}
if !strings.Contains(recorder.Body.String(), "b1946ac92492d2347c6235b4d2611184") {
t.Fatalf("checksum missing from response: %s", recorder.Body.String())
}
})
}
func TestWithTimeoutReturnsJSONTimeoutReason(t *testing.T) {
t.Parallel()
timeout := 50 * time.Millisecond
+60 -5
View File
@@ -16,6 +16,7 @@ import (
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/preview"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-logger/logger"
@@ -38,6 +39,31 @@ func isClientCancellation(ctx context.Context, err error) bool {
return errors.Is(err, context.Canceled)
}
// viewerConvertedDocExtensions are the documents the viewer opens as a rendered image through the
// "original" preview (keep in sync with pdfConvertable in frontend Preview.vue).
var viewerConvertedDocExtensions = map[string]bool{
".xps": true, ".mobi": true, ".fb2": true, ".cbz": true, ".svg": true,
".docx": true, ".pptx": true, ".xlsx": true, ".hwp": true, ".hwpx": true,
}
// isViewerOriginalPreview reports whether a preview request is the viewer opening a file the browser can't
// display by itself (camera RAW, HEIC/HEIF, rendered documents). Unlike a thumbnail, without it the file can't
// be opened at all, so it is served even when thumbnails are disabled. documentRenderable tells whether the
// server can render documents of this kind to an image (see iteminfo.HasDocConvertableExtension).
func isViewerOriginalPreview(previewSize, fileName, fileType string, documentRenderable bool) bool {
if previewSize != "original" {
return false
}
ext := strings.ToLower(filepath.Ext(fileName))
if iteminfo.IsRawImage(ext) {
return true
}
if fileType == "image/heic" || fileType == "image/heif" {
return true
}
return documentRenderable && viewerConvertedDocExtensions[ext]
}
// previewHandler handles the preview request for images.
// @Summary Get image preview
// @Description Returns a preview image based on the requested path and size.
@@ -55,7 +81,9 @@ func isClientCancellation(ctx context.Context, err error) bool {
// @Failure 501 {object} map[string]string "Preview generation not implemented"
// @Router /api/resources/preview [get]
func previewHandler(w http.ResponseWriter, r *http.Request, d *Context) (int, error) {
if settings.Config.Server.DisablePreviews {
previewsDisabled := settings.Config.Server.DisablePreviews
// Only the viewer's "original" requests can pass when previews are disabled (checked below once the file is known).
if previewsDisabled && r.URL.Query().Get("size") != "original" {
return http.StatusNotImplemented, fmt.Errorf("preview is disabled")
}
path := r.URL.Query().Get("path")
@@ -85,6 +113,9 @@ func previewHandler(w http.ResponseWriter, r *http.Request, d *Context) (int, er
return ErrToStatus(err), err
}
d.FileInfo = *fileInfo
if previewsDisabled && !isViewerOriginalPreview(r.URL.Query().Get("size"), fileInfo.Name, fileInfo.Type, iteminfo.HasDocConvertableExtension(fileInfo.Name, fileInfo.Type)) {
return http.StatusNotImplemented, fmt.Errorf("preview is disabled")
}
status, err := PreviewHelperFunc(w, r, d)
if err != nil {
// Error already logged in PreviewHelperFunc or its callees
@@ -109,7 +140,9 @@ func previewHandler(w http.ResponseWriter, r *http.Request, d *Context) (int, er
// @Failure 501 {object} map[string]string "Previews disabled globally, for this share, or for upload shares"
// @Router /public/api/resources/preview [get]
func publicPreviewHandler(w http.ResponseWriter, r *http.Request, d *Context) (int, error) {
if settings.Config.Server.DisablePreviews || d.Share.DisableThumbnails {
previewSize := r.URL.Query().Get("size")
if (settings.Config.Server.DisablePreviews || d.Share.DisableThumbnails) &&
!isViewerOriginalPreview(previewSize, d.FileInfo.Name, d.FileInfo.Type, iteminfo.HasDocConvertableExtension(d.FileInfo.Name, d.FileInfo.Type)) {
return http.StatusNotImplemented, fmt.Errorf("preview is disabled")
}
if d.Share.ShareType == "upload" {
@@ -196,13 +229,35 @@ func getDirectoryPreview(ctx context.Context, r *http.Request, d *Context, frame
// force an ffprobe probe and a subtitle scan per frame on the fast path.
fileInfo, err := files.FileInfoFaster(
utils.FileOptions{
Path: path,
Source: source,
AlbumArt: true,
Path: path,
Source: source,
AlbumArt: true,
FollowSymlinks: true,
}, user)
if err != nil {
return nil, err
}
if d.Share.Hash != "" {
idx := indexing.GetIndex(source)
if idx == nil {
return nil, fmt.Errorf("source not available")
}
userScope, scopeErr := d.ShareUser.GetScopeForSourceName(source)
if scopeErr != nil {
return nil, scopeErr
}
shareRel := strings.TrimPrefix(d.Share.Path, userScope)
bound := utils.JoinScopedIndexPath(userScope, shareRel)
childIndexPath := utils.JoinScopedIndexPath(userScope, path)
realPath, _, rerr := idx.GetRealPathScoped(bound, childIndexPath)
if rerr != nil {
if index != 0 {
return getDirectoryPreview(ctx, r, d, 0)
}
return nil, rerr
}
fileInfo.RealPath = realPath
}
tempCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
_, previewErr := preview.GetPreviewForFile(tempCtx, *fileInfo, "small", "", 0)
cancel()
@@ -0,0 +1,70 @@
package web
import (
"context"
"errors"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/share"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
commonerrors "github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
)
func TestGetDirectoryPreview_ShareRejectsSymlinkChildOutsideShare(t *testing.T) {
sourceRoot, adminUser := setupSubtitlesHTTPTest(t)
sharedDir := filepath.Join(sourceRoot, "shared")
privateDir := filepath.Join(sourceRoot, "private")
for _, dir := range []string{sharedDir, privateDir} {
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
}
if err := os.WriteFile(filepath.Join(privateDir, "secret.jpg"), []byte{0xff, 0xd8, 0xff, 0xd9}, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(filepath.Join("..", "private", "secret.jpg"), filepath.Join(sharedDir, "link.jpg")); err != nil {
t.Fatal(err)
}
ctx := context.Background()
req := httptest.NewRequest("GET", "/", nil)
adminUser.BackendScopes = []users.BackendScope{{Path: sourceRoot, Scope: "/"}}
d := &Context{
User: adminUser,
ShareUser: adminUser,
IndexPath: "/shared/",
Share: share.Share{
ShareColumns: share.ShareColumns{Hash: "h", Path: "/shared"},
SourcePath: sourceRoot,
},
FileInfo: iteminfo.ExtendedFileInfo{
Source: "Downloads",
FileInfo: iteminfo.FileInfo{
Path: "/shared/",
ItemInfo: iteminfo.ItemInfo{
Type: "directory",
},
Files: []iteminfo.ExtendedItemInfo{
{ItemInfo: iteminfo.ItemInfo{Name: "link.jpg", Type: "image/jpeg", HasPreview: true}},
},
},
},
}
idx := indexing.GetIndex("Downloads")
if idx == nil {
t.Fatal("missing index")
}
_, err := getDirectoryPreview(ctx, req, d, 0)
if err == nil {
t.Fatal("expected scope error for share child symlink outside share root")
}
if !errors.Is(err, commonerrors.ErrPathEscapesScope) {
t.Fatalf("error=%v want ErrPathEscapesScope", err)
}
}
+30
View File
@@ -9,6 +9,7 @@ import (
)
func TestGetRemoteIPUsesRightmostForwardedFor(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
@@ -27,6 +28,7 @@ func TestGetRemoteIPUsesRightmostForwardedFor(t *testing.T) {
// proxy hops so multi-hop deployments (CDN -> proxy -> filebrowser) resolve the
// real client IP instead of collapsing everyone onto the shared edge address.
func TestGetRemoteIPTrustedProxies(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
@@ -59,6 +61,7 @@ func TestGetRemoteIPTrustedProxies(t *testing.T) {
// Bare IPs (not just CIDRs) are accepted in http.trustedProxies.
func TestGetRemoteIPTrustedProxiesBareIP(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
@@ -74,6 +77,7 @@ func TestGetRemoteIPTrustedProxiesBareIP(t *testing.T) {
// Without trustProxyHeaders the XFF chain is ignored entirely.
func TestGetRemoteIPIgnoresForwardedForWhenUntrusted(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = false
@@ -90,6 +94,7 @@ func TestGetRemoteIPIgnoresForwardedForWhenUntrusted(t *testing.T) {
// A direct client that is not a trusted proxy cannot spoof its address with
// forwarded headers, even when trustProxyHeaders is enabled.
func TestGetRemoteIPIgnoresForwardedHeadersFromUntrustedPeer(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
@@ -104,8 +109,26 @@ func TestGetRemoteIPIgnoresForwardedHeadersFromUntrustedPeer(t *testing.T) {
}
}
func TestGetRemoteIPUnixSocketHonorsForwardedWithTrustAllLocalPeers(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
settings.Config.Http.Socket = "/var/run/filebrowser.sock"
settings.Config.Http.UnixSocketTrustAllLocalPeers = true
settings.Config.Http.TrustedProxies = nil
req := httptest.NewRequest("GET", "http://example.com/", nil)
req.RemoteAddr = "@"
req.Header.Set("X-Forwarded-For", "203.0.113.50")
if got := GetRemoteIP(req); got != "203.0.113.50" {
t.Fatalf("GetRemoteIP = %q, want 203.0.113.50", got)
}
}
// With no trusted proxies configured, forwarded headers are never honored.
func TestGetRemoteIPIgnoresForwardedHeadersWithoutTrustedProxies(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
@@ -121,6 +144,7 @@ func TestGetRemoteIPIgnoresForwardedHeadersWithoutTrustedProxies(t *testing.T) {
// An IPv4-mapped IPv6 trusted hop in the XFF chain must still be skipped.
func TestGetRemoteIPMappedTrustedHop(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.TrustProxyHeaders = true
@@ -135,6 +159,7 @@ func TestGetRemoteIPMappedTrustedHop(t *testing.T) {
}
func TestGetScheme(t *testing.T) {
lockWebGlobals(t)
t.Run("trusted proto", func(t *testing.T) {
settings.Config.Http.TrustProxyHeaders = true
t.Cleanup(func() { settings.Config.Http.TrustProxyHeaders = false })
@@ -158,6 +183,7 @@ func TestGetScheme(t *testing.T) {
}
func TestRequestSchemeTLSFallback(t *testing.T) {
lockWebGlobals(t)
settings.Config.Http.TrustProxyHeaders = false
req := httptest.NewRequest("GET", "https://example.com/", nil)
@@ -168,6 +194,7 @@ func TestRequestSchemeTLSFallback(t *testing.T) {
}
func TestRequestSchemeIgnoresInvalidProto(t *testing.T) {
lockWebGlobals(t)
settings.Config.Http.TrustProxyHeaders = true
t.Cleanup(func() { settings.Config.Http.TrustProxyHeaders = false })
@@ -179,6 +206,7 @@ func TestRequestSchemeIgnoresInvalidProto(t *testing.T) {
}
func TestRequestHostTrustProxyHeaders(t *testing.T) {
lockWebGlobals(t)
settings.Config.Http.TrustProxyHeaders = true
t.Cleanup(func() { settings.Config.Http.TrustProxyHeaders = false })
@@ -192,6 +220,7 @@ func TestRequestHostTrustProxyHeaders(t *testing.T) {
}
func TestRequestSchemeForPublicURL(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http.TrustProxyHeaders
t.Cleanup(func() { settings.Config.Http.TrustProxyHeaders = orig })
@@ -225,6 +254,7 @@ func TestRequestSchemeForPublicURL(t *testing.T) {
}
func TestRequestPageFullURL(t *testing.T) {
lockWebGlobals(t)
orig := settings.Config.Http
t.Cleanup(func() { settings.Config.Http = orig })
settings.Config.Http.ExternalUrl = ""
+1
View File
@@ -237,6 +237,7 @@ func TestResourcePutHandler_SuccessResponseOmitsRealPath(t *testing.T) {
func setupPutHTTPTest(t *testing.T) string {
t.Helper()
lockWebGlobals(t)
downloadsPath := filepath.Join(t.TempDir(), "downloads")
dbPath := filepath.Join(t.TempDir(), "quota-http-test.sqlite")
+5
View File
@@ -12,6 +12,7 @@ import (
"github.com/google/uuid"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
commonerrors "github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/internal/quota"
@@ -171,6 +172,10 @@ func putResourceWithQuota(w http.ResponseWriter, r *http.Request, d *Context, so
if err != nil {
return sanitizePutPathError(err)
}
permUser := accessCheckUsername(d)
if !state.AccessPermittedAtRealPath(idx, permUser, realPath, access.IsDirAtRealPath(realPath)) {
return http.StatusForbidden, fmt.Errorf("access denied to path %s", fullIndexPath)
}
if stat, statErr := os.Stat(realPath); statErr == nil && stat.IsDir() {
return http.StatusMethodNotAllowed, errPutPathIsDirectory
}
+1
View File
@@ -19,6 +19,7 @@ import (
func setupQuotaHTTPTest(t *testing.T) {
t.Helper()
lockWebGlobals(t)
dbPath := filepath.Join(t.TempDir(), "quota-http-test.sqlite")
if _, err := state.Initialize(dbPath); err != nil {
t.Fatal(err)
+51
View File
@@ -16,6 +16,7 @@ import (
"github.com/gtsteffaniak/filebrowser/backend/internal/activity"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/internal/preview"
@@ -208,7 +209,9 @@ func resourceGetHandler(w http.ResponseWriter, r *http.Request, d *Context) (int
// @Param path query string false "Path within the share to retrieve information for. Defaults to share root."
// @Param content query string false "Include file content if true"
// @Param metadata query string false "Extract audio/video metadata if true"
// @Param checksum query string false "Optional checksum algorithm (md5, sha1, sha256, sha512); requires a logged-in session"
// @Success 200 {object} iteminfo.FileInfo "File or directory metadata"
// @Failure 401 {object} map[string]string "Login required for checksum generation"
// @Failure 403 {object} map[string]string "Share unavailable or access denied"
// @Failure 404 {object} map[string]string "Share not found or file not found"
// @Failure 500 {object} map[string]string "Internal server error"
@@ -218,6 +221,24 @@ func publicGetResourceHandler(w http.ResponseWriter, r *http.Request, d *Context
if d.Share.ShareType == "upload" {
return http.StatusNotImplemented, fmt.Errorf("browsing is disabled for upload shares")
}
if algo := r.URL.Query().Get("checksum"); algo != "" {
if d.User == nil || d.User.Username == "" || d.User.Username == "anonymous" {
return http.StatusUnauthorized, fmt.Errorf("login required to generate checksums")
}
if d.FileInfo.Type == "directory" {
return http.StatusBadRequest, fmt.Errorf("checksum is only supported for files")
}
if d.Share.DisableFileViewer {
return http.StatusForbidden, fmt.Errorf("file viewing is disabled for this share")
}
checksum, err := utils.GetChecksum(d.FileInfo.RealPath, algo)
if err == errors.ErrInvalidOption {
return http.StatusBadRequest, nil
} else if err != nil {
return http.StatusInternalServerError, err
}
d.FileInfo.Checksums = map[string]string{algo: checksum}
}
return RenderJSON(w, r, d.FileInfo)
}
@@ -792,6 +813,9 @@ func ResourcePostHandler(w http.ResponseWriter, r *http.Request, d *Context) (in
if !state.AccessPermitted(idx.Path, utils.IndexPathFromNormalized(fullIndexPath, true), filePermUser.Username) {
return http.StatusForbidden, fmt.Errorf("access denied to path %s", path)
}
if !state.AccessPermittedAtRealPath(idx, filePermUser.Username, realPath, access.IsDirAtRealPath(realPath)) {
return http.StatusForbidden, fmt.Errorf("access denied to path %s", path)
}
// Check for file/folder conflicts before creation
if stat, statErr := os.Stat(realPath); statErr == nil {
@@ -1528,6 +1552,15 @@ func ResourcePatchHandler(w http.ResponseWriter, r *http.Request, d *Context) (i
response.Failed = append(response.Failed, moveCopyWithClientPaths(item, clientFromPath, clientToPath))
continue
}
if !state.AccessPermittedAtRealPath(srcIdx, d.User.Username, realSrc, isSrcDir) {
item.Message = "access denied to source path"
if d.Share.Hash != "" {
response.Failed = append(response.Failed, MoveCopyItem{Message: item.Message})
continue
}
response.Failed = append(response.Failed, moveCopyWithClientPaths(item, clientFromPath, clientToPath))
continue
}
// Resolve the destination: copy must reject escaping symlink leaves; move/rename
// must replace an existing symlink entry without following it.
@@ -1548,10 +1581,28 @@ func ResourcePatchHandler(w http.ResponseWriter, r *http.Request, d *Context) (i
response.Failed = append(response.Failed, moveCopyWithClientPaths(item, clientFromPath, clientToPath))
continue
}
if !state.AccessPermittedAtRealPath(dstIdx, d.User.Username, realDest, access.IsDirAtRealPath(realDest)) {
item.Message = "access denied to destination path"
if d.Share.Hash != "" {
response.Failed = append(response.Failed, MoveCopyItem{Message: item.Message})
continue
}
response.Failed = append(response.Failed, moveCopyWithClientPaths(item, clientFromPath, clientToPath))
continue
}
// Auto-rename if requested
if req.Rename {
realDest = addVersionSuffix(realDest)
if !state.AccessPermittedAtRealPath(dstIdx, d.User.Username, realDest, isSrcDir) {
item.Message = "access denied to destination path"
if d.Share.Hash != "" {
response.Failed = append(response.Failed, MoveCopyItem{Message: item.Message})
continue
}
response.Failed = append(response.Failed, moveCopyWithClientPaths(item, clientFromPath, clientToPath))
continue
}
}
// Validate move/rename operation to prevent circular references
@@ -0,0 +1,60 @@
package web
import (
"encoding/json"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
)
func TestResourcePatchHandler_CopyRename_DeniesDeniedVersionSuffix(t *testing.T) {
_, sourceRoot, sourceName, username := setupExtractAccessPolicyTest(t)
destDir := filepath.Join(sourceRoot, "dest")
if err := os.MkdirAll(destDir, 0o755); err != nil {
t.Fatal(err)
}
src := filepath.Join(sourceRoot, "src.txt")
if err := os.WriteFile(src, []byte("src"), 0o644); err != nil {
t.Fatal(err)
}
existing := filepath.Join(destDir, "report.txt")
if err := os.WriteFile(existing, []byte("existing"), 0o644); err != nil {
t.Fatal(err)
}
denied := filepath.Join(destDir, "report(1).txt")
if err := state.DenyUser(sourceRoot, utils.IndexPathFromNormalized("/dest/report(1).txt", false), username); err != nil {
t.Fatal(err)
}
user, err := state.GetUserByUsername(username)
if err != nil {
t.Fatal(err)
}
body := `{"action":"copy","rename":true,"items":[{"fromSource":"` + sourceName +
`","fromPath":"/src.txt","toSource":"` + sourceName + `","toPath":"/dest/report.txt"}]}`
req := httptest.NewRequest(http.MethodPatch, "/api/resources", strings.NewReader(body))
rec := httptest.NewRecorder()
status, handlerErr := ResourcePatchHandler(rec, req, &Context{User: &user})
if handlerErr != nil {
t.Fatalf("err=%v", handlerErr)
}
if status != http.StatusInternalServerError && status != http.StatusOK {
t.Fatalf("status=%d body=%s", status, rec.Body.String())
}
var resp MoveCopyResponse
if err := json.Unmarshal(rec.Body.Bytes(), &resp); err != nil {
t.Fatal(err)
}
if len(resp.Failed) != 1 || !strings.Contains(resp.Failed[0].Message, "access denied") {
t.Fatalf("failed=%+v", resp.Failed)
}
if _, err := os.Stat(denied); err == nil {
t.Fatal("denied destination file should not exist")
}
}
@@ -17,6 +17,7 @@ import (
func setupPutQuotaTest(t *testing.T) string {
t.Helper()
lockWebGlobals(t)
downloadsPath := filepath.Join(t.TempDir(), "downloads")
dbPath := filepath.Join(t.TempDir(), "quota-http-test.sqlite")
@@ -0,0 +1,82 @@
package web
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/preview"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
)
func TestResourcePostHandler_RejectsEscapingParentSymlinkMkdir(t *testing.T) {
downloadsPath := setupPutHTTPTest(t)
outside := filepath.Join(filepath.Dir(downloadsPath), "outside-mkdir")
if err := os.MkdirAll(outside, 0755); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(downloadsPath, "evil")); err != nil {
t.Fatal(err)
}
adminUser, err := state.GetUserByUsername("admin")
if err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodPost, "/api/resources?source=Downloads&path=/evil/newdir&isDir=true", nil)
rec := httptest.NewRecorder()
status, handlerErr := ResourcePostHandler(rec, req, &requestContext{User: &adminUser})
if status != http.StatusForbidden {
t.Fatalf("status=%d err=%v", status, handlerErr)
}
if handlerErr == nil {
t.Fatal("expected mkdir to be forbidden")
}
if _, err := os.Stat(filepath.Join(outside, "newdir")); !os.IsNotExist(err) {
t.Fatal("mkdir escaped outside the source via symlink parent")
}
}
func TestResourceDeleteHandler_RejectsEscapingIntermediate(t *testing.T) {
downloadsPath := setupPutHTTPTest(t)
if preview.GetService() == nil {
if err := preview.StartPreviewGenerator(1, t.TempDir()); err != nil {
t.Fatal(err)
}
}
outsideDir := filepath.Join(filepath.Dir(downloadsPath), "outside-delete")
if err := os.MkdirAll(outsideDir, 0755); err != nil {
t.Fatal(err)
}
victim := filepath.Join(outsideDir, "victim.txt")
if err := os.WriteFile(victim, []byte("keep"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outsideDir, filepath.Join(downloadsPath, "evil")); err != nil {
t.Fatal(err)
}
adminUser, err := state.GetUserByUsername("admin")
if err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodDelete, "/api/resources?source=Downloads&path=/evil/victim.txt", nil)
rec := httptest.NewRecorder()
status, handlerErr := resourceDeleteHandler(rec, req, &requestContext{User: &adminUser})
if status != http.StatusForbidden {
t.Fatalf("status=%d err=%v", status, handlerErr)
}
if handlerErr == nil {
t.Fatal("expected delete to be forbidden")
}
got, err := os.ReadFile(victim)
if err != nil || string(got) != "keep" {
t.Fatalf("delete escaped: err=%v content=%q", err, got)
}
}
+5 -1
View File
@@ -87,7 +87,11 @@ func settingsGetHandler(w http.ResponseWriter, r *http.Request, d *Context) (int
// filesystem. It is the comment/secret-marker source for both the YAML export and the settings
// JSON redactor.
func readEmbeddedConfigYaml() (string, error) {
content, err := fs.ReadFile(assetFs, "config.generated.yaml")
path := "config.generated.yaml"
if settings.Env.EmbeddedFs {
path = "embed/config.generated.yaml"
}
content, err := fs.ReadFile(assetFs, path)
if err != nil {
return "", err
}
+7 -1
View File
@@ -487,10 +487,16 @@ func ServeSingleFile(w http.ResponseWriter, r *http.Request, d *Context, source
bound = userScope
}
realPath, _, err := idx.GetRealPathScoped(bound, scopedFilePath)
realPath, isDir, err := idx.GetRealPathScoped(bound, scopedFilePath)
if err != nil {
return realPathErrStatus(err), err
}
if st, statErr := os.Stat(realPath); statErr == nil {
isDir = st.IsDir()
}
if !state.AccessPermittedAtRealPath(idx, permUser, realPath, isDir) {
return http.StatusForbidden, fmt.Errorf("access denied to path %s", scopedFilePath)
}
isOnlyOffice := IsOnlyOfficeCompatibleFile(displayFileName) && settings.Config.Integrations.OnlyOffice.Url != "" && !settings.Config.Integrations.OnlyOffice.IsCollabora()
var documentId string
+15
View File
@@ -0,0 +1,15 @@
package web
import (
"sync"
"testing"
)
// webGlobalTestMu serializes tests that mutate process-wide settings.Config or state.
var webGlobalTestMu sync.Mutex
func lockWebGlobals(t testing.TB) {
t.Helper()
webGlobalTestMu.Lock()
t.Cleanup(webGlobalTestMu.Unlock)
}
+3 -1
View File
@@ -176,6 +176,8 @@ func usersPostHandler(w http.ResponseWriter, r *http.Request, d *Context) (int,
}
// Parse the JSON into the UserRequest struct
var req UserRequest
// Seed with defaults so the decode below only overrides fields the request sent.
state.PrefillUserProfileDefaults(&req.User)
if err = json.Unmarshal(body, &req); err != nil {
return http.StatusBadRequest, err
}
@@ -199,7 +201,7 @@ func usersPostHandler(w http.ResponseWriter, r *http.Request, d *Context) (int,
return status, err
}
plaintextPassword := req.User.Password
err = state.CreateUser(&req.User, plaintextPassword)
err = state.CreateUserKeepingProfile(&req.User, plaintextPassword)
if err != nil {
return http.StatusInternalServerError, err
}
@@ -0,0 +1,73 @@
package web
import (
"bytes"
"net/http"
"net/http/httptest"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
// Omitted permissions and omitted profile keys come from userDefaults; a partially
// specified fileLoading object only overrides the keys it contains.
func TestUsersPostOmittedFieldsFallBackToDefaults(t *testing.T) {
setupTestEnv(t)
defaults := settings.Config.UserDefaults
defaults.UI.Locale = "de"
defaults.Account.Permissions.Share = true
defaults.FileLoading.MaxConcurrent = 7
defaults.FileLoading.UploadChunkSize = 9
state.SetEffectiveUserDefaultsForTest(t, defaults)
admin := createPostTestAdmin(t)
body := []byte(`{"which":[],"data":{"username":"partial","password":"secret","loginMethod":"password",
"fileLoading":{"maxConcurrentUpload":3}}}`)
req := httptest.NewRequest(http.MethodPost, "/api/users", bytes.NewReader(body))
status, err := usersPostHandler(httptest.NewRecorder(), req, &Context{User: admin})
if err != nil || status != http.StatusCreated {
t.Fatalf("usersPostHandler() status=%d err=%v", status, err)
}
got, err := state.GetUserByUsername("partial")
if err != nil {
t.Fatal(err)
}
if got.Locale != "de" {
t.Errorf("locale = %q, want default de", got.Locale)
}
if !got.Permissions.Share {
t.Errorf("share = false, want true from default permissions")
}
if got.FileLoading.MaxConcurrent != 3 {
t.Errorf("maxConcurrentUpload = %d, want requested 3", got.FileLoading.MaxConcurrent)
}
if got.FileLoading.UploadChunkSize != 9 {
t.Errorf("uploadChunkSizeMb = %d, want default 9", got.FileLoading.UploadChunkSize)
}
}
// A create request must not mutate the shared in-memory userDefaults (json.Unmarshal
// reuses slice backing arrays).
func TestUsersPostDoesNotMutateUserDefaults(t *testing.T) {
setupTestEnv(t)
defaults := settings.Config.UserDefaults
defaults.Listing.NewFileTemplate = []string{"default.txt"}
state.SetEffectiveUserDefaultsForTest(t, defaults)
admin := createPostTestAdmin(t)
body := []byte(`{"which":[],"data":{"username":"nomutate","password":"secret","loginMethod":"password",
"newFileTemplate":["custom.txt"]}}`)
req := httptest.NewRequest(http.MethodPost, "/api/users", bytes.NewReader(body))
status, err := usersPostHandler(httptest.NewRecorder(), req, &Context{User: admin})
if err != nil || status != http.StatusCreated {
t.Fatalf("usersPostHandler() status=%d err=%v", status, err)
}
if got := state.EffectiveUserDefaults().Listing.NewFileTemplate; len(got) != 1 || got[0] != "default.txt" {
t.Errorf("userDefaults newFileTemplate = %v, want [default.txt]", got)
}
}
@@ -0,0 +1,139 @@
package web
import (
"bytes"
"net/http"
"net/http/httptest"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func createPostTestAdmin(t *testing.T) *users.User {
t.Helper()
admin := &users.User{
ID: 1,
FrontendUser: users.FrontendUser{
Username: "admin",
LoginMethod: users.LoginMethodOidc,
Permissions: users.Permissions{Admin: true},
},
}
if err := state.CreateUser(admin, ""); err != nil {
t.Fatal(err)
}
return admin
}
// Issue #3322: profile/account values in POST /api/users must not be replaced by userDefaults.
func TestUsersPostKeepsRequestedProfileValues(t *testing.T) {
setupTestEnv(t)
defaults := settings.Config.UserDefaults
defaults.UI.Locale = "en"
defaults.UI.ThemeColor = "var(--blue)"
defaults.UI.DarkMode = users.DarkModeDark
defaults.Account.LockPassword = false
defaults.Account.DisableSettings = false
defaults.Account.ShowAdvancedProfile = false
defaults.Listing.PromptRightCloseButton = false
defaults.Listing.NewFileTemplate = []string{""}
defaults.Listing.ShowHidden = true
state.SetEffectiveUserDefaultsForTest(t, defaults)
admin := createPostTestAdmin(t)
body := []byte(`{"which":[],"data":{
"username":"profiletest","password":"secret","loginMethod":"password",
"locale":"zhCN","themeColor":"var(--green)","darkMode":false,
"lockPassword":true,"disableSettings":true,"showAdvancedProfile":true,
"promptRightCloseButton":true,"newFileTemplate":["custom.txt"],
"preview":{"image":false},
"permissions":{"admin":false,"api":true,"share":true,"realtime":false},
"scopes":[{"name":"/","scope":"/","permissions":{"view":true,"download":true,"modify":true,"create":true,"delete":true}}]
}}`)
req := httptest.NewRequest(http.MethodPost, "/api/users", bytes.NewReader(body))
rec := httptest.NewRecorder()
status, err := usersPostHandler(rec, req, &Context{User: admin})
if err != nil || status != http.StatusCreated {
t.Fatalf("usersPostHandler() status=%d err=%v", status, err)
}
got, err := state.GetUserByUsername("profiletest")
if err != nil {
t.Fatal(err)
}
if got.Locale != "zhCN" {
t.Errorf("locale = %q, want zhCN", got.Locale)
}
if got.ThemeColor != "var(--green)" {
t.Errorf("themeColor = %q, want var(--green)", got.ThemeColor)
}
if got.DarkMode != users.DarkModeLight {
t.Errorf("darkMode = %q, want light", got.DarkMode)
}
if !got.LockPassword {
t.Errorf("lockPassword = false, want true")
}
if !got.DisableSettings {
t.Errorf("disableSettings = false, want true")
}
if !got.ShowAdvancedProfile {
t.Errorf("showAdvancedProfile = false, want true")
}
if !got.PromptRightCloseButton {
t.Errorf("promptRightCloseButton = false, want true")
}
if len(got.NewFileTemplate) != 1 || got.NewFileTemplate[0] != "custom.txt" {
t.Errorf("newFileTemplate = %v, want [custom.txt]", got.NewFileTemplate)
}
if got.Preview.Image {
t.Errorf("preview.image = true, want false")
}
// Omitted fields still come from userDefaults.
if !got.ShowHidden {
t.Errorf("showHidden = false, want true from userDefaults")
}
if !got.Preview.Video {
t.Errorf("preview.video = false, want true from userDefaults")
}
wantPerms := users.Permissions{Api: true, Share: true}
if got.Permissions.Admin != wantPerms.Admin || got.Permissions.Api != wantPerms.Api ||
got.Permissions.Share != wantPerms.Share || got.Permissions.Realtime != wantPerms.Realtime {
t.Errorf("permissions = %+v, want %+v", got.Permissions, wantPerms)
}
}
// Enforced userDefaults still win over request values on create.
func TestUsersPostEnforcedDefaultsWinOverRequest(t *testing.T) {
setupTestEnv(t)
defaults := settings.Config.UserDefaults
defaults.UI.Locale = "en"
state.SetEffectiveUserDefaultsForTest(t, defaults)
if err := state.PatchUserDefaultsEnforced([]byte(`{"ui":{"locale":true}}`)); err != nil {
t.Fatal(err)
}
admin := createPostTestAdmin(t)
body := []byte(`{"which":[],"data":{"username":"enforced","password":"secret","loginMethod":"password","locale":"zhCN","lockPassword":true}}`)
req := httptest.NewRequest(http.MethodPost, "/api/users", bytes.NewReader(body))
rec := httptest.NewRecorder()
status, err := usersPostHandler(rec, req, &Context{User: admin})
if err != nil || status != http.StatusCreated {
t.Fatalf("usersPostHandler() status=%d err=%v", status, err)
}
got, err := state.GetUserByUsername("enforced")
if err != nil {
t.Fatal(err)
}
if got.Locale != "en" {
t.Errorf("locale = %q, want enforced en", got.Locale)
}
if !got.LockPassword {
t.Errorf("lockPassword = false, want true (not enforced)")
}
}
+39
View File
@@ -19,6 +19,7 @@ import (
"github.com/gtsteffaniak/filebrowser/backend/internal/activity"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
activitydb "github.com/gtsteffaniak/filebrowser/backend/internal/database/activity"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
commonerrors "github.com/gtsteffaniak/filebrowser/backend/internal/errors"
@@ -237,6 +238,35 @@ func (ffs *filteredFileSystem) checkScopePath(requestPath string, forRemoval boo
return err
}
// checkResolvedAccess enforces per-path DENY rules on the symlink-resolved target.
func (ffs *filteredFileSystem) checkResolvedAccess(requestPath string, forRemoval bool) error {
idx := indexing.GetIndex(ffs.source)
if idx == nil {
return fmt.Errorf("source not found")
}
userScope, err := ffs.user.GetScopeForSourceName(ffs.source)
if err != nil {
return err
}
fullIndexPath := utils.JoinScopedIndexPath(userScope, strings.TrimPrefix(requestPath, "/"))
var realPath string
if forRemoval {
realPath, err = idx.GetRealPathForRemoval(userScope, fullIndexPath)
} else {
realPath, err = idx.GetRealPathForMutation(userScope, fullIndexPath)
}
if err != nil {
if errors.Is(err, commonerrors.ErrPathEscapesScope) {
return os.ErrPermission
}
return err
}
if !state.AccessPermittedAtRealPath(idx, ffs.user.Username, realPath, access.IsDirAtRealPath(realPath)) {
return os.ErrPermission
}
return nil
}
func (ffs *filteredFileSystem) requireCreatePermissionForNewFile(requestPath string) error {
idx := indexing.GetIndex(ffs.source)
if idx == nil {
@@ -412,6 +442,9 @@ func (ffs *filteredFileSystem) Mkdir(ctx context.Context, name string, perm os.F
logger.Debugf("Mkdir: scope check denied for %s: %v", name, err)
return err
}
if err := ffs.checkResolvedAccess(name, false); err != nil {
return err
}
if err := ffs.fs.Mkdir(ctx, name, perm); err != nil {
return err
@@ -450,6 +483,9 @@ func (ffs *filteredFileSystem) OpenFile(ctx context.Context, requestPath string,
logger.Debugf("OpenFile: scope check denied for %s: %v", requestPath, err)
return nil, err
}
if err := ffs.checkResolvedAccess(requestPath, false); err != nil {
return nil, err
}
if (flag&os.O_CREATE) != 0 && !ffs.sourceFilePerms().Create {
if err := ffs.requireCreatePermissionForNewFile(requestPath); err != nil {
@@ -462,6 +498,9 @@ func (ffs *filteredFileSystem) OpenFile(ctx context.Context, requestPath string,
logger.Debugf("OpenFile: read denied by scope for %s", requestPath)
return nil, os.ErrPermission
}
if err := ffs.checkResolvedAccess(requestPath, false); err != nil {
return nil, err
}
}
var desiredMtime time.Time
+1
View File
@@ -30,6 +30,7 @@ import (
// setupWebDAVTestEnv sets up the test environment with multiple sources and users
func setupWebDAVTestEnv(t *testing.T) (string, string) {
lockWebGlobals(t)
// Create temp directories for test sources
tempDir := t.TempDir()
source1Path := filepath.Join(tempDir, "source1")
+77 -3
View File
@@ -8,6 +8,7 @@ import (
"runtime"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
@@ -24,8 +25,9 @@ import (
)
var (
RealPathCache = cache.NewCache[string](48*time.Hour, 72*time.Hour)
IsDirCache = cache.NewCache[bool](48*time.Hour, 72*time.Hour)
RealPathCache = cache.NewCache[string](48*time.Hour, 72*time.Hour)
IsDirCache = cache.NewCache[bool](48*time.Hour, 72*time.Hour)
realPathGeneration atomic.Uint64
)
// getDiskUsage returns the actual disk space used by a file in bytes
@@ -1222,6 +1224,7 @@ func (idx *Index) getRealPathInternal(boundIndexPath string, enforceScope bool,
if ok && cached != "" {
return cached, isDir, nil
}
generation := realPathGeneration.Load()
absolutePath, err := filepath.Abs(joinedPath)
if err != nil {
return absolutePath, false, fmt.Errorf("could not get real path: %v, %s", joinedPath, err)
@@ -1239,9 +1242,77 @@ func (idx *Index) getRealPathInternal(boundIndexPath string, enforceScope bool,
}
RealPathCache.Set(cacheKey, realPath)
IsDirCache.Set(cacheKey+":isdir", isDir)
if realPathGeneration.Load() != generation {
RealPathCache.Delete(cacheKey)
IsDirCache.Delete(cacheKey + ":isdir")
}
return realPath, isDir, nil
}
// realPathCacheJoinedPath builds the filesystem join used in RealPathCache keys.
func realPathCacheJoinedPath(sourceRoot, indexPath string) string {
seg := utils.CleanIndexPathSegment(indexPath)
if seg == "" {
return sourceRoot
}
return filepath.Join(sourceRoot, seg)
}
func realPathCacheIndexAncestors(indexPath string) []string {
rel := utils.CleanIndexPathSegment(indexPath)
if rel == "" {
return []string{"/"}
}
parts := strings.Split(rel, "/")
ancestors := []string{"/"}
for i := range parts {
ancestors = append(ancestors, "/"+strings.Join(parts[:i+1], "/"))
}
return ancestors
}
// ClearRealPathCaches drops every cached symlink resolution. The cache library has no
// prefix delete, so subtree invalidation uses a full clear; entries repopulate lazily.
func ClearRealPathCaches() {
realPathGeneration.Add(1)
RealPathCache.ClearAll()
IsDirCache.ClearAll()
}
// InvalidateRealPathCache drops cached symlink resolutions for indexPath and its
// ancestors. scopeBounds should include any caller scope prefixes used with
// GetRealPathScoped (e.g. user or share path).
func (idx *Index) InvalidateRealPathCache(indexPath string, scopeBounds ...string) {
realPathGeneration.Add(1)
boundSet := map[string]struct{}{"/": {}, "": {}}
for _, b := range scopeBounds {
b = strings.TrimRight(strings.TrimSpace(b), "/")
if b == "" || b == "/" {
boundSet["/"] = struct{}{}
boundSet[""] = struct{}{}
continue
}
if !strings.HasPrefix(b, "/") {
b = "/" + b
}
boundSet[b] = struct{}{}
}
bounds := make([]string, 0, len(boundSet))
for b := range boundSet {
bounds = append(bounds, b)
}
for _, ip := range realPathCacheIndexAncestors(indexPath) {
joined := realPathCacheJoinedPath(idx.Path, ip)
for _, enforceScope := range []bool{false, true} {
for _, bound := range bounds {
key := fmt.Sprintf("realpath|scoped=%t|bound=%q|path=%q", enforceScope, bound, joined)
RealPathCache.Delete(key)
IsDirCache.Delete(key + ":isdir")
}
}
}
}
// GetRealPathForMutation resolves an index path for a mutating filesystem
// operation (create/overwrite/mkdir/move-into). When the leaf exists it is fully
// resolved; otherwise the deepest existing ancestor is resolved and the missing
@@ -1603,7 +1674,10 @@ func setFilePreviewFlags(fileInfo *iteminfo.ItemInfo, realPath string) {
}
switch simpleType {
case "image":
if fileInfo.Size > iteminfo.LargeFileSizeThreshold {
// Camera RAW previews are the embedded JPEG, which is read without decoding the RAW data,
// so large RAW files (medium format, uncompressed) don't need the size limit.
isRaw := iteminfo.IsRawImage(strings.ToLower(filepath.Ext(fileInfo.Name)))
if fileInfo.Size > iteminfo.LargeFileSizeThreshold && !isRaw {
fileInfo.HasPreview = false
return
}
+34
View File
@@ -0,0 +1,34 @@
package indexing
import (
"testing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
)
func TestSetFilePreviewFlagsLargeImages(t *testing.T) {
large := iteminfo.LargeFileSizeThreshold + 1
tests := []struct {
name string
file string
typ string
size int64
want bool
}{
{"normal image under the limit", "a.jpg", "image/jpeg", 5 << 20, true},
{"normal image over the limit", "a.jpg", "image/jpeg", int64(large), false},
{"RAW under the limit", "a.raf", "image/x-fuji-raf", 26 << 20, true},
{"large RAW keeps its preview, the embedded JPEG is read without decoding", "a.RAF", "image/x-fuji-raf", int64(large), true},
{"large DNG keeps its preview", "a.dng", "image/x-adobe-dng", int64(large), true},
{"empty file has no preview", "a.raf", "image/x-fuji-raf", 0, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
info := &iteminfo.ItemInfo{Name: tt.file, Type: tt.typ, Size: tt.size}
setFilePreviewFlags(info, "/nonexistent/"+tt.file)
if info.HasPreview != tt.want {
t.Fatalf("HasPreview = %v, want %v", info.HasPreview, tt.want)
}
})
}
}
@@ -2,6 +2,7 @@ package indexing
import (
"errors"
"fmt"
"os"
"path/filepath"
"testing"
@@ -163,6 +164,66 @@ func TestGetRealPathScoped_CacheKeyDoesNotCollideWithUnscoped(t *testing.T) {
}
}
func TestRealPathCacheGeneration_BumpsOnInvalidateAndClear(t *testing.T) {
idx := symlinkTestIndex(t.TempDir())
gen := realPathGeneration.Load()
idx.InvalidateRealPathCache("/any")
if realPathGeneration.Load() != gen+1 {
t.Fatalf("InvalidateRealPathCache: gen=%d want %d", realPathGeneration.Load(), gen+1)
}
gen = realPathGeneration.Load()
ClearRealPathCaches()
if realPathGeneration.Load() != gen+1 {
t.Fatalf("ClearRealPathCaches: gen=%d want %d", realPathGeneration.Load(), gen+1)
}
}
func TestClearRealPathCaches_ClearsWarmEntries(t *testing.T) {
sourceRoot, userScope := setupSymlinkEscapeFixture(t)
idx := symlinkTestIndex(sourceRoot)
userDir := filepath.Join(sourceRoot, "user1")
if err := os.WriteFile(filepath.Join(userDir, "foo"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if _, _, err := idx.GetRealPathScoped(userScope, "/user1/foo"); err != nil {
t.Fatalf("warm cache: %v", err)
}
joined := filepath.Join(sourceRoot, "user1", "foo")
key := fmt.Sprintf("realpath|scoped=%t|bound=%q|path=%q", true, userScope, joined)
if _, ok := RealPathCache.Get(key); !ok {
t.Fatalf("expected cache key %q", key)
}
ClearRealPathCaches()
if _, ok := RealPathCache.Get(key); ok {
t.Fatal("ClearRealPathCaches should drop warm entries")
}
}
func TestInvalidateRealPathCache_ClearsGetRealPathEntry(t *testing.T) {
sourceRoot, userScope := setupSymlinkEscapeFixture(t)
idx := symlinkTestIndex(sourceRoot)
userDir := filepath.Join(sourceRoot, "user1")
if err := os.WriteFile(filepath.Join(userDir, "foo"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if _, _, err := idx.GetRealPath("/user1/foo"); err != nil {
t.Fatalf("warm cache: %v", err)
}
joined := realPathCacheJoinedPath(sourceRoot, "/user1/foo")
key := fmt.Sprintf("realpath|scoped=%t|bound=%q|path=%q", false, "/", joined)
if _, ok := RealPathCache.Get(key); !ok {
t.Fatalf("expected cache key %q", key)
}
idx.InvalidateRealPathCache("/user1/foo", userScope)
if _, ok := RealPathCache.Get(key); ok {
t.Fatal("InvalidateRealPathCache should drop unscoped cache entry")
}
}
func TestGetRealPathForMutation_ExistingInScope(t *testing.T) {
sourceRoot, userScope := setupSymlinkEscapeFixture(t)
idx := symlinkTestIndex(sourceRoot)
+9 -4
View File
@@ -120,7 +120,11 @@ func warnHttpProxyConfig() {
}
if Config.Http.TrustProxyHeaders && Config.Http.Socket != "" && !strings.HasPrefix(Config.Http.Socket, "@") &&
len(Config.Http.UnixSocketTrustedUIDs) == 0 && len(Config.Http.UnixSocketTrustedGIDs) == 0 {
logger.Warning("http.trustProxyHeaders is true and http.socket is a filesystem Unix socket, but http.unixSocketTrustedUIDs and http.unixSocketTrustedGIDs are empty; X-Forwarded-For and X-Real-IP are ignored until you list the reverse proxy's UID or GID (Linux peer credentials).")
if Config.Http.UnixSocketTrustAllLocalPeers {
logger.Warning("http.unixSocketTrustAllLocalPeers is true: any local process that can connect to the filesystem socket may present X-Forwarded-For and X-Real-IP. Prefer http.unixSocketTrustedUIDs and/or http.unixSocketTrustedGIDs so only peers with allowed Linux credentials can set forwarded headers.")
} else {
logger.Warning("http.trustProxyHeaders is true on a filesystem Unix socket, but http.unixSocketTrustedUIDs and http.unixSocketTrustedGIDs are empty; forwarded client headers from socket peers are ignored. List allowed peer UIDs and/or GIDs (Linux SO_PEERCRED), or set http.unixSocketTrustAllLocalPeers: true only if you accept legacy trust-all behavior.")
}
}
if !Env.IsDevMode {
if u := Config.Http.ExternalUrl; u != "" && strings.HasPrefix(strings.ToLower(u), "http://") {
@@ -1044,9 +1048,10 @@ func SetDefaults(generate bool) Settings {
Locale: "en",
},
FileLoading: users.FileLoading{
MaxConcurrent: 10,
UploadChunkSize: 10, // 10MB
DownloadChunkSize: 0, // 0MB, default to no chunking
MaxConcurrent: 10,
UploadChunkSize: 10, // 10MB
DownloadChunkSize: 0, // 0MB, default to no chunking
UploadStallTimeoutSeconds: 10, // pause an upload after 10 seconds without progress
},
Account: UserDefaultsAccount{
Permissions: UserDefaultsAccountPermissions{
+4 -3
View File
@@ -71,9 +71,10 @@ type UserDefaultsUIEnforcement struct {
}
type UserDefaultsFileLoadingEnforcement struct {
MaxConcurrent bool `json:"maxConcurrentUpload,omitempty"`
UploadChunkSize bool `json:"uploadChunkSizeMb,omitempty"`
DownloadChunkSize bool `json:"downloadChunkSizeMb,omitempty"`
MaxConcurrent bool `json:"maxConcurrentUpload,omitempty"`
UploadChunkSize bool `json:"uploadChunkSizeMb,omitempty"`
DownloadChunkSize bool `json:"downloadChunkSizeMb,omitempty"`
UploadStallTimeoutSeconds bool `json:"uploadStallTimeoutSeconds,omitempty"`
}
type UserDefaultsAccountEnforcement struct {
+6
View File
@@ -8,6 +8,7 @@ import (
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
"github.com/gtsteffaniak/go-ffmpeg/encode"
@@ -288,6 +289,11 @@ func ApplyMediaSettings(s FullAdminMedia) {
Config.Integrations.Media.Debug = s.Debug
Config.Integrations.Media.HardwareAcceleration = s.HardwareAcceleration
Config.Integrations.Media.AccelDevice = s.AccelDevice
if Config.Integrations.Media.AlwaysUseFfmpeg != s.AlwaysUseFfmpeg {
// The shared media cache holds values from whichever backend answered
// first. Switching backends drops them so the selected one answers fresh.
mediacache.ClearAll()
}
Config.Integrations.Media.AlwaysUseFfmpeg = s.AlwaysUseFfmpeg
Config.Integrations.Media.AlwaysUseOnlyOffice = s.AlwaysUseOnlyOffice
Config.Integrations.Media.Transcode = s.Transcode
@@ -4,6 +4,8 @@ import (
"sync"
"testing"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/mediacache"
)
func withMediaConfig(t *testing.T) {
@@ -308,3 +310,26 @@ func TestInitFFmpegIsIdempotent(t *testing.T) {
t.Fatal("InitFFmpeg must not run the probe during tests")
}
}
// TestApplyMediaSettingsClearsMediaCacheOnBackendSwitch checks that toggling
// "Always use ffmpeg" drops values the other backend cached, and that an
// unrelated settings change keeps them.
func TestApplyMediaSettingsClearsMediaCacheOnBackendSwitch(t *testing.T) {
withMediaConfig(t)
key := mediacache.Key{Path: "/settings-test.mp4", Size: 1, ModTime: 1}.String()
t.Cleanup(func() { mediacache.Durations.Delete(key) })
media := CurrentMediaSettings()
mediacache.Durations.Set(key, 1)
media.Debug = !media.Debug
ApplyMediaSettings(media)
if _, ok := mediacache.Durations.Get(key); !ok {
t.Fatal("an unrelated media settings change must keep the shared media cache")
}
media.AlwaysUseFfmpeg = !media.AlwaysUseFfmpeg
ApplyMediaSettings(media)
if _, ok := mediacache.Durations.Get(key); ok {
t.Fatal("switching Always use ffmpeg must clear the shared media cache")
}
}
+14 -10
View File
@@ -37,11 +37,14 @@ type Http struct {
TrustProxyHeaders bool `json:"trustProxyHeaders"` // honor X-Forwarded-* and X-Real-IP from a reverse proxy (default: false)
TrustedProxies []string `json:"trustedProxies"` // IPs/CIDRs of trusted proxies; forwarded client headers are only honored when the socket peer matches one of these (default: empty = forwarded headers ignored)
// UnixSocketTrustedUIDs and UnixSocketTrustedGIDs list peer credentials (Linux SO_PEERCRED) allowed to
// present X-Forwarded-* on a filesystem http.socket listener. When both are empty, forwarded headers
// are not trusted on Unix sockets. Abstract sockets (@name) never trust forwarded headers.
UnixSocketTrustedUIDs []int `json:"unixSocketTrustedUIDs"`
UnixSocketTrustedGIDs []int `json:"unixSocketTrustedGIDs"`
SocketFileMode string `json:"socketFileMode"` // octal mode for filesystem http.socket (default "0660"); ignored for abstract sockets
// present X-Forwarded-* on a filesystem http.socket listener. When both are empty, peers are not trusted
// unless UnixSocketTrustAllLocalPeers is true (legacy; startup logs a warning). A GID allowlist trusts
// every peer with that GID, not only a single reverse-proxy process. Abstract sockets (@name) never trust
// forwarded headers.
UnixSocketTrustedUIDs []int `json:"unixSocketTrustedUIDs"`
UnixSocketTrustedGIDs []int `json:"unixSocketTrustedGIDs"`
UnixSocketTrustAllLocalPeers bool `json:"unixSocketTrustAllLocalPeers"` // default false: set true to trust forwarded headers from any local filesystem socket peer when allowlists are empty
SocketFileMode string `json:"socketFileMode"` // octal mode for filesystem http.socket (default "0660"); ignored for abstract sockets
SocketGroup string `json:"socketGroup"` // optional group name or numeric gid for the socket file (filesystem sockets only)
DisableRateLimit bool `json:"disableRateLimit"` // turns off built-in auth route rate limiting and failed-login lockout (default false).
}
@@ -79,11 +82,12 @@ type Server struct {
DisableTypeDetectionByHeader bool `json:"disableTypeDetectionByHeader"` // disable type detection by header, useful if filesystem is slow.
Logging []LogConfig `json:"logging" yaml:"logging"`
Sources []*Source `json:"sources" validate:"required,dive"`
CacheDir string `json:"cacheDir"` // path to the cache directory, used for thumbnails and other cached files
CacheDirCleanup bool `json:"cacheDirCleanup"` // whether to automatically cleanup the cache directory. Note: docker must also mount a persistent volume to persist the cache (default: false)
MaxArchiveSizeGB int64 `json:"maxArchiveSize"` // maximum archive/unarchive size in GB. 0 means no limit. (default: 20)
Filesystem Filesystem `json:"filesystem"` // filesystem settings
IndexSqlConfig IndexSqlConfig `json:"indexSqlConfig"` // Index database SQL configuration
CacheDir string `json:"cacheDir"` // path to the cache directory, used for thumbnails and other cached files
CacheDirCleanup bool `json:"cacheDirCleanup"` // whether to automatically cleanup the cache directory. Note: docker must also mount a persistent volume to persist the cache (default: false)
CacheDirMaxSizeMB int64 `json:"cacheDirMaxSizeMB"` // maximum size in MB of the preview cache (cacheDir/diskcache). Past it, the least recently used files are deleted. 0 means no limit (default: 0)
MaxArchiveSizeGB int64 `json:"maxArchiveSize"` // maximum archive/unarchive size in GB. 0 means no limit. (default: 20)
Filesystem Filesystem `json:"filesystem"` // filesystem settings
IndexSqlConfig IndexSqlConfig `json:"indexSqlConfig"` // Index database SQL configuration
// not exposed to config
SourceMap map[string]*Source `json:"-" validate:"omitempty"` // uses realpath as key
NameToSource map[string]*Source `json:"-" validate:"omitempty"` // uses name as key
+8 -4
View File
@@ -113,10 +113,11 @@ func ipMatchesPrefixes(ip string, prefixes []netip.Prefix) bool {
// proxy headers. Forwarded client headers must only be honored when this returns
// true; otherwise any client that can reach the listener directly could spoof its address.
//
// When http.socket is set the server listens only on that Unix socket. Filesystem
// socket peers are trusted only when their UID or GID matches
// http.unixSocketTrustedUIDs / http.unixSocketTrustedGIDs (Linux peer credentials).
// Abstract sockets ("@name") have no filesystem permissions and their peers are never trusted.
// When http.socket is set the server listens only on that Unix socket. Abstract
// sockets ("@name") have no filesystem permissions and their peers are never trusted.
// On filesystem sockets, when either allowlist is set, only peers whose Linux SO_PEERCRED
// UID or GID matches may present forwarded headers. When both allowlists are empty,
// peers are trusted only if http.unixSocketTrustAllLocalPeers is true (legacy opt-in).
func IsTrustedProxyPeer(r *http.Request) bool {
if r == nil {
return false
@@ -125,6 +126,9 @@ func IsTrustedProxyPeer(r *http.Request) bool {
if strings.HasPrefix(socket, "@") {
return false
}
if len(Config.Http.UnixSocketTrustedUIDs) == 0 && len(Config.Http.UnixSocketTrustedGIDs) == 0 {
return Config.Http.UnixSocketTrustAllLocalPeers
}
cred, ok := UnixPeerFromContext(r.Context())
if !ok {
return false
+12 -5
View File
@@ -132,7 +132,6 @@ func TestIsTrustedProxyPeer(t *testing.T) {
}
}
// With http.socket set, forwarded headers require matching unixSocketTrustedUIDs/GIDs.
func TestIsTrustedProxyPeerUnixSocket(t *testing.T) {
resetTrustedProxyCache()
Config.Http.TrustedProxies = nil
@@ -141,15 +140,23 @@ func TestIsTrustedProxyPeerUnixSocket(t *testing.T) {
Config.Http.Socket = ""
Config.Http.UnixSocketTrustedUIDs = nil
Config.Http.UnixSocketTrustedGIDs = nil
Config.Http.UnixSocketTrustAllLocalPeers = false
resetTrustedProxyCache()
})
for _, peer := range []string{"", "@"} {
if IsTrustedProxyPeer(&http.Request{RemoteAddr: peer}) {
t.Fatalf("unix socket peer %q must not be trusted without peer credentials", peer)
}
if IsTrustedProxyPeer(&http.Request{RemoteAddr: "@"}) {
t.Fatal("filesystem unix socket with empty allowlists must not trust forwarded headers without opt-in")
}
Config.Http.UnixSocketTrustAllLocalPeers = true
if !IsTrustedProxyPeer(&http.Request{RemoteAddr: "@"}) {
t.Fatal("unixSocketTrustAllLocalPeers should restore legacy trust-all behavior")
}
Config.Http.UnixSocketTrustAllLocalPeers = false
Config.Http.UnixSocketTrustedUIDs = []int{33}
reqNoCred := &http.Request{RemoteAddr: "@"}
if IsTrustedProxyPeer(reqNoCred) {
t.Fatal("with allowlists set, peer without credentials must not be trusted")
}
req := (&http.Request{}).WithContext(ContextWithUnixPeerForTest(context.Background(), 33, 1000))
if !IsTrustedProxyPeer(req) {
t.Fatal("peer with allowed UID should be trusted")
+4
View File
@@ -7,6 +7,10 @@ import (
"net"
)
func peerCredentialFromConn(net.Conn) (UnixPeerCredential, bool) {
return UnixPeerCredential{}, false
}
func attachUnixPeerContext(ctx context.Context, _ net.Conn) context.Context {
return ctx
}
+6 -2
View File
@@ -8,8 +8,11 @@ import (
"os/user"
"strconv"
"strings"
"sync"
)
var unixListenUmaskMu sync.Mutex
type unixPeerCredKey struct{}
// UnixPeerCredential holds the peer process identity for a Unix domain connection.
@@ -69,14 +72,15 @@ func ListenUnixFilesystem(socketPath string) (net.Listener, error) {
}
setGroup = true
}
unixListenUmaskMu.Lock()
oldMask := unixUmask(0o077)
defer unixUmask(oldMask)
listener, err := net.Listen("unix", socketPath)
unixUmask(oldMask)
unixListenUmaskMu.Unlock()
if err != nil {
return nil, err
}
closeListener := func() { _ = listener.Close() }
// Assign group ownership while the socket is still owner-only (umask 077), then grant mode bits.
if setGroup {
if err := os.Chown(socketPath, -1, gid); err != nil {
closeListener()
+32 -1
View File
@@ -1,6 +1,11 @@
package settings
import "testing"
import (
"os"
"path/filepath"
"runtime"
"testing"
)
func TestParseSocketFileModeDefaults(t *testing.T) {
mode, err := parseSocketFileMode("")
@@ -51,3 +56,29 @@ func TestUnixPeerAllowed(t *testing.T) {
t.Fatal("empty allowlists must deny")
}
}
func TestListenUnixFilesystemSetsModeWithRestrictiveUmask(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("filesystem unix sockets are not used on Windows")
}
dir := t.TempDir()
socketPath := filepath.Join(dir, "fb.sock")
orig := Config.Http
Config.Http.SocketFileMode = ""
t.Cleanup(func() {
Config.Http = orig
_ = os.Remove(socketPath)
})
ln, err := ListenUnixFilesystem(socketPath)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = ln.Close() })
st, err := os.Stat(socketPath)
if err != nil {
t.Fatal(err)
}
if st.Mode().Perm() != 0o660 {
t.Fatalf("socket perm = %o, want 0660", st.Mode().Perm())
}
}
+7
View File
@@ -0,0 +1,7 @@
//go:build !unix
package settings
func unixUmask(mask int) int {
return 0
}
@@ -61,9 +61,10 @@ var previewJSONSubfields = map[string]string{
}
var fileLoadingJSONSubfields = map[string]string{
"maxConcurrentUpload": "fileLoading.maxConcurrentUpload",
"uploadChunkSizeMb": "fileLoading.uploadChunkSizeMb",
"downloadChunkSizeMb": "fileLoading.downloadChunkSizeMb",
"maxConcurrentUpload": "fileLoading.maxConcurrentUpload",
"uploadChunkSizeMb": "fileLoading.uploadChunkSizeMb",
"downloadChunkSizeMb": "fileLoading.downloadChunkSizeMb",
"uploadStallTimeoutSeconds": "fileLoading.uploadStallTimeoutSeconds",
}
// EnforcedPathsTouchedByUserJSONField returns enforcement paths a user JSON field name may affect.
+28 -1
View File
@@ -4552,6 +4552,12 @@ const docTemplate = `{
"description": "Extract audio/video metadata if true",
"name": "metadata",
"in": "query"
},
{
"type": "string",
"description": "Optional checksum algorithm (md5, sha1, sha256, sha512); requires a logged-in session",
"name": "checksum",
"in": "query"
}
],
"responses": {
@@ -4561,6 +4567,15 @@ const docTemplate = `{
"$ref": "#/definitions/iteminfo.FileInfo"
}
},
"401": {
"description": "Login required for checksum generation",
"schema": {
"type": "object",
"additionalProperties": {
"type": "string"
}
}
},
"403": {
"description": "Share unavailable or access denied",
"schema": {
@@ -6431,6 +6446,10 @@ const docTemplate = `{
"type": "string"
}
},
"unixSocketTrustAllLocalPeers": {
"description": "default false: set true to trust forwarded headers from any local filesystem socket peer when allowlists are empty",
"type": "boolean"
},
"unixSocketTrustedGIDs": {
"type": "array",
"items": {
@@ -6438,7 +6457,7 @@ const docTemplate = `{
}
},
"unixSocketTrustedUIDs": {
"description": "UnixSocketTrustedUIDs and UnixSocketTrustedGIDs list peer credentials (Linux SO_PEERCRED) allowed to\npresent X-Forwarded-* on a filesystem http.socket listener. When both are empty, forwarded headers\nare not trusted on Unix sockets. Abstract sockets (@name) never trust forwarded headers.",
"description": "UnixSocketTrustedUIDs and UnixSocketTrustedGIDs list peer credentials (Linux SO_PEERCRED) allowed to\npresent X-Forwarded-* on a filesystem http.socket listener. When both are empty, peers are not trusted\nunless UnixSocketTrustAllLocalPeers is true (legacy; startup logs a warning). A GID allowlist trusts\nevery peer with that GID, not only a single reverse-proxy process. Abstract sockets (@name) never trust\nforwarded headers.",
"type": "array",
"items": {
"type": "integer"
@@ -6965,6 +6984,10 @@ const docTemplate = `{
"description": "whether to automatically cleanup the cache directory. Note: docker must also mount a persistent volume to persist the cache (default: false)",
"type": "boolean"
},
"cacheDirMaxSizeMB": {
"description": "maximum size in MB of the preview cache (cacheDir/diskcache). Past it, the least recently used files are deleted. 0 means no limit (default: 0)",
"type": "integer"
},
"database": {
"description": "SQLite database configuration",
"allOf": [
@@ -8146,6 +8169,10 @@ const docTemplate = `{
},
"uploadChunkSizeMb": {
"type": "integer"
},
"uploadStallTimeoutSeconds": {
"description": "seconds without progress before an upload is paused, 0 uses the default (10)",
"type": "integer"
}
}
},
+28 -1
View File
@@ -4541,6 +4541,12 @@
"description": "Extract audio/video metadata if true",
"name": "metadata",
"in": "query"
},
{
"type": "string",
"description": "Optional checksum algorithm (md5, sha1, sha256, sha512); requires a logged-in session",
"name": "checksum",
"in": "query"
}
],
"responses": {
@@ -4550,6 +4556,15 @@
"$ref": "#/definitions/iteminfo.FileInfo"
}
},
"401": {
"description": "Login required for checksum generation",
"schema": {
"type": "object",
"additionalProperties": {
"type": "string"
}
}
},
"403": {
"description": "Share unavailable or access denied",
"schema": {
@@ -6420,6 +6435,10 @@
"type": "string"
}
},
"unixSocketTrustAllLocalPeers": {
"description": "default false: set true to trust forwarded headers from any local filesystem socket peer when allowlists are empty",
"type": "boolean"
},
"unixSocketTrustedGIDs": {
"type": "array",
"items": {
@@ -6427,7 +6446,7 @@
}
},
"unixSocketTrustedUIDs": {
"description": "UnixSocketTrustedUIDs and UnixSocketTrustedGIDs list peer credentials (Linux SO_PEERCRED) allowed to\npresent X-Forwarded-* on a filesystem http.socket listener. When both are empty, forwarded headers\nare not trusted on Unix sockets. Abstract sockets (@name) never trust forwarded headers.",
"description": "UnixSocketTrustedUIDs and UnixSocketTrustedGIDs list peer credentials (Linux SO_PEERCRED) allowed to\npresent X-Forwarded-* on a filesystem http.socket listener. When both are empty, peers are not trusted\nunless UnixSocketTrustAllLocalPeers is true (legacy; startup logs a warning). A GID allowlist trusts\nevery peer with that GID, not only a single reverse-proxy process. Abstract sockets (@name) never trust\nforwarded headers.",
"type": "array",
"items": {
"type": "integer"
@@ -6954,6 +6973,10 @@
"description": "whether to automatically cleanup the cache directory. Note: docker must also mount a persistent volume to persist the cache (default: false)",
"type": "boolean"
},
"cacheDirMaxSizeMB": {
"description": "maximum size in MB of the preview cache (cacheDir/diskcache). Past it, the least recently used files are deleted. 0 means no limit (default: 0)",
"type": "integer"
},
"database": {
"description": "SQLite database configuration",
"allOf": [
@@ -8135,6 +8158,10 @@
},
"uploadChunkSizeMb": {
"type": "integer"
},
"uploadStallTimeoutSeconds": {
"description": "seconds without progress before an upload is paused, 0 uses the default (10)",
"type": "integer"
}
}
},
+28 -2
View File
@@ -574,6 +574,10 @@ definitions:
items:
type: string
type: array
unixSocketTrustAllLocalPeers:
description: 'default false: set true to trust forwarded headers from any
local filesystem socket peer when allowlists are empty'
type: boolean
unixSocketTrustedGIDs:
items:
type: integer
@@ -581,8 +585,10 @@ definitions:
unixSocketTrustedUIDs:
description: |-
UnixSocketTrustedUIDs and UnixSocketTrustedGIDs list peer credentials (Linux SO_PEERCRED) allowed to
present X-Forwarded-* on a filesystem http.socket listener. When both are empty, forwarded headers
are not trusted on Unix sockets. Abstract sockets (@name) never trust forwarded headers.
present X-Forwarded-* on a filesystem http.socket listener. When both are empty, peers are not trusted
unless UnixSocketTrustAllLocalPeers is true (legacy; startup logs a warning). A GID allowlist trusts
every peer with that GID, not only a single reverse-proxy process. Abstract sockets (@name) never trust
forwarded headers.
items:
type: integer
type: array
@@ -1006,6 +1012,11 @@ definitions:
docker must also mount a persistent volume to persist the cache (default:
false)'
type: boolean
cacheDirMaxSizeMB:
description: 'maximum size in MB of the preview cache (cacheDir/diskcache).
Past it, the least recently used files are deleted. 0 means no limit (default:
0)'
type: integer
database:
allOf:
- $ref: '#/definitions/settings.Database'
@@ -1832,6 +1843,10 @@ definitions:
type: integer
uploadChunkSizeMb:
type: integer
uploadStallTimeoutSeconds:
description: seconds without progress before an upload is paused, 0 uses the
default (10)
type: integer
type: object
users.FrontendScope:
properties:
@@ -5699,6 +5714,11 @@ paths:
in: query
name: metadata
type: string
- description: Optional checksum algorithm (md5, sha1, sha256, sha512); requires
a logged-in session
in: query
name: checksum
type: string
produces:
- application/json
responses:
@@ -5706,6 +5726,12 @@ paths:
description: File or directory metadata
schema:
$ref: '#/definitions/iteminfo.FileInfo'
"401":
description: Login required for checksum generation
schema:
additionalProperties:
type: string
type: object
"403":
description: Share unavailable or access denied
schema:
+1 -1
View File
@@ -1 +1 @@
22
22.22.2
+266 -1097
View File
File diff suppressed because it is too large. Load diff
+16 -14
View File
@@ -5,7 +5,7 @@
"type": "module",
"engines": {
"npm": ">=9.0.0",
"node": "^22.13.0 || ^24.0.0 || >=26.0.0"
"node": "^22.22.2 || ^24.15.0 || >=26.0.0"
},
"scripts": {
"build": "vite build",
@@ -30,35 +30,36 @@
"test": "vitest run "
},
"dependencies": {
"@likecoin/epub-ts": "^0.7.2",
"@likecoin/epub-ts": "^0.7.3",
"@onlyoffice/document-editor-vue": "^1.7.0",
"ace-builds": "^1.45.0",
"chart.js": "^4.5.1",
"ace-builds": "^1.45.2",
"chart.js": "^4.6.1",
"dompurify": "^3.4.17",
"highlight.js": "^11.12.0",
"hls.js": "^1.7.2",
"js-yaml": "^5.5.0",
"mammoth": "^1.13.0",
"marked": "^18.1.1",
"marked": "^18.2.0",
"plyr": "^3.8.5",
"qrcode.vue": "^3.11.0",
"temml": "^0.14.0",
"three": "^0.186.1",
"vue": "^3.5.44",
"vue-i18n": "^11.4.16",
"vue-router": "^5.4.0"
"three": "^0.187.0",
"vue": "^3.5.45",
"vue-i18n": "^11.4.19",
"vue-router": "^5.5.0"
},
"overrides": {
"minimatch": "^10.2.6",
"stylelint-config-recommended": "^19.0.0",
"minimatch": "^11.0.8",
"@xmldom/xmldom": "^0.8.15",
"mammoth": {
"argparse": "^2.0.1"
"argparse": "^3.0.4"
},
"@intlify/vue-i18n-extensions": {
"vue-i18n": "$vue-i18n"
},
"@intlify/eslint-plugin-vue-i18n": {
"glob": "^13.0.0"
"glob": "^14.0.6"
}
},
"devDependencies": {
@@ -70,12 +71,13 @@
"@typescript-eslint/parser": "^8.72.0",
"@vitejs/plugin-vue": "^6.0.10",
"deepl-node": "^1.28.1",
"esbuild": "^0.28.2",
"eslint": "^10.13.0",
"eslint-plugin-vue": "^10.11.1",
"fs-extra": "^11.4.1",
"jsdom": "^30.1.3",
"stylelint": "^17.16.0",
"stylelint-config-standard": "^40.0.0",
"stylelint": "^18.0.0",
"stylelint-config-standard": "^41.0.0",
"stylelint-config-standard-vue": "^2.0.0",
"typescript": "^6.0.3",
"vite": "^8.3.4",
+1
View File
@@ -38,6 +38,7 @@ const EXTRA_ICONS = [
'create_new_folder',
'do_not_disturb_on',
'done',
'equalizer',
'error_outline',
'file',
'file_copy',
+2 -2
View File
@@ -2,7 +2,7 @@
import path from 'node:path';
import { globSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { Translator } from 'deepl-node';
import { DeepLClient } from 'deepl-node';
import fs from 'fs-extra';
// Parse command line arguments
@@ -27,7 +27,7 @@ if (requireApiKey && !DEEPL_API_KEY) {
process.exit(1);
}
const translator = requireApiKey ? new Translator(DEEPL_API_KEY) : null;
const translator = requireApiKey ? new DeepLClient(DEEPL_API_KEY) : null;
const deeplLangMap = {
'zh-cn': 'ZH-HANS',
+4 -4
View File
@@ -1,6 +1,6 @@
import { notify } from "@/notify/index.ts";
import { getApiPath, getParentDir } from "@/utils/url.js";
import { invalidateDirMetadataCache, invalidateDirMetadataItem } from "@/utils/cache/metadataCache.ts";
import { getApiPath } from "@/utils/url.js";
import { invalidateDirMetadataItem, markDirMetadataItemChanged } from "@/utils/cache/metadataCache.ts";
import { fetchURL } from "./utils.ts";
/**
@@ -35,7 +35,7 @@ export async function createArchive(opts) {
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
invalidateDirMetadataCache({ source: toSource || fromSource, path: getParentDir(destination) });
markDirMetadataItemChanged({ source: toSource || fromSource }, destination);
if (deleteAfter) {
for (const p of paths) {
invalidateDirMetadataItem({ source: fromSource }, p);
@@ -81,7 +81,7 @@ export async function unarchive(opts) {
body: JSON.stringify(body),
});
if (!preview) {
invalidateDirMetadataCache({ source: toSource || fromSource, path: destination });
invalidateDirMetadataItem({ source: toSource || fromSource }, destination);
if (deleteAfter) {
invalidateDirMetadataItem({ source: fromSource }, path);
}
+2 -2
View File
@@ -8,12 +8,12 @@
// - /api/tools/ -> tools.js
// - /api/office/ -> office.js
// - /api/wopi/ -> wopi.js
// - /api/media/ -> media.js
// - /api/media/ -> media.ts
// - /public/api/* -> public functions in respective files (e.g., resourcesApi.fetchFilesPublic)
import * as accessApi from "./access.js";
import * as authApi from "./auth.js";
import * as mediaApi from "./media.js";
import * as mediaApi from "./media.ts";
import * as officeApi from "./office.js";
import * as resourcesApi from "./resources.js";
import * as settingsApi from "./settings.js";
-247
View File
@@ -1,247 +0,0 @@
import { notify } from "@/notify/index.ts";
import { state } from "@/store/index.ts";
import { getApiPath, getPublicApiPath } from "@/utils/url.js";
import { adjustedData, fetchURL } from "./utils.ts";
import {
getDirMetadataEntry,
setCachedDirMetadata,
setDirMetadataEntryPromise,
stemOf,
takePendingDirChanges,
} from "@/utils/cache/metadataCache.ts";
// GET /api/media/subtitles or /public/api/media/subtitles
export async function getSubtitleContent(source, path, subtitleName, embedded = false) {
try {
const hash = state.shareInfo?.hash || null
const baseParams = {
path,
name: subtitleName,
...(embedded && { embedded: 'true' }),
}
let res
if (hash) {
const apiPath = getPublicApiPath('media/subtitles', {
hash,
...baseParams,
})
const sharePassword = localStorage.getItem(`sharepass:${hash}`) || ''
res = await fetchURL(apiPath, {
headers: { 'X-SHARE-PASSWORD': encodeURIComponent(sharePassword) },
})
} else {
const apiPath = getApiPath('media/subtitles', {
source,
...baseParams,
})
res = await fetchURL(apiPath)
}
const content = await res.text()
return content
} catch (err) {
notify.showError(err.message || `Error fetching subtitle ${subtitleName}`)
throw err
}
}
// GET /api/media/lyrics
export async function getLyrics(source, path) {
const apiPath = getApiPath('media/lyrics', {
source: source,
path: path,
});
const res = await fetchURL(apiPath);
const data = await res.json();
return { lyrics: data.lyrics || "", format: data.format || "" };
}
async function publicResponseError(response) {
const error = new Error(response.statusText);
try {
const data = await response.json();
if (data?.message) {
error.message = data.message;
}
} catch (_e) {
// keep the status text
}
/** @type {any} */ (error).status = response.status;
return error;
}
// GET /public/api/media/lyrics
export async function getLyricsPublic(path, hash, password = "") {
const params = {
path,
hash,
};
const apiPath = getPublicApiPath("media/lyrics", params);
const response = await fetch(apiPath, {
headers: { "X-SHARE-PASSWORD": encodeURIComponent(password || "") },
});
if (!response.ok) {
throw await publicResponseError(response);
}
const data = await response.json();
return { lyrics: data.lyrics || "", format: data.format || "" };
}
// GET /api/media/metadata — directory or file with metadata; optional albumArt for embedded cover extraction.
/** @param {boolean} albumArt when true, request embedded album art in audio metadata */
/** @returns {Promise<object>} resource (adjustedData) */
export async function fetchDirectoryMediaMetadata(source, path, albumArt = false) {
const apiPath = getApiPath("media/metadata", {
source,
path,
...(albumArt ? { albumArt: "true" } : {}),
});
const res = await fetchURL(apiPath);
const data = await res.json();
return adjustedData(data);
}
// GET /public/api/media/metadata
/** @param {boolean} albumArt when true, request embedded album art in audio metadata */
/** @returns {Promise<object>} resource (adjustedData) */
export async function fetchDirectoryMediaMetadataPublic(path, hash, password = "", albumArt = false) {
const params = {
path,
hash,
...(albumArt ? { albumArt: "true" } : {}),
};
const apiPath = getPublicApiPath("media/metadata", params);
const response = await fetch(apiPath, {
headers: { "X-SHARE-PASSWORD": encodeURIComponent(password || "") },
});
if (!response.ok) {
throw await publicResponseError(response);
}
const data = await response.json();
return adjustedData(data);
}
async function fetchWholeDirectoryMetadata({ isShare, source, hash, path, albumArt }, password) {
const payload = isShare
? await fetchDirectoryMediaMetadataPublic(path, hash, password, albumArt)
: await fetchDirectoryMediaMetadata(source, path, albumArt);
return new Map(
(payload?.items || []).filter((i) => i.metadata).map((i) => [i.name, i.metadata])
);
}
async function fetchItemMetadata({ isShare, source, hash, path, albumArt }, password, name) {
const base = !path || path === "/" ? "" : path.replace(/\/+$/, "");
const itemPath = `${base}/${name}`;
const payload = isShare
? await fetchDirectoryMediaMetadataPublic(itemPath, hash, password, albumArt)
: await fetchDirectoryMediaMetadata(source, itemPath, albumArt);
return payload && payload.type !== "directory" && payload.metadata ? payload.metadata : null;
}
// Applies the per-item changes recorded since the folder was cached, fetching only the changed items.
function resolvePendingChanges(entry, { removed, dirty, stems }, cacheOpts, password) {
const base = entry.promise;
const promise = (async () => {
const map = new Map(await base);
for (const name of removed) {
map.delete(name);
}
const toRefresh = new Set(dirty);
if (stems.size) {
for (const name of map.keys()) {
if (stems.has(stemOf(name))) {
toRefresh.add(name);
}
}
}
// returns the names that failed, others are applied
const refresh = async (names) => {
const failed = [];
await Promise.all(
names.map(async (name) => {
try {
const metadata = await fetchItemMetadata(cacheOpts, password, name);
if (metadata) {
map.set(name, metadata);
} else {
map.delete(name);
}
} catch (_e) {
failed.push(name);
}
})
);
return failed;
};
const failed = await refresh([...toRefresh]);
if (failed.length && (await refresh(failed)).length) {
return fetchWholeDirectoryMetadata(cacheOpts, password);
}
return map;
})();
setDirMetadataEntryPromise(entry, promise);
return promise;
}
/**
* Fetches media metadata, and caches it by directory, shared for all the callers to avoid unnecessary requests.
* Items changed (deleted, uploaded, moved) are patched individually instead of refetch the whole dir
* @param {string} path - directory path
* @param {{isShare?: boolean, source?: string, hash?: string, password?: string, albumArt?: boolean}} opts
* @returns {Promise<Map<string, object>>}
*/
export async function getDirectoryMetadataMap(path, opts = {}) {
const { isShare = false, source, hash, password = "", albumArt = false } = opts;
const cacheOpts = { isShare, source, hash, path, albumArt };
const entry = albumArt ? null : getDirMetadataEntry(cacheOpts);
if (entry) {
const pending = takePendingDirChanges(entry);
return pending ? resolvePendingChanges(entry, pending, cacheOpts, password) : entry.promise;
}
const promise = fetchWholeDirectoryMetadata(cacheOpts, password);
if (!albumArt) {
setCachedDirMetadata(cacheOpts, promise);
}
return promise;
}
// GET /api/media/stream — audio/video bytes via viewToken (range-based, not download-metered).
export function getStreamURL(source, path, viewToken) {
if (!source || source === undefined || source === null) {
throw new Error('no source provided')
}
if (!viewToken) {
throw new Error('view token required')
}
try {
const params = {
source: source,
file: path,
viewToken: viewToken,
sessionId: state.sessionId,
}
const apiPath = getApiPath('media/stream', params)
return window.origin + apiPath
} catch (err) {
notify.showError(err.message || 'Error getting stream URL')
throw err
}
}
// GET /public/api/media/stream
export function getStreamURLPublic(share, files, viewToken) {
if (!viewToken) {
throw new Error('view token required')
}
const fileArray = Array.isArray(files) ? files : [files]
const params = {
file: fileArray,
hash: share.hash,
viewToken: viewToken,
sessionId: state.sessionId,
}
const apiPath = getPublicApiPath('media/stream', params)
return window.origin + apiPath
}
Loaded 100 of 161 files, more files were not shown because too many files have changed in this diff. Show more