Compare commits

..
Author SHA1 Message Date
Graham Steffaniak 91bc608f0e Beta/v2.1.2 (#3142) 2026-10-13 09:58:57 -05:00
Graham Steffaniak e21d553559 updated changelog for v2.1.2 (#3140) 2026-10-13 09:53:20 -05:00
Graham Steffaniak fde6857344 Merge branch 'main' into dev/v2.1.2 2026-10-13 09:33:59 -05:00
Graham Steffaniak 37aec9ebd8 fix: reject usernames that do not map to their own home directory (#3138) 2026-10-13 09:28:42 -05:00
dl-pt 4bf1ba312a fix(auth): prevent collisions between newly issued tokens (#3135) 2026-10-12 18:14:54 -05:00
+5 c8459eb404 fix: WebDAV OPTIONS (#3123)
Co-authored-by: Kevin B. <kurami32_kbm@proton.me>
Co-authored-by: spatiumstas <79056064+spatiumstas@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Fehn <fehnomenal@fehn.systems>
Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Takahiro Shimizu <anatofuz@gmail.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
Co-authored-by: Joel Town Road <13539685+hippi345@users.noreply.github.com>
Co-authored-by: beemines <3083452174@qq.com>
Co-authored-by: Bee mines <Beemines82@gmail.com>
Co-authored-by: dweebzxx <134675545+dweebzxx@users.noreply.github.com>
Co-authored-by: Stephan Paternotte <Stephan-P@users.noreply.github.com>
2026-10-12 12:15:16 -05:00
Walid KarrayandClaude Opus 5.5 241bbd69b4 fix(download): revalidate single-file downloads so a changed file is not served stale (#3133)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 11:52:51 -05:00
Graham Steffaniak 9de21cd896 Announce v2.0.0-stable (#3126) 2026-10-11 11:52:49 -05:00
planetryanandClaude Opus 5.5 fcec78b57c add Basque (eu) translation and fix Spanish (es) strings (#3118)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 09:05:54 -05:00
Kevin B. 12f27133b5 Remove editor quick save (#3122) 2026-10-11 09:05:38 -05:00
Kevin B. 20a5d75d7e fix language switcher in builds (#3117) 2026-10-10 12:33:46 -05:00
+5 8f2e8add9a Beta/v2.1.1 (#3109)
Co-authored-by: Kevin B. <kurami32_kbm@proton.me>
Co-authored-by: spatiumstas <79056064+spatiumstas@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Fehn <fehnomenal@fehn.systems>
Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Takahiro Shimizu <anatofuz@gmail.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
Co-authored-by: Joel Town Road <13539685+hippi345@users.noreply.github.com>
Co-authored-by: beemines <3083452174@qq.com>
Co-authored-by: Bee mines <Beemines82@gmail.com>
Co-authored-by: dweebzxx <134675545+dweebzxx@users.noreply.github.com>
Co-authored-by: Stephan Paternotte <Stephan-P@users.noreply.github.com>
2026-10-09 19:18:51 -05:00
Stephan Paternotte 6e98a9d4c8 Update nl.json (#3089) 2026-10-07 14:42:20 -05:00
Andreas Fehn c74817b1ab update german translations (#3062) 2026-10-03 19:14:32 -05:00
Matt Van HornandMatt Van Horn ed9b29f2e1 fix: apply theme toggles on anonymous public shares (#3049)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-10-02 15:25:50 -05:00
+11 45a14bf2f2 Beta/v2.1.0 (#3046)
Signed-off-by: MichaIng <micha@dietpi.com>
Co-authored-by: Kevin B. <kurami32_kbm@proton.me>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Mattijs Bliek <mattijs@mattijsbliek.com>
Co-authored-by: GammelSami <39372285+GammelSami@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: yzxcj797 <54314860+yzxcj797@users.noreply.github.com>
Co-authored-by: yzxcj797 <yzxcj797@users.noreply.github.com>
Co-authored-by: DireMunchkin <dire.munchkin@proton.me>
Co-authored-by: Chris <39135018+Chri6s@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Aditya Raj Singh <github@adityaraj.com>
Co-authored-by: tbsagdic <86909795+tbsagdic@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Ethan Lee Qi Yang <pluscompanyhqDEPT@gmail.com>
Co-authored-by: Aditya Raj Singh <aditya@bncw.in>
Co-authored-by: MichaIng <micha@dietpi.com>
Co-authored-by: FandacqkaCorp <david@davidyloreto.com>
Co-authored-by: FandacqkaCorp <275647464+FandacqkaCorp@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: connorjfarrell <connorjfarrell@users.noreply.github.com>
2026-09-30 13:04:20 -05:00
e13f7075dc Beta/v2.0.9 (#3020)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: FandacqkaCorp <david@davidyloreto.com>
Co-authored-by: FandacqkaCorp <275647464+FandacqkaCorp@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-09-25 19:33:57 -05:00
670 changed files with 57262 additions and 15599 deletions

No files matched your search

+15
View File
@@ -0,0 +1,15 @@
root = true
[*]
charset = utf-8
indent_size = 2
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
end_of_line = lf
[*.go]
indent_size = 4
[{Makefile,makefile}]
indent_style = tab
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
set -euo pipefail
IMAGE_REPO="${IMAGE_REPO:-gtstef/filebrowser}"
SLIM_SUFFIX="${SLIM_SUFFIX:-}"
if [[ -z "${GITHUB_OUTPUT:-}" ]]; then
echo "GITHUB_OUTPUT is not set" >&2
exit 1
fi
write_outputs() {
local update_track="$1"
local update_major="$2"
echo "update_track=${update_track}" >> "$GITHUB_OUTPUT"
echo "update_major=${update_major}" >> "$GITHUB_OUTPUT"
}
registry_version() {
local tag="$1"
local ver err err_file
err_file="$(mktemp)"
if ! ver="$(docker buildx imagetools inspect "${IMAGE_REPO}:${tag}" \
--format '{{ with index .Image "linux/amd64" }}{{ index .Config.Labels "org.opencontainers.image.version" }}{{ end }}' 2>"$err_file")"; then
err="$(cat "$err_file")"
rm -f "$err_file"
if grep -qiE 'not found|manifest unknown' <<<"$err"; then
ver=""
else
echo "Failed to inspect ${IMAGE_REPO}:${tag}: ${err}" >&2
exit 1
fi
else
rm -f "$err_file"
if [[ -z "$ver" ]]; then
echo "Missing org.opencontainers.image.version on ${IMAGE_REPO}:${tag} (linux/amd64)" >&2
exit 1
fi
fi
ver="${ver#v}"
printf '%s' "$ver"
}
semver_allows_update() {
local new="$1"
local incumbent="$2"
if [[ -z "$incumbent" ]]; then
return 0
fi
local winner
winner="$(printf '%s\n' "$incumbent" "$new" | sort -V | tail -n1)"
[[ "$winner" == "$new" ]]
}
ref="${GITHUB_REF_NAME:-}"
ref="${ref#v}"
if [[ ! "$ref" =~ ^([0-9]+\.[0-9]+\.[0-9]+)-(.+)$ ]]; then
echo "Tag '${GITHUB_REF_NAME:-}' has no X.Y.Z-suffix form; leaving major-only floats enabled, track floats disabled"
write_outputs false true
exit 0
fi
new_version="$ref"
semver_core="${BASH_REMATCH[1]}"
suffix="${BASH_REMATCH[2]}"
major="$(echo "$semver_core" | cut -d. -f1)"
track=""
case "$suffix" in
beta) track="beta" ;;
stable) track="stable" ;;
*)
echo "Tag suffix '${suffix}' is not beta/stable; leaving major-only floats enabled, track floats disabled"
write_outputs false true
exit 0
;;
esac
track_tag="${track}${SLIM_SUFFIX}"
major_tag="${major}-${track}${SLIM_SUFFIX}"
incumbent_track="$(registry_version "$track_tag")"
incumbent_major="$(registry_version "$major_tag")"
update_track=false
if semver_allows_update "$new_version" "$incumbent_track"; then
update_track=true
fi
update_major=false
if semver_allows_update "$new_version" "$incumbent_major"; then
update_major=true
fi
echo "Floating tag policy (${SLIM_SUFFIX:-full} image):"
echo " New version: ${new_version}"
echo " Track tag: ${track_tag} (incumbent: ${incumbent_track:-<none>}) -> update_track=${update_track}"
echo " Major tag: ${major_tag} (incumbent: ${incumbent_major:-<none>}) -> update_major=${update_major}"
write_outputs "$update_track" "$update_major"
+34 -18
View File
@@ -61,18 +61,23 @@ jobs:
images: |
gtstef/filebrowser
ghcr.io/gtsteffaniak/filebrowser
- name: Decide floating Docker tags
id: floating
run: bash .github/scripts/docker-floating-tag-policy.sh
- name: Modify tags (strip 'v' and add version tags)
id: modify-json
run: |
JSON='${{ steps.meta.outputs.json }}'
MODIFIED_JSON=$(echo "$JSON" | jq '
MODIFIED_JSON=$(echo "$JSON" | jq \
--argjson updateTrack '${{ steps.floating.outputs.update_track == 'true' }}' \
--argjson updateMajor '${{ steps.floating.outputs.update_major == 'true' }}' '
.tags |= map(
sub(":v"; ":") # Strip the 'v'
| select(test(":latest") | not)
) |
# Find all version tags with suffix (X.Y.Z-suffix) and add major.minor and major tags
# Find all version tags with suffix (X.Y.Z-suffix) and add major.minor (always) and major tags (when allowed)
.tags += [
.tags[]
.tags[]
| select(test(":[0-9]+\\.[0-9]+\\.[0-9]+-"))
| split(":")
| .[-1] as $version
@@ -83,18 +88,20 @@ jobs:
| ($version_parts[0]) as $major
| "gtstef/filebrowser:\($major_minor)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major_minor)-\($suffix)",
"gtstef/filebrowser:\($major)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major)-\($suffix)"
if $updateMajor then
"gtstef/filebrowser:\($major)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major)-\($suffix)"
else empty end
] |
# Add suffix-only tags for beta (only if tag ends with -beta)
if any(.tags[]; split(":")[-1] | endswith("-beta")) then
# Add suffix-only tags for beta when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-beta")) then
.tags += [
"gtstef/filebrowser:beta",
"ghcr.io/gtsteffaniak/filebrowser:beta"
]
else . end |
# Add suffix-only tags for stable (and latest) (only if tag ends with -stable)
if any(.tags[]; split(":")[-1] | endswith("-stable")) then
# Add suffix-only tags for stable (and latest) when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-stable")) then
.tags += [
"gtstef/filebrowser:stable",
"ghcr.io/gtsteffaniak/filebrowser:stable",
@@ -146,19 +153,26 @@ jobs:
images: |
gtstef/filebrowser
ghcr.io/gtsteffaniak/filebrowser
- name: Decide floating Docker tags
id: floating
env:
SLIM_SUFFIX: -slim
run: bash .github/scripts/docker-floating-tag-policy.sh
- name: Modify tags (strip 'v' and add 'slim' if needed)
id: modify-json-slim
run: |
JSON='${{ steps.meta.outputs.json }}'
MODIFIED_JSON=$(echo "$JSON" | jq '
MODIFIED_JSON=$(echo "$JSON" | jq \
--argjson updateTrack '${{ steps.floating.outputs.update_track == 'true' }}' \
--argjson updateMajor '${{ steps.floating.outputs.update_major == 'true' }}' '
.tags |= map(
sub(":v"; ":") # Strip the 'v'
| select(test(":latest") | not) # Remove the "latest" tag
| . + "-slim" # Append "-slim" to the versioned tag
) |
# Find all version tags with suffix (X.Y.Z-suffix-slim) and add major.minor and major tags
# Find all version tags with suffix (X.Y.Z-suffix-slim) and add major.minor (always) and major tags (when allowed)
.tags += [
.tags[]
.tags[]
| select(test(":[0-9]+\\.[0-9]+\\.[0-9]+-"))
| split(":")
| .[-1] as $version
@@ -169,18 +183,20 @@ jobs:
| ($version_parts[0]) as $major
| "gtstef/filebrowser:\($major_minor)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major_minor)-\($suffix)",
"gtstef/filebrowser:\($major)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major)-\($suffix)"
if $updateMajor then
"gtstef/filebrowser:\($major)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major)-\($suffix)"
else empty end
] |
# Add suffix-only tags for beta-slim (only if tag ends with -beta-slim)
if any(.tags[]; split(":")[-1] | endswith("-beta-slim")) then
# Add suffix-only tags for beta-slim when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-beta-slim")) then
.tags += [
"gtstef/filebrowser:beta-slim",
"ghcr.io/gtsteffaniak/filebrowser:beta-slim"
]
else . end |
# Add suffix-only tags for stable-slim (only if tag ends with -stable-slim)
if any(.tags[]; split(":")[-1] | endswith("-stable-slim")) then
# Add suffix-only tags for stable-slim when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-stable-slim")) then
.tags += [
"gtstef/filebrowser:stable-slim",
"ghcr.io/gtsteffaniak/filebrowser:stable-slim"
+1
View File
@@ -46,6 +46,7 @@ frontend/.tmp-icons-*
/backend/internal/web/dist
/backend/internal/web/embed
/backend/debug
/backend/http
.DS_Store
node_modules
+174 -9
View File
@@ -2,6 +2,170 @@
All notable changes to this project will be documented in this file. For commit guidelines, please refer to [Standard Version](https://github.com/conventional-changelog/standard-version).
## v2.1.2
**Security**:
- [High] New accounts named with ".." breaks out of scoped folder on creation (GHSA-c84q-q5g6-r47f) -- reporter @dylanzonix
- [Medium] Fixed session token collision issue when multiple users login simultaneously. (GHSA-c9cr-w724-gqj3) (#3135) Thanks @dl-pt reporters: @Ishtiaque-Ahmed-Risat @AlexWhiplash @tonghuaroot
**Notes**:
- Removed "Editor quick save" setting (`userDefaults.fileViewer.quickSave`), the save button is now always present in the editor toolbar. (#3122)
- add Basque (eu) translation and fix Spanish (es) strings (#3118)
**Bug Fixes**:
- fix(webdav): OPTIONS now advertises only methods actually supported and permitted for the user; the DAV header reports class 1 only since LOCK/UNLOCK are not enabled (#3110)
- fix(download): revalidate single-file downloads so a changed file is not served stale (#3133)
- Language switcher not working (#3114) (#3117)
- Viewer showing a stale image after the file changed on disk: single-file downloads are now revalidated by the browser (#3132)
## v2.1.1
Note: this release also contains all changes from v2.0.10-beta changelog
**New Features**:
- User and group picker with partial, case-insensitive search when creating access rules (#3085), assigning groups on user edit, and choosing members on group edit.
- New `PUT /api/access/user-groups` endpoint that atomically replaces a user's group memberships.
- feat(search): configure the advanced search result limit (#3058) @beemines
- feat(listing): add sortable Date added and Type columns (#3052) @dweebzxx
- feat(archive): suggest and preview ZIP filename encodings (#3024) @AnaTofuZ
**Notes**:
- Group API breaking changes: `POST /api/access` returns 400 when `ruleCategory=group` names a nonexistent group or `ruleCategory=user` names a nonexistent user, group endpoints return 400/409 instead of 500 for client errors, and `PUT /api/access/group` accepts a `create` flag and returns `unknownMembers`. Existing database records are unaffected; no migration is required.
- update halloween theme (#3055)
- Add soft wrap option to the editor config (#3063)
- add better screenshot view handling (#3059)
- Update nl.json (#3089) @Stephan-P
- update german translations (#3062) @fehnomenal
- Update dev dependencies, add biome, fix types and linting (#3075) @Kurami32
**Bug Fixes**:
- Groups assigned in the user edit prompt now refresh the Groups settings list.
- Creating a group with an existing name now returns a conflict instead of silently overwriting its members.
- Access rules can no longer create empty groups from typos; group rules must reference an existing group.
- Empty group names are rejected and new group names must be 2-128 characters without control characters (existing records are unaffected); deleting or renaming a user now updates group memberships.
- fix(auth): use login endpoint for existing TOTP authentication (#3074)
- fix: apply theme toggles on anonymous public shares (#3049) @mvanhorn
- fix(frontend): keep source named files in extractSourceFromPath (#3099) (#3095)
- fix(auth): identify login fields for password autofill (#3077)
- fix(backend): skip denied paths in multi-file downloads, 404 when none remain (#3100) @hippi345
- fix: persist the user scope path chosen in the edit dialog (#3065) @mvanhorn
## v2.1.0
**Security**:
- [Moderate] Conflicting upload responses no longer synchronously drain the request body before returning HTTP 409; the server now closes the body and marks the connection for closure, preventing denial-of-service from clients that never send EOF (CWE-400).
- Share download links no longer create links with token, instead they link to the UI prompting for password before download. If a direct download is required, the `/api/share/direct` api exists and documented by swagger. (#2888)
**New Features**:
- Added Storage Quotas
- configurable by folder, source, user, and share.
- Administrators can create, edit, view, and delete folder quotas through the interface.
- Quota information now appears in source views, share details, sidebar progress bars, and folder prompts.
- Enhancements to the editor (#2714):
- Added a floating button to open a resizable split view with live editing for markdown files, the scrolling is synced in both sides and scrolling in single-view modes (the viewer and editor in non-split) now also tries to sync with each other so you don't lose context easily.
- Added a toolbar: This toolbar is present on all the files.
- JSON files have a button to toggle between formatted and minified versions (#2854) (#2566).
- Markdown files have a richer toolbar with lots of quick actions, for example you can now browse, navigate and insert images more easily from your current source.
- Added a prompt to configure some editor settings in UI, that prompt is accessible via the toolbar in the three-dots menu, there you can configure things like:
- Word wrap, keybinds, autocompletion, scrollbar, etc.
- The markdown viewer now supports rendering LaTeX Math and Chemistry formulas.
- Improved and fixed some styles that weren't following the app light/dark theme (#2949).
- Proxy auth group support: `groupsClaim`, `adminGroup`, and `userGroups` now control role-based access when the proxy sends a group/role header (#2755). Admin is determined by `adminGroup` only.
- Moved `auth.adminUsername` and `auth.adminPassword` to `auth.methods.password` (legacy top-level keys are migrated automatically on load).
- Support for `.elrc` (word-by-word lyrics), `.vtt`, `.srt` sidecar files for lyrics in audio files in the media player. (#2838)
- Added setting to configure the placement of the prompts close button in profile settings (#2853) (#2595).
- Added "New file templates". You can now add and configure pre-defined filenames + extension for the creation of new files in the context menu on profile settings! (#2881) (#1239).
- sidebar links can be bulk edited as yaml (#1963) (settings > user management)
- admins can customize more defaults:
- share creation defaults and enforcements (#1692) (#2434) (#2279) (#2812) (settings > share management)
- sidebar link defaults and enforcements (#2561) (settings > user management)
- tool defaults and enforcement (settings > user management)
- Add cli init CLI command (#2957)
- Sort header in file listings inside prompts like move/copy/quick jump, is now sticky. Also added a button to show a search input and filter the items of the current listing (#3028).
- Quick jump now scrolls and highlights the current item you are previewing (#3028).
- When hovering `date` and `name` in the listings, will show up the formatted/raw date (the inverse of what you have configured to display) and the full filename (#3028) (#2623).
- User groups can now be managed from the WebUI under `User Management` -> `Groups` and when editing a user (#2983) (#3040).
**Notes**:
- An Admin can remove a `defautlEnabled` source for a user and it will remove removed until an admin adds it back.
- Enhancements to cross-source copy/move permissions and behavior.
- Improved resume upload behavior.
- Play/pause on videos in mobile now is toggled by the button in the middle rather than the whole container (#2828).
- improved video thumbnail generation speed and efficiency.
- Added icons based on extension to the upload prompt and new file/folder/rename prompts (#2881).
- Profile settings show minimal options by default. Full advanced profile settings can be shown by default by enabling `account.showAdvancedProfile`.
- sidebar links/navigation is button group toggle for clearer visibility
- Pop-up preview has 200ms debounce delay so it doesn't flash from moving the cursor across files quickly.
- Added risc-v to official releases
- Polished a bit some styles across various places of the UI (#3019) (#3028).
- Moved `Access Management` under `User Management` in the UI (#3040).
- improved developer experience with native vue webhook reloads.
- improved ffmpeg handling and concurrency via library upgrades
**Bug Fixes**:
- Undo in a fresh opened file on the editor was setting the file empty (#2714)
- Added some missing styles in the markdown viewer (#2714)
- Recaptcha not working (#1925) (#2861)
- Fix disk-usage overstatement on virtiofs bind mounts (#2894)
- Support non-ASCII share passwords (#2933)
- Fall back to buffered copies when FUSE rejects fast paths (#2938)
- Preserve deleted sidebar links across restarts (#2935)
- scope padding to listing view (#2934)
- Hide the Replace option on upload/create conflict prompts when the user lacks modify permission (or when a public share disallows replacements), so create-only users are not offered an action that the server rejects (#2837)
- Preserve Ctrl-click file selection when keyboard state is stale (#2958) (#2923)
- Avoid false stalls during parallel transfers (#2950) (#2948) thanks @gudcks0305
- Cap source usage-bar percentage at 100% when indexed size exceeds partition total (#2761) (#2238)
- On Linux, source partition totals sum distinct filesystems mounted under the source root (nested mounts) so usage bars match indexed content (#2761)
- Context menu stuck when opened from a listing in a prompt like move/copy or search (#3028).
## v2.0.10
**Security**:
- [High] Restricted custom API tokens (`minimal=false`) can no longer be upgraded to a full-permission session via `POST /api/auth/renew`; renew accepts only web session tokens registered in `hashed_tokens` (GHSA-6gr6-5qpq-888p) -- thanks @tao0845.
- [High] TOTP re-enrollment via `POST /api/auth/otp/generate` and `POST /api/auth/otp/verify` no longer allows anonymous callers to replace an existing second factor using only the account password; reset or replace requires an authenticated self or admin session (first-time enrollment without MFA unchanged) (GHSA-qx86-4v5r-26g5) -- thanks @tao0845.
- [Low] Public share lyrics and subtitle media routes now honor the share's file-viewer setting, download disable flag, and download limits, matching the public download route (GHSA-p7x3-p5jj-9xfh) -- thanks Yves Soete of Blacksight LLC. @yssoe
**New Features**:
- Require password change at next login for password-based users: new user setting `requirePasswordChange` (user defaults + per-user admin toggle), login blocked until the user sets a new password; bootstrap admins with a generated initial password get this automatically ([#2977](https://github.com/gtsteffaniak/filebrowser/issues/2977)). Generated bootstrap passwords use a speakable `word-xxxxx-xx` form (random word from a fixed list plus a 5-character and 2-character code) instead of a long hex string, for easier handoff before the forced change.
- Config YAML expands `$VAR` and `${VAR}` so values such as `userPassword: "${FILEBROWSER_LDAP_USER_PASSWORD}"` work as documented (#3042).
**Notes**:
- Sidebar navigation tree rows are real hyperlinks: middle-click, Ctrl/Cmd+click, and Shift+click use the browser’s default new-tab or new-window behavior.
- Sidebar source links can now switch between aggregated usage (default) and a root-filesystem-only view via a new "Limit disk usage to source filesystem" toggle.
- [docker] upgraded ffmpeg from 9.0 to 9.0.2
- CLI setup command no longer produces full config, instead a minimal config.yaml with comments.
**Bugfixes**:
- Fixed slow or broken file listing when `http.baseURL` is a subpath (for example `/files/`): the default source redirect navigated to `/files/{source}` on top of the app base URL, producing `/files/files/{source}` and resolving the wrong storage source until the route recovered.
- Anonymous visitors on public shares could not play inline video or audio: `GET /public/api/media/stream` returned 403 because view grants were validated before share context was fully hydrated; playback now works when download and the file viewer are allowed ([#3041](https://github.com/gtsteffaniak/filebrowser/issues/3041)).
- Fixed inflated disk usage for sources spanning multiple ZFS datasets (or btrfs subvolumes): shared-pool filesystems are now grouped by pool and counted once, instead of multiplying capacity by the number of mounted datasets (#3025) (#2997) (#2761).
- LDAP `userGroups` matching now accepts CN-only config values against full `memberOf` DNs and compares case-insensitively, instead of failing the shared auth helper with strict string equality (#3044).
- OIDC session expires despite tokenExpirationHours (#3006).
- Media playback becomes stuck after opening a failing media file since v2.0.8-beta (#3031)
## v2.0.9
**Security**:
- Fresh install with default `admin`/`admin` startup generates a random initial password and logs it once (#2977). Updated documentation to follow.
- OIDC login binds OAuth `state` to an HttpOnly cookie (crypto-random) and rejects tampered callbacks; post-login redirects must be same-app relative paths.
- Login, logout, and session-expiry redirects reject open-redirect targets (`//`, `/\\`, off-site URLs); signup sends credentials in a JSON body instead of query parameters.
**Notes**:
- Webdav always shows hidden files, ignores user preference. (#3004)
- SQLite shared cache (`cache=shared`) removed from the application database connection: its table-lock conflicts bypass `busy_timeout` and fail immediately. WAL mode already allows concurrent readers.
**Bugfixes**:
- Fixed intermittent blank page / blocked inline SPA script after CSP hardening (#2995)
- ffmpeg unresponsive lock issue (#2996)
- fixed token migration lockout regression from v2.0.8-beta (#2999)
- Public shares with OnlyOffice disabled (`enableOnlyOffice=false`) now reject `/office/config` and `/office/callback` requests server-side instead of only hiding the editor in the UI
- Session/API token registrations now record the JWT expiry; expired mappings are pruned at startup, and tokens expired beyond the same 2-minute grace window used for session rotation no longer resolve a user identity on public share routes
- Startup now fails when the configured/env auth signing key differs from the key persisted in the application database, instead of warning and minting tokens with divergent keys
- OnlyOffice document downloads now re-validate redirect targets against the configured document-server host, closing an SSRF gap where a redirect could send the fetch to an internal address
- OnlyOffice "closed with changes" callbacks now keep the document key until the save succeeds, so a failed save can be retried by the document server instead of being rejected as an unknown session
- External JWT (JwtAuth) requests now reuse an existing valid session cookie instead of minting and registering a new session token on every request
- SQLite index and application databases configure `busy_timeout` on every pooled connection, and busy/locked detection now uses driver result codes; index batch writes and maintenance no longer report success when the DB stays busy, and index cache reads return busy errors instead of empty results so listing can fall back to the filesystem.
## v2.0.8
**Security**:
@@ -28,7 +192,7 @@ All notable changes to this project will be documented in this file. For commit
- Add cli init CLI command (#2957)
**Notes**:
- Pop-up preview has 200ms debounce delay so it doesn't flash from moving the cursor across files quickly.
- Pop-up preview has 200ms debounce delay so it doesn't flash from moving the cursor across files quickly.
- Added risc-v to official releases
**Bugfixes**:
@@ -108,6 +272,7 @@ All notable changes to this project will be documented in this file. For commit
- External subtitles fail to load on public video shares due to authenticated subtitle endpoint (#2822) (#2827)
- OnlyOffice is inaccessible on password-protected shares (#2811)
- FFmpeg 9.0 incorrectly detected as below minimum 5.0.0 on Windows (#2820) -- thanks @yzxcj797
- Queue buttons not clickable in the desktop panel in audio files.
## v2.0.1
@@ -181,7 +346,7 @@ This version represents the most significant change to date. It **requires** bot
- opt-in feature to send deployment analytics to filebrowser quantum developer servers
- anonymized with a viewer so users can see what info would be sent.
- if opt-in, every month a snapshot of your deployment config would be sent to developer servers
- this will help me know what features are being used and what versions everyone is on over time. I will also provide a public dashboard with this information in the future.
- this will help me know what features are being used and what versions everyone is on over time. I will also provide a public dashboard with this information in the future.
- WebDAV now supports set modification time via the `X-OC-Mtime` header for clients that support it (#2626). See [WebDAV docs](https://filebrowserquantum.com/en/docs/features/webdav/).
- Copy operations now preserve their original modification times (#2642) (#2647):
- WebUI preserves both, files and directories.
@@ -361,14 +526,14 @@ This version represents the most significant change to date. It **requires** bot
- Improved style of drag and drop into listing view (#2407)
- Edit Sidebar links has new "show tools in sidebar" toggle and all users have this enabled by default. can be disabled via for new users `userDefaults.sidebar.showTools: false`
- Update user defaults ordering (#1140)
- Save view modes and sizes into local storage instead of db (#2301)
- Save view modes and sizes into local storage instead of db (#2301)
**BugFixes**:
- Blue overlay when using gestures in video files on mobile (#2360)
- Playback queue wasn't updating when changing of folder (#2360)
- Added missing `UserDefaults.listing.showCopyPath` option to config file (#2364)
- Delete prompt thumbnail didn't worked when using it with the `delete` shortcut. (#2407)
- Searching icons in the Icon picker prompt wasn't working properly (#2407)
- Searching icons in the Icon picker prompt wasn't working properly (#2407)
- Drag and drop style state was stuck until page reload (#2407)
- Navigate close settings shows "something went wrong" (#2047)
@@ -514,7 +679,7 @@ This version represents the most significant change to date. It **requires** bot
- Cannot edit shared file in OnlyOffice #2143
- PWA install button disappeared (#2086)
- Deleting a root folder was possible #2128
- PUT resource api errors if action against a folder (#2153)
- PUT resource api errors if action against a folder (#2153)
- LDAP authentication issue if a password caontains @ symbol (#2154)
- Share banner seems to be not working for custom urls (#2120)
@@ -548,7 +713,7 @@ Note: a potentially breaking change for docker users: \the default user is now "
- New Sidebar Features
- Sidebar tree navigation (#2006) (#350)
- Source usage to be customized to show os-reported values rather than calculated. This can be changed per source by editing the source link in the sidebar. (#1266) (#982)
- Archive/Unarchive actions in UI (#1252) (#335) (#1569)
- Archive/Unarchive actions in UI (#1252) (#335) (#1569)
- new api to archive/unarchive files on the server
- requires `create` user permissions
- archiving actions respect `server.maxArchiveSize`
@@ -711,7 +876,7 @@ Note: a potentially breaking change for docker users: \the default user is now "
## v1.2.1-beta
**New Features**:
- Global disable onlyoffice editor via "*" file option to disable all files for a specific user. (#1533)
- Global disable onlyoffice editor via "*" file option to disable all files for a specific user. (#1533)
**Notes**:
- upgraded imaging package and improved thumbnail generation performance. (#1797) (#1850)
@@ -814,7 +979,7 @@ Sorry for the delay -- a lot of effort went into this release, specifically abou
- warning for low free space below 20GB
- warning if read/write latency is slow
- fatal for any errors reading or writing to the directory
- links to official docs for more info
- links to official docs for more info
- access rule changes:
- denied folders won't show up in parent directory listing view (#1684)
- tools will respect access rules
@@ -849,7 +1014,7 @@ Sorry for the delay -- a lot of effort went into this release, specifically abou
**BugFixes**:
- better index status updates, fixing delays #1649
- fixed long load times for listings with media info due sequential processing of files.
- fixed long load times for listings with media info due sequential processing of files.
- downloaded files always included `utf-8` in filename #1671
- custom sidebar links allow external links like `https://google.com`
- html title not populated correctly for links #1676
+10 -10
View File
@@ -18,7 +18,7 @@
:pushpin: [Read The Official Docs](https://filebrowserquantum.com/) (currently english-only)
:pushpin: [v2.0.0 is now in beta!](https://filebrowserquantum.com/en/docs/getting-started/v2/about/)
:pushpin: [v2.0.0 is now stable!](https://filebrowserquantum.com/en/docs/getting-started/v2/about/) --> See [migration support discussion](https://github.com/gtsteffaniak/filebrowser/discussions/3105)
## About
@@ -86,7 +86,7 @@ See the [Official Docs](https://filebrowserquantum.com/). Contributions are welc
## Comparison Chart
Application Name | <img width="48" alt="" src="https://github.com/user-attachments/assets/c40b22c9-33da-47b7-bc4c-ce69bb5cc174" > Quantum | <img width="48" alt="" src="https://github.com/filebrowser/filebrowser/blob/master/frontend/public/img/logo.svg" > Filebrowser | <img width="48" alt="" src="https://github.com/mickael-kerjean/filestash/blob/master/public/assets/logo/app_icon.png?raw=true" > Filestash | <img width="48" alt="" src="https://avatars.githubusercontent.com/u/19211038?s=200&v=4" > Nextcloud | <img width="48" alt="" src="https://cdn.iconscout.com/icon/free/png-256/free-google-drive-logo-icon-svg-download-png-2476481.png" > Google_Drive | <img width="48" alt="" src="https://avatars.githubusercontent.com/u/6422152?v=4" > FileRun
--- | --- | --- | --- | --- | --- | --- |
Filesystem support | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
Filesystem support | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
Linux | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
Windows | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
Mac | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
@@ -95,7 +95,7 @@ Has Stable Release? | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
S3 support | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
webdav support | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
FTP support | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ |
Dedicated docs site? | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
Dedicated docs site? | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Multiple sources at once | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
Docker image size | 180 MB (with ffmpeg) | 31 MB | 240 MB (main image) | 250 MB | ❌ | > 2 GB |
Min. Memory Requirements | 512 MB | 128 MB | 128 MB (main image) | 512 MB | ❌ | 512 MB |
@@ -109,25 +109,25 @@ Content-aware search? | ❌ | ❌ | ✅ | configurable | ✅ | ✅ |
Custom job support | :construction: | ✅ | ✅ | ✅ | ❌ | ✅ |
Multiple users | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Single sign-on support | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
LDAP sign-on support | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
LDAP sign-on support | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
Long-live API key support | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
API documentation page | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
Mobile App | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ |
API documentation page | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
Mobile App | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ |
open source? | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
tags support | :construction: | ❌ | ✅ | ✅ | ❌ | ✅ |
shareable web links? | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Event-based notifications | :construction: | ❌ | ✅ | ❌ | ❌ | ✅ |
Metrics | :construction: | ❌ | ✅ | ❌ | ❌ | ❌ |
file space quotas | :construction: | ❌ | ✅ | ❌ | ✅ | ✅ |
Metrics | :construction: | ❌ | ✅ | ✅ | ❌ | ❌ |
file space quotas | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ |
text-based files editor | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Office file support | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
Office file previews | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ |
Themes | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
Themes | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
Branding support | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ |
activity log | ✅ | ❌ | ✅ | ✅ | ✅ | ✅
Comments support | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
trash support | :construction: | ❌ | ✅ | ✅ | ✅ | ✅ |
Starred/pinned files | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ |
Starred/pinned files | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |
Chromecast support | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
Share collections of files | :construction: | ❌ | ❌ | ❌ | ❌ | ✅ |
Can archive selected files | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ |
+1 -1
View File
@@ -1,4 +1,4 @@
FROM gtstef/ffmpeg:9.0-decode AS ffmpeg
FROM gtstef/ffmpeg:9.0.2-decode AS ffmpeg
FROM golang:alpine AS base
ARG VERSION
ARG REVISION
+1 -1
View File
@@ -4,4 +4,4 @@ WORKDIR /app/frontend
COPY frontend/package-lock.json /tmp/package-lock.json
RUN PW_VERSION=$(node -p "require('/tmp/package-lock.json').packages['node_modules/@playwright/test'].version") \
&& npm i "@playwright/test@${PW_VERSION}" oauth2-mock-server @types/node
RUN npx playwright install --with-deps chromium
RUN npx playwright install --with-deps chromium-headless-shell
+1
View File
@@ -15,4 +15,5 @@ COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
# Run tests
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
ENV FILEBROWSER_ADMIN_PASSWORD=playwright-password
RUN ./filebrowser & sleep 2 && cd ../frontend && npx playwright test
+3
View File
@@ -5,6 +5,9 @@ http:
frontend:
<<: *frontend-main
auth:
<<: *auth-main
userDefaults:
preview:
image: true
@@ -21,4 +21,7 @@ server-main: &server-main
defaultEnabled: true
rules:
- folderPath: "/"
viewable: true
viewable: true
auth-main: &auth-main
adminPassword: "playwright-password"
+2 -1
View File
@@ -4,7 +4,8 @@ http:
<<: *http-main
frontend:
<<: *frontend-main
auth:
<<: *auth-main
userDefaults:
preview:
image: true
@@ -21,4 +21,7 @@ server-main: &server-main
defaultEnabled: true
rules:
- folderPath: "/"
viewable: true
viewable: true
auth-main: &auth-main
adminPassword: "playwright-password"
@@ -1,6 +1,9 @@
http:
port: 8080
baseURL: "/files/"
auth:
adminPassword: "playwright-password"
server:
logging:
- levels: "info|error|warning|debug"
@@ -10,3 +13,10 @@ server:
name: "playwright"
config:
defaultEnabled: true
userDefaults:
listing:
viewMode: "normal"
# Size 4: switch-view "gallery" family resolves to icons (not gallery); list family → list.
gallerySize: 3
account:
showAdvancedProfile: true
@@ -12,7 +12,7 @@ import type { PlaywrightFixtureOptions } from "./tests/playwright/test-setup";
*/
export default defineConfig<PlaywrightFixtureOptions>({
globalSetup: "./tests/playwright/screenshots-setup.ts",
timeout: 5000,
timeout: 60_000,
testDir: "./tests/playwright/screenshots",
/* Run tests in files in parallel */
fullyParallel: false,
+1
View File
@@ -41,6 +41,7 @@ frontend:
title: "Playwright"
auth:
adminPassword: "playwright-password"
methods:
password:
signup: true
+3
View File
@@ -16,6 +16,9 @@ server:
private: true
defaultEnabled: true
auth:
adminPassword: "playwright-password"
frontend:
name: "Graham's Filebrowser"
disableDefaultLinks: true
+1
View File
@@ -0,0 +1 @@
FILEBROWSER_DEVMODE=true
+10 -4
View File
@@ -1,16 +1,22 @@
root = "."
tmp_dir = "tmp"
env_files = [".air.env"]
[build]
pre_cmd = ["go run ./scripts/setup-gofitz-cgo"]
cmd = "go build -o ./tmp/filebrowser --tags=mupdf --ldflags=\"-w -s -X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA=commitSHA' -X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=version'\" ."
cmd = "bash ./scripts/air-build.sh"
entrypoint = "./tmp/filebrowser"
full_bin = "FILEBROWSER_DEVMODE=true CGO_ENABLED=1 ./tmp/filebrowser -c test_config.yaml"
args_bin = ["-c", "test_config.yaml"]
include_ext = ["go", "tpl", "tmpl", "html"]
exclude_dir = ["tmp", "vendor", "internal/web/dist", "internal/web/embed", "internal/preview/testdata", "internal/preview/thumbnails", "internal/preview/heic", "swagger/docs"]
exclude_dir = ["tmp", "vendor", "internal/web/dist", "internal/web/embed", "internal/preview/testdata", "internal/preview/thumbnails", "internal/preview/heic", "swagger/docs", "internal/preview/gofitzinclude"]
poll = true
poll_interval = 1000
log = "air.log"
delay = 1000 # ms
[build.windows]
cmd = "bash ./scripts/air-build.sh"
entrypoint = "./tmp/filebrowser.exe"
[log]
time = true
-22
View File
@@ -1,22 +0,0 @@
root = "."
tmp_dir = "tmp"
[build]
pre_cmd = ["go run ./scripts/setup-gofitz-cgo"]
cmd = "go build -o ./tmp/filebrowser.exe --tags=mupdf --ldflags=\"-w -s -X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA=commitSHA' -X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=version'\" ."
entrypoint = "./tmp/filebrowser.exe"
full_bin = "FILEBROWSER_DEVMODE=true CGO_ENABLED=1 ./tmp/filebrowser.exe -c test_config.yaml"
include_ext = ["go", "tpl", "tmpl", "html"]
exclude_dir = ["tmp", "vendor", "internal\\web\\dist", "internal\\web\\embed", "internal\\preview\\testdata", "internal\\preview\\thumbnails", "internal\\preview\\heic", "swagger\\docs"]
exclude_unchanged = true
follow_symlink = false
poll = true
poll_interval = 1000
log = "air.log"
delay = 1000 # ms
[log]
time = true
[misc]
clean_on_exit = false
+5 -5
View File
@@ -28,7 +28,7 @@ require (
github.com/OpenPeeDeeP/depguard/v2 v2.2.1 // indirect
github.com/air-verse/air v1.67.4 // indirect
github.com/alecthomas/chroma/v2 v2.27.0 // indirect
github.com/alecthomas/go-check-sumtype v0.4.0 // indirect
github.com/alecthomas/go-check-sumtype v0.5.1-0.20260828200218-ae6904d28606 // indirect
github.com/alexkohler/nakedret/v2 v2.0.6 // indirect
github.com/alexkohler/prealloc v1.1.0 // indirect
github.com/alfatraining/structtag v1.0.0 // indirect
@@ -54,7 +54,7 @@ require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charithe/durationcheck v0.0.11 // indirect
github.com/charmbracelet/colorprofile v0.4.3 // indirect
github.com/charmbracelet/ultraviolet v0.0.0-20260910203606-6c9e17dc7a16 // indirect
github.com/charmbracelet/ultraviolet v0.0.0-20260922123528-4e49372c11f9 // indirect
github.com/charmbracelet/x/ansi v0.11.8 // indirect
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/charmbracelet/x/termios v0.1.1 // indirect
@@ -105,7 +105,7 @@ require (
github.com/golangci/dupl v0.0.0-20260401084720-c99c5cf5c202 // indirect
github.com/golangci/go-printf-func-name v0.1.1 // indirect
github.com/golangci/gofmt v0.0.0-20260820135601-e84e05053792 // indirect
github.com/golangci/golangci-lint/v2 v2.13.2 // indirect
github.com/golangci/golangci-lint/v2 v2.14.0 // indirect
github.com/golangci/golines v0.15.0 // indirect
github.com/golangci/misspell v0.8.0 // indirect
github.com/golangci/plugin-module-register v0.1.2 // indirect
@@ -150,7 +150,7 @@ require (
github.com/mattn/go-colorable v0.1.15 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mattn/go-runewidth v0.0.30 // indirect
github.com/mgechev/revive v1.16.0 // indirect
github.com/mgechev/revive v1.17.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/moricho/tparallel v0.3.2 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
@@ -200,7 +200,7 @@ require (
github.com/swaggo/swag v1.16.6 // indirect
github.com/tdewolff/parse/v2 v2.8.16 // indirect
github.com/tetafro/godot v1.5.6 // indirect
github.com/timakin/bodyclose v0.0.0-20260723120731-857993a2939c // indirect
github.com/timakin/bodyclose v0.0.0-20260923065923-98ded9ab4ef2 // indirect
github.com/timonwong/loggercheck v0.12.0 // indirect
github.com/tomarrell/wrapcheck/v2 v2.12.0 // indirect
github.com/tommy-muehle/go-mnd/v2 v2.5.1 // indirect
+10
View File
@@ -58,6 +58,8 @@ github.com/alecthomas/chroma/v2 v2.27.0 h1:FodwmyOBgJULFYmDqibcp9pvfDLWdtPRh9v/r
github.com/alecthomas/chroma/v2 v2.27.0/go.mod h1:NjJ3ciIgrqBNeIkWZ4e46nseoLDslxU1LmfCoL+wcY8=
github.com/alecthomas/go-check-sumtype v0.4.0 h1:gd4mE4QsA62sXlnFIs8GUv8/0xH3DOUqTkYQR1l2v28=
github.com/alecthomas/go-check-sumtype v0.4.0/go.mod h1:aYYllI8IXNjo7ZYUzI2+wyLRECMg3zo+8CZj9JTBAWU=
github.com/alecthomas/go-check-sumtype v0.5.1-0.20260828200218-ae6904d28606 h1:dXPJ+uOJLWb8rjlAuLX7JPiRJtZKw/53Ol+WxdDJ1Eo=
github.com/alecthomas/go-check-sumtype v0.5.1-0.20260828200218-ae6904d28606/go.mod h1:ZH31IJyNpxpStxP2ayCcPQz+3T2G/OnxIQqe4LWvSDw=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alexkohler/nakedret/v2 v2.0.6 h1:ME3Qef1/KIKr3kWX3nti3hhgNxw6aqN5pZmQiFSsuzQ=
@@ -140,6 +142,8 @@ github.com/charmbracelet/colorprofile v0.4.3 h1:QPa1IWkYI+AOB+fE+mg/5/4HRMZcaXex
github.com/charmbracelet/colorprofile v0.4.3/go.mod h1:/zT4BhpD5aGFpqQQqw7a+VtHCzu+zrQtt1zhMt9mR4Q=
github.com/charmbracelet/ultraviolet v0.0.0-20260910203606-6c9e17dc7a16 h1:/hSCZO1puetbGd2JikHKN0nAcb2mFevzxJElFoduhaw=
github.com/charmbracelet/ultraviolet v0.0.0-20260910203606-6c9e17dc7a16/go.mod h1:JFvmQPZh3uAu7PpSZU91bcR0BrIy24HhW+0EsW1uGxE=
github.com/charmbracelet/ultraviolet v0.0.0-20260922123528-4e49372c11f9 h1:ZGHc3eaN2TIhkpwRibtr8N9jLQJOY1rh6GEByRoRmp4=
github.com/charmbracelet/ultraviolet v0.0.0-20260922123528-4e49372c11f9/go.mod h1:D9SVXVpAsD1pP6xR6BiERbe4+dUU6EW9US1TFrNutGA=
github.com/charmbracelet/x/ansi v0.11.8 h1:JMFwp0CgDC2+jcOB162HH5k7I3FVbgFSMMYg7dSPBQQ=
github.com/charmbracelet/x/ansi v0.11.8/go.mod h1:ZNN+3mXny/516oTQPLMPIBeSINvNJJQ8uQXDgbeJxY0=
github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk=
@@ -292,6 +296,8 @@ github.com/golangci/gofmt v0.0.0-20260820135601-e84e05053792 h1:WL8YKrt3UbOBqSRU
github.com/golangci/gofmt v0.0.0-20260820135601-e84e05053792/go.mod h1:te5hX0dW4C5r6YbXs+6ysNr8Q5UTmdIqGbb+mlFiYmA=
github.com/golangci/golangci-lint/v2 v2.13.2 h1:bCyq3E4vo9qwzifjpzJqYndEt7Ncva80qkk8G2B4TXU=
github.com/golangci/golangci-lint/v2 v2.13.2/go.mod h1:5xaMd1kAxV7GSBPEyngw8gnnjoRocqW9UVKdBR6627w=
github.com/golangci/golangci-lint/v2 v2.14.0 h1:ot8QffRa4LzAAEgtvNYrVs9esxQ7xoTwoAv6uhp4ngA=
github.com/golangci/golangci-lint/v2 v2.14.0/go.mod h1:amHimCzNDP1GfVno3hIiNssjvqXmuW+CPZAHOdR4cIc=
github.com/golangci/golines v0.15.0 h1:Qnph25g8Y1c5fdo1X7GaRDGgnMHgnxh4Gk4VfPTtRx0=
github.com/golangci/golines v0.15.0/go.mod h1:AZjXd23tbHMpowhtnGlj9KCNsysj72aeZVVHnVcZx10=
github.com/golangci/misspell v0.8.0 h1:qvxQhiE2/5z+BVRo1kwYA8yGz+lOlu5Jfvtx2b04Jbg=
@@ -416,6 +422,8 @@ github.com/mattn/go-runewidth v0.0.30 h1:+KUuiDA4fF0R1p5FeueHefjDm+GIM+kWfFnDjyb
github.com/mattn/go-runewidth v0.0.30/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/mgechev/revive v1.16.0 h1:9OZVO8xOPUwIYynplHXv7liMllPQ/BTNZLp07sM/Im0=
github.com/mgechev/revive v1.16.0/go.mod h1:nT854jMmvQsS/qqxEaMPM4pJSUpxhcPoHM788ZxDvdQ=
github.com/mgechev/revive v1.17.0 h1:hwTTVC3svSv9X2FA2hiYT17Uws+lfxDWhPxzBZZCREg=
github.com/mgechev/revive v1.17.0/go.mod h1:uiVNUF8ruVh4WHm/aPs7kKEeuo+m6oO3tuOfniyzP/g=
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
@@ -569,6 +577,8 @@ github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZ
github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0=
github.com/timakin/bodyclose v0.0.0-20260723120731-857993a2939c h1:d/FUP8GD4QiY/rdFzZmBu3KqL9eBNVzMu7W/5pGkaMM=
github.com/timakin/bodyclose v0.0.0-20260723120731-857993a2939c/go.mod h1:sDHLK7rb/59v/ZxZ7KtymgcoxuUMxjXq8gtu9VMOK8M=
github.com/timakin/bodyclose v0.0.0-20260923065923-98ded9ab4ef2 h1:HECVafB2AKntWWRfsiQ4gdunLx5WOYQzFdCsxqOrzp8=
github.com/timakin/bodyclose v0.0.0-20260923065923-98ded9ab4ef2/go.mod h1:JLoPVtROtNgJ99yXTIxuGNttvmJa85tkJAO6lr05qBA=
github.com/timonwong/loggercheck v0.12.0 h1:7Cb9XGLJZSWYNjgL459DIhpSiEHupD+MLQ6vaN2Pl3g=
github.com/timonwong/loggercheck v0.12.0/go.mod h1:Wa6Y+SJHQR01mViZlLLY5GAZVOWOLNMbkXddMjFWQCs=
github.com/tomarrell/wrapcheck/v2 v2.12.0 h1:H/qQ1aNWz/eeIhxKAFvkfIA+N7YDvq6TWVFL27Of9is=
+38 -20
View File
@@ -8,7 +8,6 @@ import (
"strconv"
"strings"
"github.com/goccy/go-yaml"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
@@ -225,19 +224,11 @@ func createConfig(configpath string, noInput bool) error {
fmt.Printf("Error: '%s' is not a valid port. Please enter a number between 1 and 65535.\n", portStr)
}
for {
levels := askQuestion(reader, "What should the log levels be?", "info|warning|error")
checkLevels := SplitByMultiple(levels)
invalidOptions := []string{}
for _, level := range checkLevels {
if !(level == "info" || level == "warning" || level == "error" || level == "debug") {
invalidOptions = append(invalidOptions, level)
}
}
if len(invalidOptions) == 0 {
break
}
fmt.Printf("Error: invalid options given '%s'. valid options: 'info|warning|error|debug'.\n", invalidOptions)
config.Server.Logging = []settings.LogConfig{
{
Output: "stdout",
Levels: "info|warning|error",
},
}
for {
@@ -250,9 +241,20 @@ func createConfig(configpath string, noInput bool) error {
}
fmt.Printf("Error: '%s' is not a valid path. Please enter a path to a file ending in .sqlite", config.Server.DatabaseV2.Path)
}
config.Frontend.Name = askQuestion(reader, "What should the application brand name be?", "FileBrowser Quantum")
config.Auth.AdminUsername = askQuestion(reader, "What should the default admin username be?", "admin")
config.Auth.AdminPassword = askQuestion(reader, "What should the default admin password be?", "admin")
const defaultFrontendName = "FileBrowser Quantum"
brandName := askQuestion(reader, "What should the application brand name be?", defaultFrontendName)
if brandName != defaultFrontendName {
config.Frontend.Name = brandName
}
adminUsername := askQuestion(reader, "What should the default admin username be?", "admin")
if adminUsername != "admin" {
config.Auth.Methods.PasswordAuth.AdminUsername = adminUsername
}
const adminPasswordGenerateDefault = "generate on startup"
adminPassword := askQuestion(reader, "What should the default admin password be?", adminPasswordGenerateDefault)
if adminPassword != "" && !strings.EqualFold(adminPassword, adminPasswordGenerateDefault) {
config.Auth.Methods.PasswordAuth.AdminPassword = adminPassword
}
modifyDefault := askYesNoQuestion(reader, "Should a new user be able to modify content by default?", "no")
config.Server.Sources[0].Config.DefaultPermissions = settings.NormalizeSourceFilePermissions(users.SourceFilePermissions{
@@ -266,12 +268,12 @@ func createConfig(configpath string, noInput bool) error {
fmt.Println("--- Configuration Complete ---")
yamlData, err := yaml.Marshal(&config)
yamlData, err := generateSetupConfigYAML(&config)
if err != nil {
return fmt.Errorf("marshaling config: %w", err)
return fmt.Errorf("generating config YAML: %w", err)
}
err = os.WriteFile(configpath, yamlData, 0600)
err = os.WriteFile(configpath, []byte(yamlData), 0600)
if err != nil {
return fmt.Errorf("writing config file: %w", err)
}
@@ -291,6 +293,22 @@ func createConfig(configpath string, noInput bool) error {
return nil
}
func generateSetupConfigYAML(config *settings.Settings) (string, error) {
embeddedPaths := []string{
"frontend/public/config.generated.yaml",
"../frontend/public/config.generated.yaml",
"internal/web/dist/config.generated.yaml",
"../internal/web/dist/config.generated.yaml",
}
for _, path := range embeddedPaths {
embeddedYaml, err := os.ReadFile(path)
if err == nil && len(embeddedYaml) > 0 {
return settings.GenerateConfigYamlWithEmbedded(config, true, false, false, string(embeddedYaml))
}
}
return settings.GenerateConfigYaml(config, true, false, false)
}
func generateYaml() {
generateConfig := os.Getenv("FILEBROWSER_GENERATE_CONFIG") == "true"
if generateConfig || settings.Env.IsDevMode {
+24
View File
@@ -9,6 +9,7 @@ import (
"testing"
"github.com/alecthomas/kong"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -191,3 +192,26 @@ func TestSetupNoInputRejected(t *testing.T) {
require.Error(t, err)
assert.Contains(t, err.Error(), "interactive input")
}
func TestGenerateSetupConfigYAML_MinimalWithComments(t *testing.T) {
t.Chdir("..")
cfg := settings.SetDefaults(false)
cfg.Server.Sources = []*settings.Source{{Path: "/var/lib/files"}}
cfg.Server.DatabaseV2.Path = "./filebrowser.sqlite"
cfg.Http.Port = 8080
cfg.Server.Logging = []settings.LogConfig{
{Output: "stdout", Levels: "info|warning|error"},
}
out, err := generateSetupConfigYAML(&cfg)
require.NoError(t, err)
assert.Contains(t, out, "#")
assert.Contains(t, out, "/var/lib/files")
assert.Contains(t, out, "8080")
assert.NotContains(t, out, "FileBrowser Quantum")
assert.NotContains(t, out, "darkMode")
assert.NotContains(t, out, "minSearchLength:")
assert.NotContains(t, out, "frontend:")
assert.NotContains(t, out, "rules:")
}
+12
View File
@@ -0,0 +1,12 @@
package cmd
import (
"os"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/testutil"
)
func TestMain(m *testing.M) {
os.Exit(testutil.Main(m))
}
+31 -2
View File
@@ -296,8 +296,9 @@ func migrateUsers(oldDB *storm.DB, sqlStore *sqldb.SQLStore) error {
users.MigrateToSourcePermissions(user)
}
normalizeUserTokensBeforeSQLite(user)
promoteLegacyApiKeysBeforeSQLite(user)
updateTokens(user)
user.Version = users.ProfileStorageVersion
user.Version = users.NewestUserVersion
if newScopesCount > oldScopesCount {
promoted++
logger.Infof(" user %q: legacy database had %d scopes, SQLite now has %d",
@@ -347,6 +348,34 @@ func normalizeUserTokensBeforeSQLite(user *users.User) {
user.Tokens = normalized
}
// promoteLegacyApiKeysBeforeSQLite merges Bolt apiKeys entries into Tokens so
// they survive the SQLite migration and the hashed-tokens backfill (strict
// auth ignores BelongsTo and requires a hash row per raw JWT).
func promoteLegacyApiKeysBeforeSQLite(user *users.User) {
if len(user.ApiKeys) == 0 {
return
}
if user.Tokens == nil {
user.Tokens = make(map[string]users.AuthToken)
}
for name, token := range user.ApiKeys {
if token.Name == "" {
token.Name = name
}
if token.Token == "" {
token.Token = token.Key
}
if token.Token == "" {
continue
}
if _, exists := user.Tokens[token.Name]; exists {
continue
}
token.Permissions = users.SanitizeTokenPermissions(token.Permissions)
users.StoreToken(user.Tokens, token)
}
}
// migrateShares migrates all shares from BoltDB to SQLite.
//
// Legacy Bolt "Link" / "Share" JSON (flat CommonShare + Link fields):
@@ -476,7 +505,7 @@ func migrateAccessRules(oldDB *storm.DB, sqlStore *sqldb.SQLStore) error {
logger.Warningf(" skipping hashed token: invalid user id 0")
continue
}
err := sqlStore.SaveHashedToken(tokenHash, uint64(userID), false)
err := sqlStore.SaveHashedToken(tokenHash, uint64(userID), false, 0)
if err != nil {
return fmt.Errorf("failed to save hashed token: %w", err)
}
+86
View File
@@ -2,6 +2,7 @@ package cmd
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"testing"
@@ -72,3 +73,88 @@ func TestMigrateAdminTokensRoundTripSQLite(t *testing.T) {
os.Stdout.Write(out)
os.Stdout.Write([]byte("\n"))
}
func TestPromoteLegacyApiKeysBeforeSQLite(t *testing.T) {
user := &users.User{
FrontendUser: users.FrontendUser{Username: "legacy-tokens"},
Tokens: map[string]users.AuthToken{},
}
user.ApiKeys = map[string]users.AuthToken{
"legacy-sync": {Key: "raw-jwt-1"},
"empty": {},
}
user.Tokens["obsidian"] = users.AuthToken{Name: "obsidian", Token: "raw-jwt-existing"}
promoteLegacyApiKeysBeforeSQLite(user)
// Existing Tokens entries win; empty raw tokens are skipped.
if got := user.Tokens["obsidian"].Token; got != "raw-jwt-existing" {
t.Fatalf("existing token must win, got %q", got)
}
if got := user.Tokens["legacy-sync"].Token; got != "raw-jwt-1" {
t.Fatalf("promoted token must carry raw JWT, got %q", got)
}
if _, ok := user.Tokens["raw-jwt-1"]; !ok {
t.Fatal("expected raw-JWT alias key after promotion")
}
if _, ok := user.Tokens["empty"]; ok {
t.Fatal("token without raw material must be skipped")
}
}
func TestUpdateTokenHashBackfillSuccessStampsVersion(t *testing.T) {
user := &users.User{FrontendUser: users.FrontendUser{Username: "no-tokens"}}
user.Version = users.ProfileStorageVersion
changed, failed := updateTokenHashBackfill(user)
if failed {
t.Fatal("tokenless user must not fail backfill")
}
if !changed {
t.Fatal("expected version bump to count as changed")
}
if user.Version != users.TokenHashBackfillVersion {
t.Fatalf("version = %d, want %d", user.Version, users.TokenHashBackfillVersion)
}
}
func TestUpdateTokenHashBackfillFailureSkipsBump(t *testing.T) {
// Force token hash registration to fail; the version must stay below
// newest so the backfill retries on next startup.
orig := addApiToken
addApiToken = func(string, uint64) error {
return errors.New("simulated token registration failure")
}
t.Cleanup(func() { addApiToken = orig })
user := &users.User{FrontendUser: users.FrontendUser{Username: "backfill-retry"}}
user.Version = users.ProfileStorageVersion
user.ApiKeys = map[string]users.AuthToken{
"legacy": {Key: "raw-jwt-retry"},
}
_, failed := updateTokenHashBackfill(user)
if !failed {
t.Fatal("expected failure when token registration fails")
}
if user.Version != users.ProfileStorageVersion {
t.Fatalf("failed backfill must not bump version, got %d", user.Version)
}
if bumpToNewestVersion(user, failed) {
t.Fatal("catch-all must not bump past a failed migration")
}
}
func TestBumpToNewestVersion(t *testing.T) {
user := &users.User{FrontendUser: users.FrontendUser{Username: "stale"}}
user.Version = users.ProfileStorageVersion
if !bumpToNewestVersion(user, false) {
t.Fatal("expected bump for stale version without failure")
}
if user.Version != users.NewestUserVersion {
t.Fatalf("version = %d, want %d", user.Version, users.NewestUserVersion)
}
if bumpToNewestVersion(user, false) {
t.Fatal("must not bump when already newest")
}
}
+2 -2
View File
@@ -200,9 +200,9 @@ func rootCMD(ctx context.Context, serverConfig *settings.Server, a *app.App, shu
if err != nil {
logger.Fatalf("Failed to create sub filesystem: %v", err)
}
fileutils.InitAssetFS(subAssets, true)
fileutils.InitAssetFS(subAssets, true, false)
} else {
fileutils.InitAssetFS(nil, false)
fileutils.InitAssetFS(nil, false, settings.Env.IsDevMode)
}
// Start preview service
+58 -13
View File
@@ -16,6 +16,10 @@ import (
var createBackup = false
// addApiToken registers a raw API token's hash with access storage. It is a
// variable so tests can inject a failing registration deterministically.
var addApiToken = state.AddApiToken
func validateUserInfo(newDB bool) {
// update source info for users if names/sources/paths might have changed
usersList, err := state.GetAllUsers()
@@ -55,20 +59,22 @@ func validateUserInfo(newDB bool) {
if normalizeApiTokenPermissions(user) {
changedFields = append(changedFields, "tokens")
}
tokenBackfillChanged, tokenBackfillFailed := updateTokenHashBackfill(user)
if tokenBackfillChanged && !tokenBackfillFailed {
changedFields = append(changedFields, "tokens", "version")
} else if tokenBackfillChanged {
changedFields = append(changedFields, "tokens")
}
if state.ApplyEnforcedSyncToUser(user) {
changedFields = append(changedFields, settings.UserJSONFieldsForEnforcedSync()...)
}
if state.ApplyEnforcedSourcePermissionsSyncToUser(user) {
changedFields = append(changedFields, "backendScopes", "backendSourcePermissions")
}
if user.Version < users.ProfileStorageVersion {
user.Version = users.ProfileStorageVersion
if bumpToNewestVersion(user, tokenBackfillFailed) {
changedFields = append(changedFields, "version")
}
adminUser := settings.Config.Auth.AdminUsername
if adminUser == "" {
adminUser = "admin"
}
adminUser := settings.PasswordAdminUsername()
if user.Username == adminUser && user.Permissions.Admin {
adminPerms := settings.AdminPerms()
if user.Permissions.Share != adminPerms.Share || user.Permissions.Api != adminPerms.Api {
@@ -78,10 +84,10 @@ func validateUserInfo(newDB bool) {
changedFields = append(changedFields, "permissions")
}
}
if user.Username == adminUser && settings.Config.Auth.AdminPassword != "" && user.LoginMethod == users.LoginMethodPassword {
if user.Username == adminUser && settings.ConfigTriggersAdminPasswordReset() && user.LoginMethod == users.LoginMethodPassword {
logger.Info("Resetting admin user to default username and password.")
user.Permissions = settings.AdminPerms()
user.Password = settings.Config.Auth.AdminPassword
user.Password = settings.PasswordAdminPassword()
user.LoginMethod = users.LoginMethodPassword
changedFields = append(changedFields, "permissions", "password", "loginMethod")
changePass = true
@@ -129,7 +135,7 @@ func dedupeFields(fields []string) []string {
}
func updateUserScopes(user *users.User) bool {
newScopes := settings.MergeDefaultEnabledBackendScopes(user.BackendScopes)
newScopes := settings.MergeDefaultEnabledBackendScopes(user.BackendScopes, user.DeclinedDefaultSources)
changed := !reflect.DeepEqual(user.BackendScopes, newScopes)
user.BackendScopes = newScopes
return changed
@@ -162,7 +168,7 @@ func updateShowFirstLogin(user *users.User) bool {
// func to convert legacy user with perm key to permissions
func updatePermissions(user *users.User) bool {
if user.Version >= 1 {
if user.Version >= users.PermMigrationVersion {
return false
}
updateUser := true
@@ -208,7 +214,7 @@ func updatePermissions(user *users.User) bool {
user.Permissions.Delete = true
updateUser = true
}
user.Version = 2
user.Version = users.PermMigrationVersion
if updateUser {
createBackup = true
}
@@ -248,7 +254,7 @@ func updateSidebarLinks(user *users.User, scopesChanged bool) bool {
}
func updateTokens(user *users.User) bool {
if user.Version >= 2 {
if user.Version >= users.TokenMigrationVersion {
return false
}
if user.ApiKeys != nil {
@@ -259,7 +265,7 @@ func updateTokens(user *users.User) bool {
users.StoreToken(user.Tokens, token)
}
}
user.Version = 2
user.Version = users.TokenMigrationVersion
return true
}
@@ -281,3 +287,42 @@ func normalizeApiTokenPermissions(user *users.User) bool {
}
return changed
}
// updateTokenHashBackfill promotes legacy ApiKeys into Tokens and registers a
// hashed_tokens row for every stored raw JWT so strict server-side auth (which
// ignores BelongsTo) accepts pre-2.0.8 tokens. Runs once per user via
// TokenHashBackfillVersion. It reports (changed, failed): when hash
// registration fails the version is left alone so callers skip the newest
// bump and the backfill retries on next startup.
func updateTokenHashBackfill(user *users.User) (bool, bool) {
if user == nil || user.Version >= users.TokenHashBackfillVersion {
return false, false
}
before := len(user.Tokens)
promoteLegacyApiKeysBeforeSQLite(user)
changed := len(user.Tokens) != before
hashesRegistered := true
for _, raw := range state.CollectStoredRawTokens(user) {
if err := addApiToken(raw, user.ID); err != nil {
logger.Errorf("could not register token hash for user %s: %v", user.Username, err)
hashesRegistered = false
}
}
if hashesRegistered {
user.Version = users.TokenHashBackfillVersion
createBackup = true
return true, false
}
return changed, true
}
// bumpToNewestVersion stamps the user at NewestUserVersion. It is skipped when
// a migration failed so the failed migration retries on next startup instead
// of being buried by the catch-all bump.
func bumpToNewestVersion(user *users.User, migrationFailed bool) bool {
if migrationFailed || user == nil || user.Version >= users.NewestUserVersion {
return false
}
user.Version = users.NewestUserVersion
return true
}
+1 -1
View File
@@ -18,7 +18,6 @@ server:
userDefaults:
fileViewer:
editorQuickSave: false
disableOnlyOfficeExt: ".md .txt .pdf"
disableViewingExt: ""
sidebar:
@@ -53,6 +52,7 @@ userDefaults:
lockPassword: false
disableSettings: false
disableUpdateNotifications: false
showAdvancedProfile: false
permissions:
api: false
admin: false
+15 -16
View File
@@ -9,17 +9,17 @@ require (
github.com/coreos/go-systemd/v22 v22.7.0
github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8
github.com/dsoprea/go-exif/v3 v3.0.1
github.com/evanoberholster/imagemeta v1.0.0
github.com/evanoberholster/imagemeta v1.1.0
github.com/gen2brain/go-fitz v1.28.2
github.com/go-ldap/ldap/v3 v3.4.14
github.com/go-playground/validator/v10 v10.30.4
github.com/go-webauthn/webauthn v0.18.1
github.com/go-playground/validator/v10 v10.30.5
github.com/go-webauthn/webauthn v0.18.2
github.com/goccy/go-yaml v1.19.2
github.com/golang-jwt/jwt/v4 v4.5.2
github.com/google/go-cmp v0.7.0
github.com/google/uuid v1.6.0
github.com/gtsteffaniak/go-cache v1.1.0
github.com/gtsteffaniak/go-ffmpeg v1.0.0
github.com/gtsteffaniak/go-ffmpeg v1.2.0
github.com/gtsteffaniak/go-logger v1.1.0
github.com/kovidgoyal/imaging v1.8.23
github.com/mattn/go-sqlite3 v1.14.52
@@ -35,10 +35,11 @@ require (
golang.org/x/oauth2 v0.37.0
golang.org/x/sys v0.48.0
golang.org/x/term v0.46.0
golang.org/x/text v0.42.0
golang.org/x/time v0.16.0
golang.org/x/tools v0.50.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.59.0
modernc.org/sqlite v1.60.1
)
require (
@@ -48,14 +49,14 @@ require (
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd // indirect
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 // indirect
github.com/dustin/go-humanize v1.1.0 // indirect
github.com/ebitengine/purego v0.11.0 // indirect
github.com/ebitengine/purego v0.11.1 // indirect
github.com/fxamacker/cbor/v2 v2.9.4 // indirect
github.com/gabriel-vasile/mimetype v1.4.15 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8 // indirect
github.com/go-errors/errors v1.5.1 // indirect
github.com/go-jose/go-jose/v4 v4.1.5 // indirect
github.com/go-openapi/jsonpointer v1.0.1 // indirect
github.com/go-openapi/jsonreference v1.0.2 // indirect
github.com/go-openapi/jsonpointer v1.0.2 // indirect
github.com/go-openapi/jsonreference v1.0.3 // indirect
github.com/go-openapi/spec v1.0.1 // indirect
github.com/go-openapi/swag/conv v0.29.2 // indirect
github.com/go-openapi/swag/jsonutils v0.29.2 // indirect
@@ -64,12 +65,12 @@ require (
github.com/go-openapi/swag/stringutils v0.29.2 // indirect
github.com/go-openapi/swag/typeutils v0.29.2 // indirect
github.com/go-openapi/swag/yamlutils v0.29.2 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/locales v0.14.2 // indirect
github.com/go-playground/universal-translator v0.18.2 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/go-webauthn/x v0.3.1 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/golang/geo v0.0.0-20260818125358-b200a1149890 // indirect
github.com/golang/geo v0.0.0-20260928092222-7d12f68cfadb // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/go-tpm v0.9.8 // indirect
@@ -78,24 +79,23 @@ require (
github.com/kr/pretty v0.3.1 // indirect
github.com/leodido/go-urn v1.5.0 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/ncruces/go-strftime v1.1.0 // indirect
github.com/philhofer/fwd v1.2.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rogpeppe/go-internal v1.16.0 // indirect
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd // indirect
github.com/swaggo/files v1.0.1 // indirect
github.com/tinylib/msgp v1.6.4 // indirect
github.com/tinylib/msgp v1.6.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
go.etcd.io/bbolt v1.5.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/text v0.42.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/protobuf v1.36.12 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
modernc.org/libc v1.76.0 // indirect
modernc.org/libc v1.77.1 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
@@ -103,5 +103,4 @@ require (
exclude (
github.com/alecthomas/go-check-sumtype v0.5.0
github.com/gtsteffaniak/go-ffmpeg v1.1.0
github.com/gtsteffaniak/go-ffmpeg v1.2.0
)
+32 -32
View File
@@ -47,10 +47,10 @@ github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 h1:DilThiXje
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349/go.mod h1:4GC5sXji84i/p+irqghpPFZBF8tRN/Q7+700G0/DLe8=
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
github.com/ebitengine/purego v0.11.0 h1:jhp/D+Nyv7UUW8HAcmcjt2N2rYrYi9m3SL21k0Ua/NI=
github.com/ebitengine/purego v0.11.0/go.mod h1:DCHPP08djqhNSoTfImcnHYQRZmd0qhakvrozqaEYhGQ=
github.com/evanoberholster/imagemeta v1.0.0 h1:FlSihlsjkWIoS83wmgPOP3CbAxlRpxM8jQGZgLu+MWQ=
github.com/evanoberholster/imagemeta v1.0.0/go.mod h1:2ITZJjD1ygzWIhepvn8coRvpaFcnDNk3E6hx6v7z8DU=
github.com/ebitengine/purego v0.11.1 h1:2zpWRSQNVKN4eKsKO9eM1ILDgWfYMY9GwqRmK6XeQ/0=
github.com/ebitengine/purego v0.11.1/go.mod h1:DCHPP08djqhNSoTfImcnHYQRZmd0qhakvrozqaEYhGQ=
github.com/evanoberholster/imagemeta v1.1.0 h1:0iBtgVxLORZTZAlKxRcNodBmjMfHCSFwUh8Asmi83oI=
github.com/evanoberholster/imagemeta v1.1.0/go.mod h1:nmf6le+qSyoRAXgp2psPbtfkxcEgf/AZGgai9pnN8d0=
github.com/fxamacker/cbor/v2 v2.9.4 h1:xwjVlxEMR3S605oUlgBjKLTTeGFciYPGYCtF/35LKGo=
github.com/fxamacker/cbor/v2 v2.9.4/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
@@ -69,10 +69,10 @@ github.com/go-jose/go-jose/v4 v4.1.5 h1:RjgjO2LOtWOJKUC5wpwY9LR3B3vwVAz6JS2YHfYU
github.com/go-jose/go-jose/v4 v4.1.5/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-ldap/ldap/v3 v3.4.14 h1:D6PYdEgsaVzsXyr6w/yDC06Ria4uUhWm+Rb+er8lfAs=
github.com/go-ldap/ldap/v3 v3.4.14/go.mod h1:S4eJUMUNjDkE0ZJtIZdybwyb03sGGLW6gxXT1Hs8VKA=
github.com/go-openapi/jsonpointer v1.0.1 h1:2KxywRmNwJkT/FMBa3iRNHEaAxSJvjqoufQZy3au1Mg=
github.com/go-openapi/jsonpointer v1.0.1/go.mod h1:wI7ZYsFmbIi9nBXOZqgDaS/bqOchRGZjqxFli7FBYxY=
github.com/go-openapi/jsonreference v1.0.2 h1:oS4et8FOf3p3UQxEo4Xt0esijmBUM+F259Xl72OSZsc=
github.com/go-openapi/jsonreference v1.0.2/go.mod h1:TbUNSOo+fcorZjFaNoiSDSoaNnnZtqJtLGR1PuvE/Cs=
github.com/go-openapi/jsonpointer v1.0.2 h1:NbW7xT2b+l7x+JefiPGDsZWsrLk1ogrz4AHK1TTzMt8=
github.com/go-openapi/jsonpointer v1.0.2/go.mod h1:N0rI4MbNKD7xAFTiHtB+5Nhrg2j/wwYg7m7Fmb4/jk8=
github.com/go-openapi/jsonreference v1.0.3 h1:5rDbq+JsjCv6/nUDDJt83nLZoADI4w03uvHXer+4aYQ=
github.com/go-openapi/jsonreference v1.0.3/go.mod h1:OJSlMXHysgH272D9z2tiy2MGe5A9vQLHaCPb6KbYdE4=
github.com/go-openapi/spec v1.0.1 h1:lj2vdGpNDcVgwRc6qXdw6qt/KQpCtSa9tnUH6vpDPDk=
github.com/go-openapi/spec v1.0.1/go.mod h1:M//GWQGtDUAjnP37gE6fInLgaczB+FatoipV3H1fYw8=
github.com/go-openapi/swag v0.19.15 h1:D2NRCBzS9/pEY3gP9Nl8aDqGUcPFrwG2p+CNFrLyrCM=
@@ -94,20 +94,20 @@ github.com/go-openapi/swag/yamlutils v0.29.2 h1:IFKFFeDnuIwzfsuWRQU+rI8iL3g4XyAY
github.com/go-openapi/swag/yamlutils v0.29.2/go.mod h1:7MGqtcrK73sxQ4ceiyIf8qiXS/s09JLMdR5esK5euYQ=
github.com/go-openapi/testify/enable/yaml/v2 v2.7.0 h1:wPW6YRgx3+SID1yUy/Xwa17L8kFEaEKod2VRbJDZNUs=
github.com/go-openapi/testify/enable/yaml/v2 v2.7.0/go.mod h1:mI1M88etYbc3PhgHsWQK2kwvNwW5aGFqMPbmib+SGIs=
github.com/go-openapi/testify/v2 v2.7.0 h1:bycOreEj6wfBvijg3YFogZ/sFjTCDmQnwSodSzHa3X8=
github.com/go-openapi/testify/v2 v2.7.0/go.mod h1:SgsVHtfooshd0tublTtJ50FPKhujf47YRqauXXOUxfw=
github.com/go-openapi/testify/v2 v2.8.0 h1:19QDx5b57p8KjO2E4sgs/woL4Akp+FGLRlVm2xn0Rho=
github.com/go-openapi/testify/v2 v2.8.0/go.mod h1:4gUN8jC+RqOE2qjW/vDmfyTvl5BCht4qL7oca0odrak=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.30.4 h1:9Rcod2ZPO6mOEG6b4GqyoHE/H6//Ze0RuhOo1hT1x0w=
github.com/go-playground/validator/v10 v10.30.4/go.mod h1:numpT+RPLE91R9oYWMY/R9zRgJBewr3IXHko4OISPpk=
github.com/go-playground/locales v0.14.2 h1:d8UmcrM6Nip0hfGZKLGpAvZH37XB4TS0xzK9B56YNCY=
github.com/go-playground/locales v0.14.2/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.2 h1:LCsMLC9RzmbUMNUPVYD15dmcjwYAJhmX8mPZRW4rAVU=
github.com/go-playground/universal-translator v0.18.2/go.mod h1:67VZIMp5lQpDWlnStOct22q1bkdJGJqHghbOtmkawxk=
github.com/go-playground/validator/v10 v10.30.5 h1:YyCXvVShZbs2Sm3Mb53eNOlhRXctSOzW5QJAouCTZL4=
github.com/go-playground/validator/v10 v10.30.5/go.mod h1:wEqiaov48pXX1kjhc3Da8y0M0Dtg/BK7gurFBLgwFrQ=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/go-webauthn/webauthn v0.18.1 h1:KaQw6M+ODLvxHwddyeo6zFhhmicfLup/BClhigB6A+0=
github.com/go-webauthn/webauthn v0.18.1/go.mod h1:s4rZTQnKHWxIh6G3yEGqlxvtiLceA1jigll8FpqSgQ8=
github.com/go-webauthn/webauthn v0.18.2 h1:0BeftmEHU7i3Dv0VFwBtidy/ba37Vcdjvqst9EYu8Sk=
github.com/go-webauthn/webauthn v0.18.2/go.mod h1:hEXaOuLxvZ3zG9miZe3ehlyeVso9AtklXG+kTn36k+A=
github.com/go-webauthn/x v0.3.1 h1:1ff37z3XfmTTomkhlURgGizLIDyOvPgTt2t9nlzKLRo=
github.com/go-webauthn/x v0.3.1/go.mod h1:ZInxAynYXfBPvvm5gzKZ7geBlL23K71xASMgohHl/Rg=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
@@ -119,8 +119,8 @@ github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArs
github.com/golang/geo v0.0.0-20190916061304-5b978397cfec/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI=
github.com/golang/geo v0.0.0-20200319012246-673a6f80352d/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI=
github.com/golang/geo v0.0.0-20210211234256-740aa86cb551/go.mod h1:QZ0nwyI2jOfgRAoBvP+ab5aRr7c9x7lhGEJrKvBwjWI=
github.com/golang/geo v0.0.0-20260818125358-b200a1149890 h1:m+G0ip1+N4CF0ex34SeojAon6htIIBwvzsyXNx1fGWg=
github.com/golang/geo v0.0.0-20260818125358-b200a1149890/go.mod h1:Mymr9kRGDc64JPr03TSZmuIBODZ3KyswLzm1xL0HFA8=
github.com/golang/geo v0.0.0-20260928092222-7d12f68cfadb h1:z//sB+BhCYNvUrk19E8aWqNx+BBom8p/uD/nh95N/5U=
github.com/golang/geo v0.0.0-20260928092222-7d12f68cfadb/go.mod h1:Mymr9kRGDc64JPr03TSZmuIBODZ3KyswLzm1xL0HFA8=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
@@ -143,8 +143,8 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gtsteffaniak/go-cache v1.1.0 h1:ni/2cJpCxpGmaMZd92iTDIGOy+yUcMcxQuW7bSKhkCE=
github.com/gtsteffaniak/go-cache v1.1.0/go.mod h1:rHUZvf4huhMX/bnNK4/oQ3sNAE2fYC2jFpAWZiBPWac=
github.com/gtsteffaniak/go-ffmpeg v1.0.0 h1:dUWvlZmWnkiujDtIMLGwrpfSAOnM+IOtk93HXF6HR0U=
github.com/gtsteffaniak/go-ffmpeg v1.0.0/go.mod h1:FefhoD+KZzHWqBaUpntcKp6lXtUzdFW00F74axnENUQ=
github.com/gtsteffaniak/go-ffmpeg v1.2.0 h1:WSFfRL5xVUpEXBNI4D3f2Uz8yxbc3OEqTtjp4Z9yPUo=
github.com/gtsteffaniak/go-ffmpeg v1.2.0/go.mod h1:FefhoD+KZzHWqBaUpntcKp6lXtUzdFW00F74axnENUQ=
github.com/gtsteffaniak/go-logger v1.1.0 h1:baFds+iibuLLNpNLwygz32bDqdsunI6dGWcdxwMIVKI=
github.com/gtsteffaniak/go-logger v1.1.0/go.mod h1:rapapTju4D7bwt6h4JOdZNUXDjnp0WkTBd6TN5yWglI=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
@@ -187,8 +187,8 @@ github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsRe
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-sqlite3 v1.14.52 h1:wVbm2Qnf4OXkqhBTSPuCRZDRnxfbVrrmiCEroVdog8U=
github.com/mattn/go-sqlite3 v1.14.52/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/ncruces/go-strftime v1.1.0 h1:vtPmPWMkb539RGV9WpWT9EwJGvrOCKvMNI4obWL47Hk=
github.com/ncruces/go-strftime v1.1.0/go.mod h1:yVaq7iwCN09vHRrKBrrflXT0M9tTMG2KmZ9Fj7xeG/g=
github.com/philhofer/fwd v1.2.0 h1:e6DnBTl7vGY+Gz322/ASL4Gyp1FspeMvx1RNDoToZuM=
github.com/philhofer/fwd v1.2.0/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
@@ -217,8 +217,8 @@ github.com/swaggo/http-swagger v1.3.4 h1:q7t/XLx0n15H1Q9/tk3Y9L4n210XzJF5WtnDX64
github.com/swaggo/http-swagger v1.3.4/go.mod h1:9dAh0unqMBAlbp1uE2Uc2mQTxNMU/ha4UbucIg1MFkQ=
github.com/swaggo/swag v1.16.6 h1:qBNcx53ZaX+M5dxVyTrgQ0PJ/ACK+NzhwcbieTt+9yI=
github.com/swaggo/swag v1.16.6/go.mod h1:ngP2etMK5a0P3QBizic5MEwpRmluJZPHjXcMoj4Xesg=
github.com/tinylib/msgp v1.6.4 h1:mOwYbyYDLPj35mkA2BjjYejgJk9BuHxDdvRnb6v2ZcQ=
github.com/tinylib/msgp v1.6.4/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
github.com/tinylib/msgp v1.6.5 h1:iAH6XTP7BpzErjBZlujJQXxA+GmRi2YWs6M4KdLtNbE=
github.com/tinylib/msgp v1.6.5/go.mod h1:RSp0LW9oSxFut3KzESt5Voq4GVWyS+PSulT77roAqEA=
github.com/vmihailenco/msgpack v4.0.4+incompatible h1:dSLoQfGFAo3F6OoNhwUmLwVgaUXK79GlxNBwueZn0xI=
github.com/vmihailenco/msgpack v4.0.4+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
@@ -313,8 +313,8 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.29.7 h1:q+NXGJ0bK3b4TXFYQQVr9pYETGnmwFWkrUzJnMya/Tg=
modernc.org/cc/v4 v4.29.7/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.35.2 h1:JPAIttQRHdY7aRdr04+iTW7Sx+6OSZcmKJ0OZl/tNaA=
modernc.org/ccgo/v4 v4.35.2/go.mod h1:9sddcpn4NuDAFGtBPa2Dk3NHfnQfcoKveCC5crwWp8I=
modernc.org/ccgo/v4 v4.36.1 h1:ZNIUZAryN0UgnJwtyxrdEzcFc3yD4Cu4AzjfPXsLsIE=
modernc.org/ccgo/v4 v4.36.1/go.mod h1:rrtGc2QkS239nYb/mQNuBMyjq3/y3ZXWbBjPoV3wqzA=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
@@ -323,8 +323,8 @@ modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.76.0 h1:eaJHMv2zn5oXT6IPXPwxAMVpzmQzSDsCdKcNl1ZpaRg=
modernc.org/libc v1.76.0/go.mod h1:2h0dedmVSE8qH2DrxzYDXbQaxLMl0XNg8Z7/HJRdk2M=
modernc.org/libc v1.77.1 h1:Ct8j47QtiZ1Enj2DtFXQtUqrPCAjdCmPjtCuvrYQ0Hs=
modernc.org/libc v1.77.1/go.mod h1:87/pZ4L6nD1zqW4nItuS12YO7hN1igAah34xjnQo/W0=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
@@ -333,8 +333,8 @@ modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.59.0 h1:X1es1GpqBlS/5T+vbM4HLUdaa8OtQx468DF2vrx+38A=
modernc.org/sqlite v1.59.0/go.mod h1:+paeT2A3iPRHkQDwG7oA6Tk0zQd5woMEI8q7orfry8k=
modernc.org/sqlite v1.60.1 h1:/blz53O951KWFOso4QQvEs/Fq6cDBKLtMVrYNSeJVKw=
modernc.org/sqlite v1.60.1/go.mod h1:1dIoEagfDE72QytD5scH1lxARtaUgKgHC/NuApA27r0=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
+1 -1
View File
@@ -53,7 +53,7 @@ func TestUserUpdateChangesFiltersUnchangedFields(t *testing.T) {
which := []string{
"preview", "stickySidebar", "darkMode", "locale", "singleClick", "sorting",
"showHidden", "dateFormat", "themeColor", "quickDownload", "fileLoading",
"sidebarLinks", "deleteAfterArchive", "preferEditorForMarkdown", "showFirstLogin",
"sidebarLinks", "deleteAfterArchive", "preferEditorForMarkdown", "showFirstLogin", "requirePasswordChange",
}
changes := UserUpdateChanges(before, &after, which, false)
+74
View File
@@ -7,6 +7,7 @@ import (
"time"
activitydb "github.com/gtsteffaniak/filebrowser/backend/internal/database/activity"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/quota"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
)
@@ -287,6 +288,79 @@ func RecordAccessDelete(r *http.Request, actor *Actor, source, path string, chan
})
}
func RecordQuotaCreate(r *http.Request, actor *Actor, source, path string, changes []activitydb.FieldChange) {
RecordQuotaMutation(r, actor, activitydb.EventQuotaCreate, source, path, changes)
}
func RecordQuotaUpdate(r *http.Request, actor *Actor, source, path string, changes []activitydb.FieldChange) {
RecordQuotaMutation(r, actor, activitydb.EventQuotaUpdate, source, path, changes)
}
func RecordQuotaDelete(r *http.Request, actor *Actor, source, path string, changes []activitydb.FieldChange) {
RecordQuotaMutation(r, actor, activitydb.EventQuotaDelete, source, path, changes)
}
func RecordQuotaMutation(r *http.Request, actor *Actor, eventType activitydb.EventType, source, path string, changes []activitydb.FieldChange) {
RecordUser(r, actor, activitydb.Entry{
EventType: eventType,
Source: source,
Path: path,
Details: activitydb.Details{
Source: source,
Path: path,
Changes: changes,
},
})
}
func quotaFolderChanges(q quota.FolderQuota) []activitydb.FieldChange {
changes := []activitydb.FieldChange{
{Field: "limitBytes", To: strconv.FormatInt(q.LimitBytes, 10)},
{Field: "meter", To: q.Meter},
}
if q.UserID > 0 {
changes = append(changes, activitydb.FieldChange{
Field: "userId",
To: strconv.FormatUint(q.UserID, 10),
})
}
return changes
}
func QuotaFolderCreateChanges(q quota.FolderQuota) []activitydb.FieldChange {
return quotaFolderChanges(q)
}
func QuotaFolderDeleteChanges(q quota.FolderQuota) []activitydb.FieldChange {
return quotaFolderChanges(q)
}
func QuotaFolderUpdateChanges(before, after quota.FolderQuota) []activitydb.FieldChange {
var changes []activitydb.FieldChange
if before.LimitBytes != after.LimitBytes {
changes = append(changes, activitydb.FieldChange{
Field: "limitBytes",
From: strconv.FormatInt(before.LimitBytes, 10),
To: strconv.FormatInt(after.LimitBytes, 10),
})
}
if before.Meter != after.Meter {
changes = append(changes, activitydb.FieldChange{
Field: "meter",
From: before.Meter,
To: after.Meter,
})
}
if before.UserID != after.UserID {
changes = append(changes, activitydb.FieldChange{
Field: "userId",
From: strconv.FormatUint(before.UserID, 10),
To: strconv.FormatUint(after.UserID, 10),
})
}
return changes
}
func AccessRuleCreateChanges(allow bool, ruleCategory, value string) []activitydb.FieldChange {
ruleType := "deny"
if allow {
+87 -96
View File
@@ -1,13 +1,11 @@
package files
import (
"bufio"
"context"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
@@ -25,8 +23,6 @@ import (
"github.com/gtsteffaniak/go-logger/logger"
)
var reDuration = regexp.MustCompile(`^\[(?:(\d{1,2}):)?(\d{1,2}):(\d{1,2})\.(\d+)\](.*)`)
// CheckPermissionsFunc allows tests to override CheckPermissions behavior
var CheckPermissionsFunc = func(opts utils.FileOptions, user *users.User) (string, string, error) {
return checkPermissionsImpl(opts, user, svc())
@@ -50,7 +46,7 @@ func addMetadataToChildren(response *iteminfo.ExtendedFileInfo, opts utils.FileO
wg.Go(func() {
fullPath := response.RealPath + "/" + fileItem.Name
if isAudio {
if err := extractAudioMetadata(context.Background(), fileItem, fullPath, opts.AlbumArt, opts.Metadata, false, sharedFFmpegService); err != nil {
if err := extractAudioMetadata(context.Background(), fileItem, fullPath, opts.AlbumArt, opts.Metadata, sharedFFmpegService); err != nil {
logger.Debugf("failed to extract metadata for file: %s, error: %v", fileItem.Name, err)
}
} else {
@@ -393,7 +389,7 @@ func processContent(info *iteminfo.ExtendedFileInfo, idx *indexing.Index, opts u
extItem := &iteminfo.ExtendedItemInfo{
ItemInfo: info.ItemInfo,
}
err := extractAudioMetadata(context.Background(), extItem, info.RealPath, opts.AlbumArt || opts.Content, opts.Metadata || opts.Content, false, nil)
err := extractAudioMetadata(context.Background(), extItem, info.RealPath, opts.AlbumArt || opts.Content, opts.Metadata || opts.Content, nil)
if err != nil {
logger.Debugf("failed to extract audio metadata for file: "+info.RealPath, info.Name, err)
} else {
@@ -431,57 +427,10 @@ func generateOfficeId(realPath string) string {
return key
}
// Parses raw LRC text into a slice of Lyric.
// Lines with [hh:mm:ss.xx] (timestamps) will be parsed to ms for synced lyrics
// The ones without timestamp are kept as unsynced (timestamp 0).
func parseLRC(raw string) []iteminfo.Lyric {
var lines []iteminfo.Lyric
scanner := bufio.NewScanner(strings.NewReader(raw))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
matches := reDuration.FindStringSubmatch(line)
if len(matches) == 6 {
// Hours default to 0 (if not present in the timestamps)
h := 0
if matches[1] != "" {
h, _ = strconv.Atoi(matches[1])
}
min, _ := strconv.Atoi(matches[2])
sec, _ := strconv.Atoi(matches[3])
msStr := matches[4]
// Normalise to 3 digits, some songs have more than 3 digits in the milliseconds
switch {
case len(msStr) < 3:
msStr += strings.Repeat("0", 3-len(msStr))
case len(msStr) > 3:
msStr = msStr[:3]
}
ms, _ := strconv.Atoi(msStr)
timestamp := int64((h*3600+min*60+sec)*1000 + ms)
text := strings.TrimSpace(matches[5])
lines = append(lines, iteminfo.Lyric{
Text: text,
Timestamp: timestamp,
})
} else {
// Line without a valid timestamp – treat as unsynced (timestamp 0)
lines = append(lines, iteminfo.Lyric{
Text: line,
Timestamp: 0,
})
}
}
return lines
}
// extractAudioMetadata extracts metadata from an audio file using dhowden/tag
// and optionally extracts duration using the ffmpeg service with concurrency control
// If ffmpegService is nil, a new service will be created (for backward compatibility)
func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPath string, getArt bool, getDuration bool, getLyrics bool, ffmpegService *ffmpeg.FFmpegService) error {
func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPath string, getArt bool, getDuration bool, ffmpegService *ffmpeg.FFmpegService) error {
file, err := os.Open(realPath)
if err != nil {
return err
@@ -518,28 +467,7 @@ func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
item.Metadata.Track = track
// Check if lyrics are available without extract them
if rawLyrics := m.Lyrics(); rawLyrics != "" {
item.Metadata.HasLyrics = true
} else {
// Check for sidecar .lrc file
dir := filepath.Dir(realPath)
base := filepath.Base(realPath)
ext := filepath.Ext(base)
nameWithoutExt := strings.TrimSuffix(base, ext)
lrcPath := filepath.Join(dir, nameWithoutExt+".lrc")
if _, err := os.Stat(lrcPath); err == nil {
item.Metadata.HasLyrics = true
}
}
if getLyrics {
lyrics, err := ExtractLyrics(realPath)
if err != nil {
logger.Debugf("failed to extract lyrics for %s: %v", realPath, err)
} else if len(lyrics) > 0 {
item.Metadata.Lyrics = lyrics
}
}
item.Metadata.HasLyrics = hasLyricsSidecar(realPath) || m.Lyrics() != ""
// Extract duration ONLY if explicitly requested using the ffmpeg VideoService
// This respects concurrency limits and gracefully handles missing ffmpeg
@@ -572,35 +500,91 @@ func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
return nil
}
// Returns lyrics from an audio file (from embedded tags or from a .lrc file with the same name).
func ExtractLyrics(realPath string) ([]iteminfo.Lyric, error) {
var lyricsSidecarExts = []string{".elrc", ".lrc", ".srt", ".vtt"}
func lyricsSidecar(dir, nameWithoutExt, ext string) (path string, ok bool) {
sidecarPath := filepath.Join(dir, nameWithoutExt+ext)
info, err := os.Lstat(sidecarPath)
if err != nil {
return "", false
}
if !info.Mode().IsRegular() {
return "", false
}
if info.Size() > 15*1024*1024 { // 15MB
return "", false
}
return sidecarPath, true
}
func findLyricsSidecar(realPath string) (data []byte, ext string, ok bool) {
dir := filepath.Dir(realPath)
base := filepath.Base(realPath)
nameWithoutExt := strings.TrimSuffix(base, filepath.Ext(base))
for _, ext = range lyricsSidecarExts {
sidecarPath, candOk := lyricsSidecar(dir, nameWithoutExt, ext)
if !candOk {
continue
}
data, err := os.ReadFile(sidecarPath)
if err != nil {
continue
}
if strings.TrimSpace(string(data)) == "" {
continue
}
return data, strings.TrimPrefix(ext, "."), true
}
return nil, "", false
}
func hasLyricsSidecar(realPath string) bool {
dir := filepath.Dir(realPath)
base := filepath.Base(realPath)
nameWithoutExt := strings.TrimSuffix(base, filepath.Ext(base))
for _, ext := range lyricsSidecarExts {
sidecarPath, ok := lyricsSidecar(dir, nameWithoutExt, ext)
if !ok {
continue
}
f, err := os.Open(sidecarPath)
if err != nil {
continue
}
buf := make([]byte, 4096)
for {
n, readErr := f.Read(buf)
if n > 0 && strings.TrimSpace(string(buf[:n])) != "" {
f.Close()
return true
}
if readErr != nil {
break
}
}
f.Close()
}
return false
}
// Returns raw lyrics from an audio file (from embedded tags or from sidecar files with the same name).
func ExtractLyrics(realPath string) (string, string, error) {
if data, ext, ok := findLyricsSidecar(realPath); ok {
return string(data), ext, nil
}
file, err := os.Open(realPath)
if err != nil {
return nil, err
return "", "", err
}
defer file.Close()
m, err := tag.ReadFrom(file)
if err != nil {
return nil, err
return "", "", err
}
var lyrics []iteminfo.Lyric
if raw := m.Lyrics(); raw != "" {
lyrics = parseLRC(raw)
return raw, "embedded", nil
}
if len(lyrics) == 0 {
// Check for sidecar .lrc file
dir := filepath.Dir(realPath)
base := filepath.Base(realPath)
ext := filepath.Ext(base)
nameWithoutExt := strings.TrimSuffix(base, ext)
lrcPath := filepath.Join(dir, nameWithoutExt+".lrc")
if data, err := os.ReadFile(lrcPath); err == nil {
lyrics = parseLRC(string(data))
}
}
return lyrics, nil
return "", "", nil
}
// extractVideoMetadata extracts duration and codec info from video files using ffprobe.
@@ -961,7 +945,14 @@ func WriteDirectory(opts utils.FileOptions) error {
return nil
}
// WriteFileFunc is a variable that can be mocked in tests
var WriteFileFunc = writeFile
func WriteFile(source, path string, in io.Reader) error {
return WriteFileFunc(source, path, in)
}
func writeFile(source, path string, in io.Reader) error {
idx := indexing.GetIndex(source)
if idx == nil {
return fmt.Errorf("could not get index: %v ", source)
@@ -447,6 +447,7 @@ var types = map[string]string{
".vtt": "text/subtitle-vtt",
".ssa": "text/subtitle-ssa",
".lrc": "text/lyrics",
".elrc": "text/lyrics",
".sbv": "text/plain",
".ssi": "text/x-server-parsed-html",
".ssm": "application/streamingmedia",
@@ -21,11 +21,13 @@ func (d dirFS) Open(name string) (fs.File, error) {
// InitAssetFS initializes the asset filesystem for the application
// This should be called once during startup before http or preview services start
func InitAssetFS(embeddedAssets fs.FS, useEmbedded bool) {
func InitAssetFS(embeddedAssets fs.FS, useEmbedded bool, devMode bool) {
if useEmbedded {
assetFs = embeddedAssets
} else if devMode {
// Dev mode: serve frontend source assets (templates, fonts, icons)
assetFs = dirFS{Dir: http.Dir("../frontend/public")}
} else {
// Dev mode: Serve files from internal/web/dist directory
assetFs = dirFS{Dir: http.Dir("internal/web/dist")}
}
}
+55 -27
View File
@@ -56,48 +56,62 @@ func unixModeToFileMode(u uint32) os.FileMode {
}
// MoveFile moves a file from src to dst.
// By default, the rename system call is used. If src and dst point to different volumes,
// the file copy is used as a fallback.
// By default, the rename system call is used. If src and dst are on different volumes,
// CopyFile is used as a fallback and the source is removed synchronously on success.
func MoveFile(src, dst string) error {
err := os.Rename(src, dst)
if err == nil {
return nil
}
// fallback
err = CopyFile(src, dst)
if err != nil {
logger.Errorf("CopyFile failed %v", err)
return err
}
go func() {
err = os.RemoveAll(src)
if err != nil {
logger.Errorf("os.Remove failed %v", err)
if err := os.Rename(src, dst); err != nil {
if !isCrossDeviceRenameError(err) {
return err
}
}()
if err := CopyFile(src, dst); err != nil {
return err
}
return os.RemoveAll(src)
}
return nil
}
func preserveFileMode(path string, mode os.FileMode) error {
return os.Chmod(path, mode.Perm())
}
// CopyFile copies a file or directory from source to dest and returns an error if any.
func CopyFile(source, dest string) error {
// Check if the source exists and whether it's a file or directory.
info, err := os.Stat(source)
info, err := os.Lstat(source)
if err != nil {
return err
}
if info.Mode()&os.ModeSymlink != 0 {
return copySymlink(source, dest)
}
if info.IsDir() {
// If the source is a directory, copy it recursively.
return copyDirectory(source, dest)
}
// If the source is a file, copy the file.
return copySingleFile(source, dest)
}
func copySymlink(source, dest string) error {
target, err := os.Readlink(source)
if err != nil {
return err
}
destDir := filepath.Dir(dest)
if err = os.MkdirAll(destDir, EffectiveDirPerm()); err != nil {
return err
}
tmpDir, err := os.MkdirTemp(destDir, ".fb-link-*")
if err != nil {
return err
}
defer func() { _ = os.RemoveAll(tmpDir) }()
tmpPath := filepath.Join(tmpDir, "link")
if err := os.Symlink(target, tmpPath); err != nil {
return err
}
return os.Rename(tmpPath, dest)
}
// copySingleFile handles copying a single file.
func copySingleFile(source, dest string) error {
// Get source file info to preserve permissions
@@ -161,7 +175,7 @@ func copyFileContents(dst io.Writer, src io.Reader) error {
// copyDirectory handles copying directories recursively.
func copyDirectory(source, dest string) error {
srcInfo, err := os.Stat(source)
srcInfo, err := os.Lstat(source)
if err != nil {
return err
}
@@ -170,6 +184,9 @@ func copyDirectory(source, dest string) error {
if err != nil {
return err
}
if err = preserveFileMode(dest, srcInfo.Mode()); err != nil {
logger.Debugf("Could not set directory permissions for %s: %v", dest, err)
}
// Read the contents of the source directory.
entries, err := os.ReadDir(source)
@@ -182,6 +199,17 @@ func copyDirectory(source, dest string) error {
srcPath := filepath.Join(source, entry.Name())
destPath := filepath.Join(dest, entry.Name())
if entry.Type()&os.ModeSymlink != 0 {
linkTarget := ""
linkTarget, err = os.Readlink(srcPath)
if err != nil {
return err
}
if err = os.Symlink(linkTarget, destPath); err != nil {
return err
}
continue
}
if entry.IsDir() {
// Recursively copy subdirectories.
err = copyDirectory(srcPath, destPath)
@@ -139,6 +139,223 @@ func TestCopyFilePreservesModTime(t *testing.T) {
}
}
// moveViaCrossDeviceFallback mirrors MoveFile's EXDEV fallback (CopyFile + RemoveAll).
func moveViaCrossDeviceFallback(t *testing.T, src, dst string) {
t.Helper()
if err := CopyFile(src, dst); err != nil {
t.Fatal(err)
}
if err := os.RemoveAll(src); err != nil {
t.Fatal(err)
}
}
func TestMoveFileDoesNotFallbackOnNonCrossDeviceError(t *testing.T) {
srcDir := t.TempDir()
src := filepath.Join(srcDir, "file.txt")
if err := os.WriteFile(src, []byte("data"), 0o644); err != nil {
t.Fatal(err)
}
dst := filepath.Join(srcDir, "existing-dir")
if err := os.Mkdir(dst, 0o755); err != nil {
t.Fatal(err)
}
err := MoveFile(src, dst)
if err == nil {
t.Fatal("expected rename error")
}
if _, statErr := os.Stat(src); statErr != nil {
t.Fatalf("source file should remain after rename failure: %v", statErr)
}
entries, err := os.ReadDir(dst)
if err != nil {
t.Fatal(err)
}
if len(entries) != 0 {
t.Fatalf("destination directory should be unchanged, got %d entries", len(entries))
}
}
func TestMoveFileCrossDevicePreservesFilePermissions(t *testing.T) {
srcDir := t.TempDir()
dstDir := t.TempDir()
src := filepath.Join(srcDir, "secret.txt")
if err := os.WriteFile(src, []byte("data"), 0o640); err != nil {
t.Fatal(err)
}
dst := filepath.Join(dstDir, "secret.txt")
moveViaCrossDeviceFallback(t, src, dst)
got, err := os.Stat(dst)
if err != nil {
t.Fatal(err)
}
if got.Mode().Perm() != 0o640 {
t.Fatalf("dst perm = %#o, want 0640", got.Mode().Perm())
}
if _, err := os.Stat(src); !os.IsNotExist(err) {
t.Fatalf("src should be removed, stat err = %v", err)
}
}
func TestMoveFileCrossDeviceMovesDirectory(t *testing.T) {
srcDir := t.TempDir()
dstParent := t.TempDir()
nested := filepath.Join(srcDir, "nested.txt")
if err := os.WriteFile(nested, []byte("nested"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Chmod(srcDir, 0o755); err != nil {
t.Fatal(err)
}
dst := filepath.Join(dstParent, "moved-dir")
moveViaCrossDeviceFallback(t, srcDir, dst)
movedNested := filepath.Join(dst, "nested.txt")
if _, err := os.Stat(movedNested); err != nil {
t.Fatalf("expected nested file at destination: %v", err)
}
got, err := os.Stat(dst)
if err != nil {
t.Fatal(err)
}
if got.Mode().Perm() != 0o755 {
t.Fatalf("dst dir perm = %#o, want 0755", got.Mode().Perm())
}
if _, err := os.Stat(srcDir); !os.IsNotExist(err) {
t.Fatalf("source directory should be removed, stat err = %v", err)
}
}
func TestMoveFileCrossDevicePreservesSymlinkToFile(t *testing.T) {
targetDir := t.TempDir()
target := filepath.Join(targetDir, "target.txt")
if err := os.WriteFile(target, []byte("data"), 0o644); err != nil {
t.Fatal(err)
}
srcDir := t.TempDir()
dstDir := t.TempDir()
link := filepath.Join(srcDir, "link")
if err := os.Symlink(target, link); err != nil {
t.Fatal(err)
}
dst := filepath.Join(dstDir, "link")
moveViaCrossDeviceFallback(t, link, dst)
gotTarget, err := os.Readlink(dst)
if err != nil {
t.Fatal(err)
}
if gotTarget != target {
t.Fatalf("link target = %q, want %q", gotTarget, target)
}
info, err := os.Lstat(dst)
if err != nil {
t.Fatal(err)
}
if info.Mode()&os.ModeSymlink == 0 {
t.Fatal("expected destination to remain a symlink")
}
if _, err := os.Stat(link); !os.IsNotExist(err) {
t.Fatalf("source symlink should be removed, stat err = %v", err)
}
}
func TestMoveFileCrossDevicePreservesSymlinkToDirectory(t *testing.T) {
targetDir := t.TempDir()
if err := os.Mkdir(filepath.Join(targetDir, "inside"), 0o755); err != nil {
t.Fatal(err)
}
srcDir := t.TempDir()
dstDir := t.TempDir()
link := filepath.Join(srcDir, "dirlink")
if err := os.Symlink(targetDir, link); err != nil {
t.Fatal(err)
}
dst := filepath.Join(dstDir, "dirlink")
moveViaCrossDeviceFallback(t, link, dst)
gotTarget, err := os.Readlink(dst)
if err != nil {
t.Fatal(err)
}
if gotTarget != targetDir {
t.Fatalf("link target = %q, want %q", gotTarget, targetDir)
}
info, err := os.Lstat(dst)
if err != nil {
t.Fatal(err)
}
if info.Mode()&os.ModeSymlink == 0 {
t.Fatal("expected destination to remain a symlink")
}
}
func TestMoveFileCrossDeviceMovesDirectoryWithNestedSymlink(t *testing.T) {
target := filepath.Join(t.TempDir(), "target.txt")
if err := os.WriteFile(target, []byte("linked"), 0o644); err != nil {
t.Fatal(err)
}
srcDir := t.TempDir()
if err := os.Symlink(target, filepath.Join(srcDir, "nested-link")); err != nil {
t.Fatal(err)
}
dstParent := t.TempDir()
dst := filepath.Join(dstParent, "moved-dir")
moveViaCrossDeviceFallback(t, srcDir, dst)
movedLink := filepath.Join(dst, "nested-link")
gotTarget, err := os.Readlink(movedLink)
if err != nil {
t.Fatal(err)
}
if gotTarget != target {
t.Fatalf("nested link target = %q, want %q", gotTarget, target)
}
info, err := os.Lstat(movedLink)
if err != nil {
t.Fatal(err)
}
if info.Mode()&os.ModeSymlink == 0 {
t.Fatal("expected nested entry to remain a symlink")
}
}
func TestCopySymlinkReplacesExistingDestination(t *testing.T) {
target := filepath.Join(t.TempDir(), "target.txt")
if err := os.WriteFile(target, []byte("data"), 0o644); err != nil {
t.Fatal(err)
}
workDir := t.TempDir()
src := filepath.Join(workDir, "link")
if err := os.Symlink(target, src); err != nil {
t.Fatal(err)
}
dst := filepath.Join(workDir, "existing")
if err := os.WriteFile(dst, []byte("old"), 0o644); err != nil {
t.Fatal(err)
}
if err := copySymlink(src, dst); err != nil {
t.Fatal(err)
}
gotTarget, err := os.Readlink(dst)
if err != nil {
t.Fatal(err)
}
if gotTarget != target {
t.Fatalf("link target = %q, want %q", gotTarget, target)
}
info, err := os.Lstat(dst)
if err != nil {
t.Fatal(err)
}
if info.Mode()&os.ModeSymlink == 0 {
t.Fatal("expected destination to be a symlink")
}
}
func TestCopyFileContentsFallsBackAfterCopyFileRangeBadFileDescriptor(t *testing.T) {
const content = "complete file contents"
src := struct{ io.Reader }{bytes.NewBufferString(content)}
@@ -0,0 +1,30 @@
package fileutils
// PartitionUsage is a capacity view of a source path: total and used bytes.
type PartitionUsage struct {
Total uint64 `json:"total"`
Used uint64 `json:"used"`
}
// singlePathPartitionUsage statfs's a single path: the filesystem covering it.
func singlePathPartitionUsage(path string) (PartitionUsage, error) {
total, err := GetPartitionSize(path)
if err != nil {
return PartitionUsage{}, err
}
used, err := GetPartitionUsed(path)
if err != nil {
return PartitionUsage{}, err
}
return PartitionUsage{Total: total, Used: used}, nil
}
// GetAggregatedPartitionUsage returns total and used bytes summed across distinct
// capacity groups mounted at or under root. See GetPartitionUsageVariants.
func GetAggregatedPartitionUsage(root string) (total, used uint64, err error) {
aggregate, _, err := GetPartitionUsageVariants(root)
if err != nil {
return 0, 0, err
}
return aggregate.Total, aggregate.Used, nil
}
@@ -0,0 +1,188 @@
//go:build freebsd
package fileutils
import (
"fmt"
"path/filepath"
"strings"
"golang.org/x/sys/unix"
)
// bsdMount is one Getfsstat record reduced to what capacity aggregation
// needs: fstype, source, mountpoint, usage, and the mount's own fsid.
type bsdMount struct {
fstype string
source string
mountpoint string
usage rawUsage
fsid unix.Fsid
}
// resolvedMount is what a path lookup reports for a mountpoint: the
// mountpoint of the filesystem that actually owns the path and that
// filesystem's fsid.
type resolvedMount struct {
mountpoint string
fsid unix.Fsid
ok bool
}
// statfsMountpoint resolves which filesystem a mountpoint path lands on.
func statfsMountpoint(mp string) resolvedMount {
var st unix.Statfs_t
if err := unix.Statfs(mp, &st); err != nil {
return resolvedMount{}
}
return resolvedMount{
mountpoint: filepath.Clean(unix.ByteSliceToString(st.Mntonname[:])),
fsid: st.Fsid,
ok: true,
}
}
// GetPartitionUsageVariants returns the aggregate and root-only capacity views
// of root. FreeBSD has no mountinfo; Getfsstat already carries per-mount statfs
// data, so grouping happens on the result set directly (ZFS datasets grouped by
// pool name, everything else by fsid).
func GetPartitionUsageVariants(root string) (aggregate, rootOnly PartitionUsage, err error) {
if root == "" {
return aggregate, rootOnly, fmt.Errorf("empty path")
}
root = filepath.Clean(root)
rootOnly, rootErr := singlePathPartitionUsage(root)
count, statErr := unix.Getfsstat(nil, unix.MNT_WAIT)
if statErr != nil || count <= 0 {
if rootErr != nil {
return aggregate, rootOnly, rootErr
}
return rootOnly, rootOnly, nil
}
stats := make([]unix.Statfs_t, count)
if _, statErr = unix.Getfsstat(stats, unix.MNT_WAIT); statErr != nil {
if rootErr != nil {
return aggregate, rootOnly, rootErr
}
return rootOnly, rootOnly, nil
}
var mounts []bsdMount
for i := range stats {
s := &stats[i]
avail := uint64(0)
if s.Bavail > 0 {
avail = uint64(s.Bavail)
}
mounts = append(mounts, bsdMount{
fstype: unix.ByteSliceToString(s.Fstypename[:]),
source: unix.ByteSliceToString(s.Mntfromname[:]),
mountpoint: filepath.Clean(unix.ByteSliceToString(s.Mntonname[:])),
usage: rawUsage{
total: s.Blocks * s.Bsize,
avail: avail * s.Bsize,
},
fsid: s.Fsid,
})
}
covering, nested := selectCapacityMounts(mounts, root, statfsMountpoint)
key := func(m bsdMount) string {
if isZFSType(m.fstype) {
return "zfs:" + zfsPoolName(m.source)
}
if m.fsid.Val[0] == 0 && m.fsid.Val[1] == 0 {
// fsid-less filesystems (e.g. some tmpfs mounts): keep per mountpoint.
return "mp:" + m.mountpoint
}
return fmt.Sprintf("fsid:%x:%x", m.fsid.Val[0], m.fsid.Val[1])
}
groups := make(map[string][]rawUsage)
fstypes := make(map[string]string)
add := func(m bsdMount) {
k := key(m)
fstypes[k] = m.fstype
groups[k] = append(groups[k], m.usage)
}
if covering != nil {
add(*covering)
}
for _, m := range nested {
add(m)
}
if len(groups) == 0 {
if rootErr != nil {
return aggregate, rootOnly, rootErr
}
return rootOnly, rootOnly, nil
}
for k, stats := range groups {
u := combineGroupUsage(stats)
if !isZFSType(fstypes[k]) && len(stats) > 1 {
// Non-shared-pool mounts dedupe to a single measurement.
u = combineGroupUsage(stats[:1])
}
aggregate.Total += u.Total
aggregate.Used += u.Used
}
return aggregate, rootOnly, rootErr
}
// selectCapacityMounts picks the mounts contributing capacity under root: the
// filesystem covering root plus mounts nested beneath it.
//
// Getfsstat retains records that path lookups no longer reach: mounts stacked
// under a later mount at the same mountpoint, and mounts below an overmounted
// ancestor (stacked records have no documented order, so position cannot
// identify the visible one). A record counts only when a lookup on its
// mountpoint still lands on it: the resolved mountpoint must equal the
// record's and, for filesystems carrying an fsid, the resolved fsid must match
// the record's (every stacked record resolves to the topmost fsid). fsid-less
// records keep the first match at a mountpoint; stacked fsid-less mounts are
// indistinguishable.
func selectCapacityMounts(mounts []bsdMount, root string, lookup func(string) resolvedMount) (*bsdMount, []bsdMount) {
var covering *bsdMount
var nested []bsdMount
claimed := make(map[string]bool)
resolved := make(map[string]resolvedMount)
for i := range mounts {
m := mounts[i]
covers := m.mountpoint == root || m.mountpoint == "/" || strings.HasPrefix(root, m.mountpoint+"/")
nests := strings.HasPrefix(m.mountpoint, root+"/")
if !covers && !nests {
continue
}
r, seen := resolved[m.mountpoint]
if !seen {
r = lookup(m.mountpoint)
resolved[m.mountpoint] = r
}
if !r.ok || r.mountpoint != m.mountpoint {
continue
}
if m.fsid.Val[0] != 0 || m.fsid.Val[1] != 0 {
if r.fsid != m.fsid {
continue
}
} else if claimed[m.mountpoint] {
continue
}
claimed[m.mountpoint] = true
if covers && (covering == nil || len(m.mountpoint) > len(covering.mountpoint)) {
c := m
covering = &c
}
if nests {
if isZFSType(m.fstype) && isZFSSnapshotMount(m.source, m.mountpoint) {
continue
}
nested = append(nested, m)
}
}
return covering, nested
}
@@ -0,0 +1,161 @@
//go:build freebsd
package fileutils
import (
"testing"
"golang.org/x/sys/unix"
)
func fsidVal(a, b int32) unix.Fsid {
return unix.Fsid{Val: [2]int32{a, b}}
}
func resolvedOK(mp string, fsid unix.Fsid) resolvedMount {
return resolvedMount{mountpoint: mp, fsid: fsid, ok: true}
}
// TestSelectCapacityMountsOvermountedAncestorHidesNested reproduces the
// reviewed layout: /srv/child was mounted first, then /srv was overmounted.
// Getfsstat still reports the unique /srv/child record, but a lookup of that
// path lands inside the covering filesystem, so its usage must be excluded.
func TestSelectCapacityMountsOvermountedAncestorHidesNested(t *testing.T) {
rootFs, childFs, coverFs := fsidVal(1, 1), fsidVal(2, 2), fsidVal(3, 3)
mounts := []bsdMount{
{fstype: "ufs", source: "/dev/da0p2", mountpoint: "/", fsid: rootFs, usage: rawUsage{total: 100, avail: 50}},
{fstype: "ufs", source: "/dev/da1p1", mountpoint: "/srv/child", fsid: childFs, usage: rawUsage{total: 200, avail: 100}},
{fstype: "ufs", source: "/dev/da2p1", mountpoint: "/srv", fsid: coverFs, usage: rawUsage{total: 300, avail: 150}},
}
lookup := func(mp string) resolvedMount {
switch mp {
case "/":
return resolvedOK("/", rootFs)
case "/srv":
return resolvedOK("/srv", coverFs)
case "/srv/child":
// Hidden: resolves to the covering filesystem at /srv.
return resolvedOK("/srv", coverFs)
}
return resolvedMount{}
}
covering, nested := selectCapacityMounts(mounts, "/srv", lookup)
if covering == nil || covering.mountpoint != "/srv" {
t.Fatalf("covering = %+v, want mountpoint /srv", covering)
}
if len(nested) != 0 {
t.Fatalf("nested = %+v, want hidden /srv/child excluded", nested)
}
}
// TestSelectCapacityMountsUnreachableNestedSkipped covers the same hidden
// layout when the covered path no longer exists in the new filesystem: the
// lookup fails outright and the stale record is dropped.
func TestSelectCapacityMountsUnreachableNestedSkipped(t *testing.T) {
rootFs, coverFs := fsidVal(1, 1), fsidVal(3, 3)
mounts := []bsdMount{
{fstype: "ufs", mountpoint: "/", fsid: rootFs},
{fstype: "ufs", source: "/dev/da2p1", mountpoint: "/srv", fsid: coverFs},
{fstype: "ufs", source: "/dev/da1p1", mountpoint: "/srv/gone", fsid: fsidVal(2, 2), usage: rawUsage{total: 200, avail: 100}},
}
lookup := func(mp string) resolvedMount {
switch mp {
case "/":
return resolvedOK("/", rootFs)
case "/srv":
return resolvedOK("/srv", coverFs)
}
return resolvedMount{} // ENOENT under the overmount
}
_, nested := selectCapacityMounts(mounts, "/srv", lookup)
if len(nested) != 0 {
t.Fatalf("nested = %+v, want unreachable /srv/gone excluded", nested)
}
}
// TestSelectCapacityMountsStackedTopmostOnly checks that of several records
// at one mountpoint only the one a lookup resolves to is counted.
func TestSelectCapacityMountsStackedTopmostOnly(t *testing.T) {
rootFs, lowerFs, upperFs := fsidVal(1, 1), fsidVal(4, 4), fsidVal(5, 5)
mounts := []bsdMount{
{fstype: "ufs", mountpoint: "/", fsid: rootFs},
{fstype: "ufs", source: "/dev/da1p1", mountpoint: "/srv/data", fsid: lowerFs, usage: rawUsage{total: 200, avail: 100}},
{fstype: "tmpfs", source: "tmpfs", mountpoint: "/srv/data", fsid: upperFs, usage: rawUsage{total: 60, avail: 30}},
}
lookup := func(mp string) resolvedMount {
switch mp {
case "/":
return resolvedOK("/", rootFs)
case "/srv/data":
return resolvedOK("/srv/data", upperFs)
}
return resolvedMount{}
}
_, nested := selectCapacityMounts(mounts, "/srv", lookup)
if len(nested) != 1 || nested[0].fsid != upperFs {
t.Fatalf("nested = %+v, want only the topmost /srv/data record", nested)
}
}
// TestSelectCapacityMountsHiddenCoveringAncestor: when the deepest ancestor
// mount of root is itself hidden by an overmount, covering must fall back to
// the deepest visible ancestor.
func TestSelectCapacityMountsHiddenCoveringAncestor(t *testing.T) {
rootFs, overFs, staleFs := fsidVal(1, 1), fsidVal(8, 8), fsidVal(9, 9)
mounts := []bsdMount{
{fstype: "ufs", mountpoint: "/", fsid: rootFs},
{fstype: "ufs", source: "/dev/da2p1", mountpoint: "/x", fsid: overFs},
{fstype: "ufs", source: "/dev/da1p1", mountpoint: "/x/y", fsid: staleFs, usage: rawUsage{total: 200, avail: 100}},
}
// /x was mounted over the parent chain of /x/y: lookups of /x/y resolve
// inside the /x filesystem.
lookup := func(mp string) resolvedMount {
switch mp {
case "/":
return resolvedOK("/", rootFs)
case "/x":
return resolvedOK("/x", overFs)
case "/x/y":
return resolvedOK("/x", overFs)
}
return resolvedMount{}
}
covering, nested := selectCapacityMounts(mounts, "/x/y/z", lookup)
if covering == nil || covering.mountpoint != "/x" {
t.Fatalf("covering = %+v, want deepest visible ancestor /x", covering)
}
if len(nested) != 0 {
t.Fatalf("nested = %+v, want none", nested)
}
}
// TestSelectCapacityMountsNestedVisibleKept is the baseline: a nested mount
// that still resolves to itself contributes its usage.
func TestSelectCapacityMountsNestedVisibleKept(t *testing.T) {
rootFs, zfsFs := fsidVal(1, 1), fsidVal(7, 7)
mounts := []bsdMount{
{fstype: "ufs", mountpoint: "/", fsid: rootFs},
{fstype: "zfs", source: "tank/data", mountpoint: "/srv/data", fsid: zfsFs, usage: rawUsage{total: 400, avail: 200}},
}
lookup := func(mp string) resolvedMount {
switch mp {
case "/":
return resolvedOK("/", rootFs)
case "/srv/data":
return resolvedOK("/srv/data", zfsFs)
}
return resolvedMount{}
}
covering, nested := selectCapacityMounts(mounts, "/srv", lookup)
if covering == nil || covering.mountpoint != "/" {
t.Fatalf("covering = %+v, want /", covering)
}
if len(nested) != 1 || nested[0].mountpoint != "/srv/data" {
t.Fatalf("nested = %+v, want [/srv/data]", nested)
}
}
@@ -7,94 +7,64 @@ import (
"os"
"path/filepath"
"syscall"
"golang.org/x/sys/unix"
)
// GetAggregatedPartitionUsage returns total and used bytes summed across distinct
// filesystems mounted at or under root (deduped by major:minor from mountinfo).
// Falls back to a single-path probe when mountinfo cannot be read.
func GetAggregatedPartitionUsage(root string) (total, used uint64, err error) {
// GetPartitionUsageVariants returns two capacity views of root:
// - aggregate: total/used summed across distinct capacity groups mounted at or
// under root, deduped so shared-pool filesystems (ZFS datasets, btrfs
// subvolumes) are counted once per pool.
// - rootOnly: a single statfs of root, ignoring nested mounts.
//
// Falls back to the single-path probe for both when mountinfo cannot be read.
func GetPartitionUsageVariants(root string) (aggregate, rootOnly PartitionUsage, err error) {
if root == "" {
return 0, 0, fmt.Errorf("empty path")
return aggregate, rootOnly, fmt.Errorf("empty path")
}
root = filepath.Clean(root)
paths, parseErr := distinctMountPathsUnder(root)
if parseErr != nil || len(paths) == 0 {
return singlePathPartitionUsage(root)
rootOnly, rootErr := singlePathPartitionUsage(root)
groups, parseErr := mountGroupsUnder(root)
if parseErr != nil || len(groups) == 0 {
if rootErr != nil {
return aggregate, rootOnly, rootErr
}
return rootOnly, rootOnly, nil
}
var firstErr error
for _, path := range paths {
t, u, stErr := partitionUsageAt(path)
if stErr != nil {
if firstErr == nil {
firstErr = stErr
for _, group := range groups {
stats := make([]rawUsage, 0, len(group.paths))
for _, p := range group.paths {
stat, stErr := partitionStatfs(p)
if stErr != nil {
if firstErr == nil {
firstErr = stErr
}
continue
}
continue
stats = append(stats, stat)
}
total += t
used += u
u := combineGroupUsage(stats)
aggregate.Total += u.Total
aggregate.Used += u.Used
}
if total == 0 && used == 0 {
if aggregate.Total == 0 && aggregate.Used == 0 {
if firstErr != nil {
return 0, 0, firstErr
return aggregate, rootOnly, firstErr
}
return singlePathPartitionUsage(root)
aggregate = rootOnly
}
return total, used, nil
return aggregate, rootOnly, rootErr
}
func singlePathPartitionUsage(root string) (total, used uint64, err error) {
total, err = GetPartitionSize(root)
if err != nil {
return 0, 0, err
}
used, err = GetPartitionUsed(root)
if err != nil {
return 0, 0, err
}
return total, used, nil
}
func distinctMountPathsUnder(root string) ([]string, error) {
func mountGroupsUnder(root string) ([]mountGroup, error) {
data, err := readMountinfo()
if err != nil {
return nil, err
}
paths, err := distinctMountPathsFromMountinfo(data, root)
if err != nil {
return nil, err
}
// If mountinfo had no covering mount for root, ensure root is probed.
if len(paths) == 0 {
return []string{root}, nil
}
hasRoot := false
for _, p := range paths {
if p == root {
hasRoot = true
break
}
}
if !hasRoot {
if dev, stErr := deviceIDFromPath(root); stErr == nil {
// Avoid duplicating a device already selected under another path.
already := false
for _, p := range paths {
if d, e := deviceIDFromPath(p); e == nil && d == dev {
already = true
break
}
}
if !already {
paths = append(paths, root)
}
}
}
return paths, nil
return mountGroupsFromMountinfo(data, root)
}
func readMountinfo() (string, error) {
@@ -107,24 +77,14 @@ func readMountinfo() (string, error) {
return "", fmt.Errorf("mountinfo not available")
}
func deviceIDFromPath(path string) (string, error) {
var st unix.Stat_t
if err := unix.Stat(path, &st); err != nil {
return "", err
}
return fmt.Sprintf("%d:%d", unix.Major(uint64(st.Dev)), unix.Minor(uint64(st.Dev))), nil
}
func partitionUsageAt(path string) (total, used uint64, err error) {
func partitionStatfs(path string) (rawUsage, error) {
var stat syscall.Statfs_t
if err := syscall.Statfs(path, &stat); err != nil {
return 0, 0, err
return rawUsage{}, err
}
bsize := blockSize(&stat)
total = uint64(stat.Blocks) * bsize
free := uint64(stat.Bavail) * bsize
if total >= free {
used = total - free
}
return total, used, nil
return rawUsage{
total: uint64(stat.Blocks) * bsize,
avail: uint64(stat.Bavail) * bsize,
}, nil
}
@@ -12,55 +12,111 @@ import (
type mountEntry struct {
deviceID string
mountpoint string
fstype string
source string
}
// distinctMountPathsFromMountinfo selects one path per distinct filesystem that
// contributes capacity for root: the FS that owns root, plus nested mounts under
// root. Deduped by major:minor from mountinfo.
func distinctMountPathsFromMountinfo(mountinfo string, root string) ([]string, error) {
// mountGroup is a set of mountpoints that share one capacity pool. Groups keyed
// by filesystem identity get one probe path; shared-pool filesystems (zfs) keep
// every dataset mountpoint so per-dataset used space can be summed.
type mountGroup struct {
key string
fstype string
paths []string
}
// capacityKey identifies mounts that share the same underlying capacity.
// - zfs: every dataset has its own device ID but draws from the pool; group by pool.
// - btrfs: subvolume mounts have distinct device IDs but report the whole
// filesystem; group by mount source.
// - everything else: device major:minor, as before.
func (e *mountEntry) capacityKey() string {
switch e.fstype {
case "zfs", "zfsfuse":
return "zfs:" + zfsPoolName(e.source)
case "btrfs":
return "btrfs:" + e.source
default:
return "dev:" + e.deviceID
}
}
// mountGroupsFromMountinfo selects the mounts contributing capacity for root:
// the filesystem that owns root, plus nested mounts under root, grouped by
// capacity key. ZFS snapshot mounts are excluded.
func mountGroupsFromMountinfo(mountinfo string, root string) ([]mountGroup, error) {
root = path.Clean(root)
entries, err := parseAllMounts(mountinfo)
if err != nil {
return nil, err
}
seen := make(map[string]string)
groupIndex := make(map[string]int)
var groups []mountGroup
addPath := func(e *mountEntry, p string) {
key := e.capacityKey()
i, ok := groupIndex[key]
if !ok {
i = len(groups)
groupIndex[key] = i
groups = append(groups, mountGroup{key: key, fstype: e.fstype})
}
// Only shared-pool filesystems (zfs) need every mountpoint; other groups
// report identical statfs per mount, so one probe path suffices.
if !isZFSType(groups[i].fstype) && len(groups[i].paths) > 0 {
return
}
for _, existing := range groups[i].paths {
if existing == p {
return
}
}
groups[i].paths = append(groups[i].paths, p)
}
var covering *mountEntry
for i := range entries {
e := &entries[i]
mp := e.mountpoint
if mp == root || strings.HasPrefix(root, mp+"/") || mp == "/" {
if covering == nil || len(mp) > len(covering.mountpoint) {
// >= keeps the later entry: stacked mounts at one mountpoint are
// shadowed by the topmost (last in mountinfo order).
if covering == nil || len(mp) >= len(covering.mountpoint) {
covering = e
}
}
}
if covering != nil {
seen[covering.deviceID] = root
// Probe root itself for the covering filesystem: root may be a plain
// directory inside a mount whose mountpoint is an ancestor.
addPath(covering, root)
} else {
// No mount covers root (e.g. hidden bind mount); probe it directly so
// it is not dropped when nested mounts under root do exist.
groups = append(groups, mountGroup{key: "self:" + root, paths: []string{root}})
}
for _, e := range entries {
// Stacked mounts: only the topmost (last in mountinfo order) is reachable.
lastAtMountpoint := make(map[string]int)
for i := range entries {
lastAtMountpoint[entries[i].mountpoint] = i
}
for i := range entries {
e := &entries[i]
mp := e.mountpoint
if mp != root && !strings.HasPrefix(mp, root+"/") {
continue
}
if _, ok := seen[e.deviceID]; ok {
if lastAtMountpoint[mp] != i {
continue
}
seen[e.deviceID] = mp
if isZFSType(e.fstype) && isZFSSnapshotMount(e.source, mp) {
continue
}
addPath(e, mp)
}
out := make([]string, 0, len(seen))
uniq := make(map[string]struct{}, len(seen))
for _, p := range seen {
if _, ok := uniq[p]; ok {
continue
}
uniq[p] = struct{}{}
out = append(out, p)
}
return out, nil
return groups, nil
}
func parseAllMounts(mountinfo string) ([]mountEntry, error) {
@@ -88,9 +144,16 @@ func parseMountinfoLine(line string) (mountEntry, bool) {
if len(fields) < 5 {
return mountEntry{}, false
}
// After " - ": "fstype source super-options".
post := strings.Fields(parts[1])
if len(post) < 2 {
return mountEntry{}, false
}
return mountEntry{
deviceID: fields[2],
mountpoint: unescapeFstab(fields[4]),
fstype: post[0],
source: unescapeFstab(post[1]),
}, true
}
@@ -7,6 +7,25 @@ import (
"testing"
)
// allGroupPaths flattens groups into a sorted list of probe paths.
func allGroupPaths(groups []mountGroup) []string {
var out []string
for _, g := range groups {
out = append(out, g.paths...)
}
sort.Strings(out)
return out
}
func findGroup(groups []mountGroup, prefix string) *mountGroup {
for i := range groups {
if len(groups[i].key) >= len(prefix) && groups[i].key[:len(prefix)] == prefix {
return &groups[i]
}
}
return nil
}
func TestDistinctMountPathsNestedDisk(t *testing.T) {
// /srv lives on rootfs 8:1; /srv/data is a separate 2TB disk 8:16.
mountinfo := "" +
@@ -14,11 +33,11 @@ func TestDistinctMountPathsNestedDisk(t *testing.T) {
"45 22 8:16 /data /srv/data rw - btrfs /dev/sdb1 rw\n" +
"50 22 0:47 / /proc rw - proc proc rw\n"
paths, err := distinctMountPathsFromMountinfo(mountinfo, "/srv")
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv")
if err != nil {
t.Fatal(err)
}
sort.Strings(paths)
paths := allGroupPaths(groups)
want := []string{"/srv", "/srv/data"}
if len(paths) != len(want) {
t.Fatalf("paths = %v, want %v", paths, want)
@@ -37,11 +56,11 @@ func TestDistinctMountPathsOvermountSamePathDeduped(t *testing.T) {
"45 40 8:16 / /srv/data rw - btrfs /dev/sdb1 rw\n" +
"46 45 0:47 / /srv/data rw - tmpfs tmpfs rw\n"
paths, err := distinctMountPathsFromMountinfo(mountinfo, "/srv")
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv")
if err != nil {
t.Fatal(err)
}
sort.Strings(paths)
paths := allGroupPaths(groups)
want := []string{"/srv", "/srv/data"}
if len(paths) != len(want) {
t.Fatalf("paths = %v, want %v (no duplicate probe paths)", paths, want)
@@ -56,15 +75,15 @@ func TestDistinctMountPathsOvermountSamePathDeduped(t *testing.T) {
func TestDistinctMountPathsBindSameDeviceNotDuplicated(t *testing.T) {
mountinfo := "" +
"22 1 8:1 / / rw - ext4 /dev/sda1 rw\n" +
"45 22 8:16 / /srv/data rw - btrfs /dev/sdb1 rw\n" +
"46 22 8:16 / /srv/data/bind rw,bind - btrfs /dev/sdb1 rw\n"
"45 22 8:16 / /srv/data rw - ext4 /dev/sdb1 rw\n" +
"46 22 8:16 / /srv/data/bind rw,bind - ext4 /dev/sdb1 rw\n"
paths, err := distinctMountPathsFromMountinfo(mountinfo, "/srv")
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv")
if err != nil {
t.Fatal(err)
}
if len(paths) != 2 {
t.Fatalf("expected 2 distinct devices, got %v", paths)
if len(groups) != 2 {
t.Fatalf("expected 2 distinct capacity groups, got %v", groups)
}
}
@@ -74,11 +93,11 @@ func TestDistinctMountPathsSourceIsOwnMount(t *testing.T) {
"40 22 8:2 / /srv rw - ext4 /dev/sda2 rw\n" +
"45 40 8:16 / /srv/data rw - btrfs /dev/sdb1 rw\n"
paths, err := distinctMountPathsFromMountinfo(mountinfo, "/srv")
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv")
if err != nil {
t.Fatal(err)
}
sort.Strings(paths)
paths := allGroupPaths(groups)
want := []string{"/srv", "/srv/data"}
if len(paths) != 2 || paths[0] != want[0] || paths[1] != want[1] {
t.Fatalf("paths = %v, want %v", paths, want)
@@ -90,10 +109,11 @@ func TestDistinctMountPathsIgnoresUnrelated(t *testing.T) {
"22 1 8:1 / / rw - ext4 /dev/sda1 rw\n" +
"30 22 8:3 / /home rw - ext4 /dev/sda3 rw\n"
paths, err := distinctMountPathsFromMountinfo(mountinfo, "/srv")
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv")
if err != nil {
t.Fatal(err)
}
paths := allGroupPaths(groups)
if len(paths) != 1 || paths[0] != "/srv" {
t.Fatalf("paths = %v, want [/srv]", paths)
}
@@ -110,4 +130,124 @@ func TestParseMountinfoEscapedSpace(t *testing.T) {
if entry.deviceID != "98:0" {
t.Fatalf("deviceID = %q, want 98:0", entry.deviceID)
}
if entry.fstype != "ext3" {
t.Fatalf("fstype = %q, want ext3", entry.fstype)
}
if entry.source != "/dev/root" {
t.Fatalf("source = %q, want /dev/root", entry.source)
}
}
// TestMountGroupsZFSPoolDatasetsGrouped reproduces issue #3025: every ZFS
// dataset gets its own device ID, so major:minor dedup counts pool capacity
// once per dataset. They must share one capacity group.
func TestMountGroupsZFSPoolDatasetsGrouped(t *testing.T) {
mountinfo := "" +
"22 1 8:1 / / rw - ext4 /dev/sda1 rw\n" +
"40 22 0:55 / /srv/pool rw - zfs pool rw\n" +
"41 40 0:56 / /srv/pool/ds01 rw - zfs pool/ds01 rw\n" +
"42 41 0:57 / /srv/pool/ds01/a rw - zfs pool/ds01/a rw\n" +
"43 40 0:58 / /srv/pool/ds02 rw - zfs pool/ds02 rw\n" +
"44 22 0:59 / /other/pool2 rw - zfs pool2 rw\n"
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv/pool")
if err != nil {
t.Fatal(err)
}
g := findGroup(groups, "zfs:pool")
if g == nil || g.key != "zfs:pool" {
t.Fatalf("expected one zfs:pool group, got %v", groups)
}
want := []string{"/srv/pool", "/srv/pool/ds01", "/srv/pool/ds01/a", "/srv/pool/ds02"}
got := append([]string{}, g.paths...)
sort.Strings(got)
sort.Strings(want)
if len(got) != len(want) {
t.Fatalf("zfs group paths = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("zfs group paths = %v, want %v", got, want)
}
}
// The unrelated second pool must not join the group.
if g2 := findGroup(groups, "zfs:pool2"); g2 != nil {
t.Fatalf("unrelated pool2 should not be included: %v", groups)
}
}
func TestMountGroupsZFSSnapshotsSkipped(t *testing.T) {
mountinfo := "" +
"22 1 8:1 / / rw - ext4 /dev/sda1 rw\n" +
"40 22 0:55 / /srv/pool rw - zfs pool rw\n" +
"41 40 0:56 / /srv/pool/ds01 rw - zfs pool/ds01 rw\n" +
"42 41 0:57 / /srv/pool/ds01/.zfs/snapshot/snap1 rw - zfs pool/ds01@snap1 rw\n"
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv/pool")
if err != nil {
t.Fatal(err)
}
paths := allGroupPaths(groups)
for _, p := range paths {
if p == "/srv/pool/ds01/.zfs/snapshot/snap1" {
t.Fatalf("snapshot mount should be skipped: %v", paths)
}
}
if len(paths) != 2 {
t.Fatalf("paths = %v, want 2 live datasets", paths)
}
}
func TestMountGroupsBtrfsSubvolumesDeduped(t *testing.T) {
// Each btrfs subvolume mount has a distinct anonymous device but reports
// whole-filesystem statfs; they must collapse into one group.
mountinfo := "" +
"22 1 8:1 / / rw - ext4 /dev/sda1 rw\n" +
"40 22 0:33 /@ /srv rw - btrfs /dev/sdb1 rw\n" +
"41 40 0:34 /@home /srv/home rw - btrfs /dev/sdb1 rw\n" +
"42 40 0:35 /@data /srv/data rw - btrfs /dev/sdb1 rw\n"
groups, err := mountGroupsFromMountinfo(mountinfo, "/srv")
if err != nil {
t.Fatal(err)
}
g := findGroup(groups, "btrfs:")
if g == nil || g.key != "btrfs:/dev/sdb1" {
t.Fatalf("expected one btrfs group, got %v", groups)
}
// Subvolume mounts report identical fs-wide statfs; one probe path suffices.
if len(g.paths) != 1 || g.paths[0] != "/srv" {
t.Fatalf("btrfs group paths = %v, want [/srv]", g.paths)
}
if len(groups) != 1 {
t.Fatalf("expected exactly 1 group for /srv, got %v", groups)
}
}
// TestCombineGroupUsageZFSPool verifies the arithmetic from #3025: 26 datasets
// each reporting ~19.7T avail must yield used+19.7T total, not 26*19.7T+used.
func TestCombineGroupUsageZFSPool(t *testing.T) {
const avail = uint64(19_700_000_000_000) // ~19.7 TiB pool free
stats := []rawUsage{{total: 20_000_000_000_000, avail: avail}} // root ds: 0.3T used
for i := 0; i < 25; i++ {
stats = append(stats, rawUsage{total: avail + 100_000_000_000, avail: avail})
}
// One quota'd child: reports a smaller avail; must not shrink the pool.
stats = append(stats, rawUsage{total: 1_000_000_000_000, avail: 500_000_000_000})
u := combineGroupUsage(stats)
wantUsed := uint64(300_000_000_000) + 25*uint64(100_000_000_000) + 500_000_000_000
if u.Used != wantUsed {
t.Fatalf("used = %d, want %d", u.Used, wantUsed)
}
if u.Total != wantUsed+avail {
t.Fatalf("total = %d, want %d (used+single pool avail)", u.Total, wantUsed+avail)
}
}
func TestCombineGroupUsageSingleMountIdentity(t *testing.T) {
u := combineGroupUsage([]rawUsage{{total: 1000, avail: 400}})
if u.Total != 1000 || u.Used != 600 {
t.Fatalf("single mount = %+v, want total 1000 used 600", u)
}
}
@@ -1,17 +1,14 @@
//go:build !linux
//go:build !linux && !freebsd
package fileutils
// GetAggregatedPartitionUsage returns total and used bytes for the filesystem
// that owns root. Non-Linux builds use a single statfs/GetDiskFreeSpaceEx path.
func GetAggregatedPartitionUsage(root string) (total, used uint64, err error) {
total, err = GetPartitionSize(root)
// GetPartitionUsageVariants returns the aggregate and root-only capacity views
// of root. Without mount info only a single statfs of the source path is
// available, so both variants are identical.
func GetPartitionUsageVariants(root string) (aggregate, rootOnly PartitionUsage, err error) {
u, err := singlePathPartitionUsage(root)
if err != nil {
return 0, 0, err
return aggregate, rootOnly, err
}
used, err = GetPartitionUsed(root)
if err != nil {
return 0, 0, err
}
return total, used, nil
return u, u, nil
}
@@ -0,0 +1,50 @@
//go:build linux || freebsd
package fileutils
import "strings"
// rawUsage is a single filesystem measurement in bytes.
type rawUsage struct {
total uint64
avail uint64
}
// combineGroupUsage folds statfs results for mounts that share one capacity pool
// into a single total/used pair. Per-mount used (total-avail) is summed, while
// shared free space is counted once via the largest reported avail. This keeps
// ZFS datasets or btrfs subvolumes from multiplying the pool's capacity by the
// number of mounts under a source.
func combineGroupUsage(stats []rawUsage) PartitionUsage {
var used, avail uint64
for _, s := range stats {
if s.total > s.avail {
used += s.total - s.avail
}
if s.avail > avail {
avail = s.avail
}
}
return PartitionUsage{Total: used + avail, Used: used}
}
// isZFSType reports whether fstype is a ZFS variant whose datasets share pool
// capacity across mounts (kernel zfs or zfsfuse).
func isZFSType(fstype string) bool {
return fstype == "zfs" || fstype == "zfsfuse"
}
// zfsPoolName returns the pool component of a ZFS mount source ("pool/ds@snap" -> "pool").
func zfsPoolName(source string) string {
if i := strings.IndexAny(source, "/@"); i >= 0 {
return source[:i]
}
return source
}
// isZFSSnapshotMount reports whether a ZFS mount is a snapshot (.zfs snapdir or
// an "@snapshot" source). Snapshots share pool space with the live datasets and
// are excluded from capacity aggregation.
func isZFSSnapshotMount(source, mountpoint string) bool {
return strings.Contains(source, "@") || strings.Contains(mountpoint, "/.zfs/")
}
@@ -0,0 +1,17 @@
//go:build !windows
package fileutils
import (
"errors"
"os"
"syscall"
)
func isCrossDeviceRenameError(err error) bool {
if errors.Is(err, syscall.EXDEV) {
return true
}
var linkErr *os.LinkError
return errors.As(err, &linkErr) && errors.Is(linkErr.Err, syscall.EXDEV)
}
@@ -0,0 +1,18 @@
//go:build windows
package fileutils
import (
"errors"
"os"
"syscall"
)
func isCrossDeviceRenameError(err error) bool {
const errNotSameDevice = syscall.Errno(17)
if errors.Is(err, errNotSameDevice) {
return true
}
var linkErr *os.LinkError
return errors.As(err, &linkErr) && errors.Is(linkErr.Err, errNotSameDevice)
}
+12
View File
@@ -0,0 +1,12 @@
package analytics
import (
"os"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/testutil"
)
func TestMain(m *testing.M) {
os.Exit(testutil.Main(m))
}
+213
View File
@@ -0,0 +1,213 @@
// Package archiveencoding decodes ZIP entry names before path normalization.
package archiveencoding
import (
"archive/zip"
"encoding/binary"
"errors"
"fmt"
"hash/crc32"
"strings"
"unicode"
"unicode/utf8"
"golang.org/x/text/encoding"
"golang.org/x/text/encoding/charmap"
"golang.org/x/text/encoding/japanese"
"golang.org/x/text/encoding/korean"
"golang.org/x/text/encoding/simplifiedchinese"
"golang.org/x/text/encoding/traditionalchinese"
)
// ErrEncoding indicates a filename encoding that needs a different user selection.
var ErrEncoding = errors.New("invalid ZIP filename encoding")
type codec struct {
name string
encoding encoding.Encoding
}
var codecs = []codec{
{"utf-8", nil},
{"cp932", japanese.ShiftJIS},
{"gb18030", simplifiedchinese.GB18030},
{"big5", traditionalchinese.Big5},
{"euc-kr", korean.EUCKR},
{"cp437", charmap.CodePage437},
{"windows-1252", charmap.Windows1252},
}
// Candidate contains a bounded preview, decoded exactly as during extraction.
type Candidate struct {
Encoding string `json:"encoding"`
Names []string `json:"names"`
}
// Preview describes compatible encodings. An empty suggestion requires a choice;
// a heuristic suggestion is never a guarantee that the names are correct.
type Preview struct {
Suggested string `json:"suggested"`
Candidates []Candidate `json:"candidates"`
}
// unicodeName prefers the UTF-8 flag, then a verified Info-ZIP Unicode Path field.
// A Unicode Path field is usable only when its version and original-name CRC match.
func unicodeName(f *zip.File) (string, bool) {
if f.Flags&0x800 != 0 {
return f.Name, true
}
extra := f.Extra
for len(extra) >= 4 {
tag := binary.LittleEndian.Uint16(extra)
size := int(binary.LittleEndian.Uint16(extra[2:]))
extra = extra[4:]
if size > len(extra) {
break
}
field := extra[:size]
extra = extra[size:]
if tag == 0x7075 && len(field) > 5 && field[0] == 1 &&
binary.LittleEndian.Uint32(field[1:5]) == crc32.ChecksumIEEE([]byte(f.Name)) && utf8.Valid(field[5:]) {
return string(field[5:]), true
}
}
return "", false
}
func decode(f *zip.File, c codec) (string, error) {
name, declared := unicodeName(f)
if !declared {
name = f.Name
if c.encoding != nil {
var err error
name, err = c.encoding.NewDecoder().String(name)
if err != nil || strings.ContainsRune(name, utf8.RuneError) {
return "", fmt.Errorf("%w: cannot decode filename as %s", ErrEncoding, c.name)
}
}
}
if !utf8.ValidString(name) || strings.ContainsRune(name, 0) {
return "", fmt.Errorf("%w: invalid filename for %s", ErrEncoding, c.name)
}
return name, nil
}
// Inspect considers all entry names without decompressing file contents. Candidate
// previews prefer names lacking Unicode metadata so that users can compare them.
func Inspect(files []*zip.File) Preview {
result := Preview{Candidates: []Candidate{}}
bestScore, tied := 0, false
for _, c := range codecs {
candidate := Candidate{Encoding: c.name, Names: []string{}}
fallback := []string{}
score, valid := 0, true
for _, f := range files {
name, err := decode(f, c)
if err != nil {
valid = false
break
}
if len(fallback) < 5 {
fallback = append(fallback, name)
}
if _, declared := unicodeName(f); !declared && !isASCII(f.Name) {
if len(candidate.Names) < 5 {
candidate.Names = append(candidate.Names, name)
}
// The EUC-KR decoder also accepts CP949 extension bytes, including
// CP932 kana pairs. Do not use those bytes as Korean evidence.
if c.name != "euc-kr" || isEUCKR(f.Name) {
score += scriptEvidence(name, c.name)
}
}
}
if !valid {
continue
}
if len(candidate.Names) == 0 {
candidate.Names = fallback
}
result.Candidates = append(result.Candidates, candidate)
// Valid UTF-8 (including metadata-backed names) needs no legacy guess.
if c.name == "utf-8" {
result.Suggested = c.name
}
if score > bestScore {
bestScore, tied = score, false
if result.Suggested != "utf-8" {
result.Suggested = c.name
}
} else if score == bestScore && score > 0 {
tied = true
}
}
if result.Suggested != "utf-8" && (bestScore < 3 || tied) {
result.Suggested = ""
}
return result
}
func isASCII(s string) bool {
for i := 0; i < len(s); i++ {
if s[i] >= utf8.RuneSelf {
return false
}
}
return true
}
// isEUCKR excludes CP949 extension sequences from automatic Korean suggestions.
func isEUCKR(raw string) bool {
for i := 0; i < len(raw); i++ {
if raw[i] < utf8.RuneSelf {
continue
}
if raw[i] < 0xa1 || raw[i] > 0xfe || i+1 >= len(raw) || raw[i+1] < 0xa1 || raw[i+1] > 0xfe {
return false
}
i++
}
return true
}
// Kana and Hangul give useful script evidence; Han alone does not distinguish
// Japanese, simplified Chinese and traditional Chinese. Half-width kana are not
// evidence: unrelated legacy bytes frequently decode to half-width kana.
func scriptEvidence(name, charset string) int {
score := 0
for _, r := range name {
if charset == "cp932" && r < 0xff00 && (unicode.Is(unicode.Hiragana, r) || unicode.Is(unicode.Katakana, r)) {
score++
}
if charset == "euc-kr" && unicode.Is(unicode.Hangul, r) {
score++
}
}
return score
}
// DecodeNames resolves the selection once for the entire archive. Explicit
// Unicode metadata always wins over the selected fallback encoding.
func DecodeNames(files []*zip.File, charset string) ([]string, error) {
if charset == "" {
charset = Inspect(files).Suggested
if charset == "" {
return nil, fmt.Errorf("%w: choose an encoding before extracting", ErrEncoding)
}
}
for _, c := range codecs {
if c.name != charset {
continue
}
names := make([]string, 0, len(files))
for _, f := range files {
name, err := decode(f, c)
if err != nil {
return nil, err
}
names = append(names, name)
}
return names, nil
}
return nil, fmt.Errorf("%w: unsupported encoding %s", ErrEncoding, charset)
}
@@ -0,0 +1,119 @@
package archiveencoding
import (
"archive/zip"
"encoding/binary"
"golang.org/x/text/encoding/japanese"
"hash/crc32"
"reflect"
"testing"
)
func entry(name string) *zip.File { return &zip.File{FileHeader: zip.FileHeader{Name: name}} }
func cp932(t *testing.T, name string) string {
t.Helper()
s, err := japanese.ShiftJIS.NewEncoder().String(name)
if err != nil {
t.Fatal(err)
}
return s
}
func TestJapaneseSuggestionMatchesDecoding(t *testing.T) {
t.Parallel()
want := []string{"日本語/テストソ001.txt", "日本語/テストソ002.txt"}
files := []*zip.File{entry(cp932(t, want[0])), entry(cp932(t, want[1]))}
preview := Inspect(files)
if preview.Suggested != "cp932" {
t.Fatalf("suggestion: %+v", preview)
}
names, err := DecodeNames(files, preview.Suggested)
if err != nil || !reflect.DeepEqual(names, want) {
t.Fatalf("names=%q err=%v", names, err)
}
for _, c := range preview.Candidates {
if c.Encoding == preview.Suggested && !reflect.DeepEqual(c.Names, names) {
t.Fatalf("preview mismatch: %q", c.Names)
}
}
}
func TestAmbiguousNamesRequireSelection(t *testing.T) {
t.Parallel()
files := []*zip.File{entry("caf\x82.txt")}
if p := Inspect(files); p.Suggested != "" {
t.Fatalf("ambiguous name was guessed: %+v", p)
}
if _, err := DecodeNames(files, ""); err == nil {
t.Fatal("expected explicit choice")
}
names, err := DecodeNames(files, "cp437")
if err != nil || names[0] != "café.txt" {
t.Fatalf("names=%q err=%v", names, err)
}
for _, c := range []string{"utf-8", "unknown"} {
if _, err = DecodeNames(files, c); err == nil {
t.Fatalf("invalid encoding accepted: %s", c)
}
}
}
func unicodeExtra(raw, name string) []byte {
payload := append([]byte{1, 0, 0, 0, 0}, []byte(name)...)
binary.LittleEndian.PutUint32(payload[1:], crc32.ChecksumIEEE([]byte(raw)))
extra := []byte{0x75, 0x70, 0, 0}
binary.LittleEndian.PutUint16(extra[2:], uint16(len(payload)))
return append(extra, payload...)
}
func TestUnicodeMetadataTakesPrecedence(t *testing.T) {
t.Parallel()
flagged := entry("日本語.txt")
flagged.Flags = 0x800
extra := entry("legacy.txt")
extra.Extra = unicodeExtra(extra.Name, "別の名前.txt")
names, err := DecodeNames([]*zip.File{flagged, extra}, "cp437")
if err != nil || !reflect.DeepEqual(names, []string{"日本語.txt", "別の名前.txt"}) {
t.Fatalf("names=%q err=%v", names, err)
}
extra.Extra[5] ^= 1
names, err = DecodeNames([]*zip.File{extra}, "cp437")
if err != nil || names[0] != "legacy.txt" {
t.Fatalf("names=%q err=%v", names, err)
}
}
func TestInvalidUnicodeMetadata(t *testing.T) {
t.Parallel()
for _, raw := range []string{"bad\xff.txt", "bad\x00.txt"} {
f := entry(raw)
f.Flags = 0x800
if _, err := DecodeNames([]*zip.File{f}, "cp932"); err == nil {
t.Fatalf("invalid declared UTF-8 accepted: %q", raw)
}
}
f := entry("ok.txt")
f.Extra = []byte{0x75, 0x70, 0xff, 0xff}
if _, err := DecodeNames([]*zip.File{f}, "utf-8"); err != nil {
t.Fatal(err)
}
}
func TestUTF8AndASCIIArchives(t *testing.T) {
t.Parallel()
for _, files := range [][]*zip.File{nil, {entry("readme.txt")}, {entry("日本語.txt")}} {
if p := Inspect(files); p.Suggested != "utf-8" {
t.Fatalf("expected UTF-8: %+v", p)
}
}
}
func TestPreviewBoundAndAllNamesValidated(t *testing.T) {
t.Parallel()
files := make([]*zip.File, 20)
for i := range files {
files[i] = entry("readme.txt")
}
files = append(files, entry("invalid\xff.txt"))
for _, c := range Inspect(files).Candidates {
if len(c.Names) > 5 {
t.Fatal("unbounded preview")
}
if c.Encoding == "utf-8" {
t.Fatal("entry beyond preview was not validated")
}
}
}
+7 -1
View File
@@ -6,9 +6,10 @@ import (
"time"
"github.com/golang-jwt/jwt/v4"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
// Auther is the authentication interface.
@@ -39,12 +40,17 @@ func MakeSignedTokenAPI(user *users.User, name string, duration time.Duration, p
if _, ok := user.Tokens[name]; ok {
return "", users.AuthToken{}, fmt.Errorf("key already exists with same name %v ", name)
}
id, err := utils.RandomHex(16)
if err != nil {
return "", users.AuthToken{}, fmt.Errorf("generate token id: %w", err)
}
now := time.Now()
expires := now.Add(duration)
// Create minimal token with only JWT standard claims
claim := users.AuthToken{
MinimalAuthToken: users.MinimalAuthToken{
RegisteredClaims: jwt.RegisteredClaims{
ID: id,
IssuedAt: jwt.NewNumericDate(now),
ExpiresAt: jwt.NewNumericDate(expires),
Issuer: FB_ISSUER,
+1 -1
View File
@@ -14,5 +14,5 @@ type JSONAuth struct {
// Auth authenticates the user via a json in content body (legacy method for compatibility).
func (auther JSONAuth) Auth(r *http.Request, _ *users.Storage) (*users.User, error) {
return AuthenticatePassword(r, auther.ReCaptcha)
return AuthenticatePassword(r, auther.ReCaptcha, false)
}
+12
View File
@@ -0,0 +1,12 @@
package auth
import (
"os"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/testutil"
)
func TestMain(m *testing.M) {
os.Exit(testutil.Main(m))
}
+1 -4
View File
@@ -15,10 +15,7 @@ type NoAuth struct{}
// AuthenticateNoAuth authenticates as the configured admin user with no credentials required.
func AuthenticateNoAuth(r *http.Request, user *users.Storage) (*users.User, error) {
admin := settings.Config.Auth.AdminUsername
if admin == "" {
admin = "admin"
}
admin := settings.PasswordAdminUsername()
id, err := users.ResolveUsernameToID(admin)
if err != nil {
return nil, err
+31
View File
@@ -1,8 +1,10 @@
package auth
import (
"encoding/json"
"net/http"
"os"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
@@ -38,3 +40,32 @@ func AuthenticateProxy(r *http.Request, usr *users.Storage, headerName string) (
func (a ProxyAuth) Auth(r *http.Request, usr *users.Storage) (*users.User, error) {
return AuthenticateProxy(r, usr, a.Header)
}
// ExtractGroupsFromHeader reads group/role values from a proxy header named by groupsClaim.
// Returns present=false when the header name is empty or the header is missing/blank.
func ExtractGroupsFromHeader(r *http.Request, headerName string) ([]string, bool) {
if headerName == "" {
return nil, false
}
headerVal := strings.TrimSpace(r.Header.Get(headerName))
if headerVal == "" {
return nil, false
}
if strings.HasPrefix(headerVal, "[") {
var groups []string
if err := json.Unmarshal([]byte(headerVal), &groups); err == nil {
return groups, true
}
}
groups := ExtractGroupsFromClaims(map[string]interface{}{headerName: headerVal}, headerName)
if len(groups) == 1 && strings.Contains(groups[0], ",") {
parts := strings.Split(groups[0], ",")
groups = make([]string, 0, len(parts))
for _, part := range parts {
if part = strings.TrimSpace(part); part != "" {
groups = append(groups, part)
}
}
}
return groups, true
}
@@ -0,0 +1,81 @@
package auth_test
import (
"net/http"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/auth"
)
func TestExtractGroupsFromHeader_SingleValue(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "/", http.NoBody)
if err != nil {
t.Fatal(err)
}
req.Header.Set("X-Cosmos-Role", "2")
groups, present := auth.ExtractGroupsFromHeader(req, "x-cosmos-role")
if !present {
t.Fatal("expected groups to be present")
}
if len(groups) != 1 || groups[0] != "2" {
t.Fatalf("groups = %#v, want [\"2\"]", groups)
}
}
func TestExtractGroupsFromHeader_CommaSeparated(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "/", http.NoBody)
if err != nil {
t.Fatal(err)
}
req.Header.Set("X-Groups", "1, 2")
groups, present := auth.ExtractGroupsFromHeader(req, "X-Groups")
if !present {
t.Fatal("expected groups to be present")
}
if len(groups) != 2 || groups[0] != "1" || groups[1] != "2" {
t.Fatalf("groups = %#v, want [\"1\", \"2\"]", groups)
}
}
func TestExtractGroupsFromHeader_JSONArray(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "/", http.NoBody)
if err != nil {
t.Fatal(err)
}
req.Header.Set("X-Groups", `["admin","users"]`)
groups, present := auth.ExtractGroupsFromHeader(req, "X-Groups")
if !present {
t.Fatal("expected groups to be present")
}
if len(groups) != 2 || groups[0] != "admin" || groups[1] != "users" {
t.Fatalf("groups = %#v, want [\"admin\", \"users\"]", groups)
}
}
func TestExtractGroupsFromHeader_Missing(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "/", http.NoBody)
if err != nil {
t.Fatal(err)
}
groups, present := auth.ExtractGroupsFromHeader(req, "X-Groups")
if present {
t.Fatalf("expected groups to be absent, got %#v", groups)
}
}
func TestExtractGroupsFromHeader_EmptyHeaderName(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "/", http.NoBody)
if err != nil {
t.Fatal(err)
}
req.Header.Set("X-Groups", "admin")
groups, present := auth.ExtractGroupsFromHeader(req, "")
if present {
t.Fatalf("expected groups to be absent, got %#v", groups)
}
}
+7 -3
View File
@@ -30,7 +30,8 @@ func SetDefault(s *Service) {
}
// AuthenticatePassword authenticates the user via password in request headers.
func (s *Service) AuthenticatePassword(r *http.Request, disableOtp bool) (*users.User, error) {
// When skipRequirePasswordChange is true, a valid password is accepted even if RequirePasswordChange is set (change-required flow).
func (s *Service) AuthenticatePassword(r *http.Request, disableOtp bool, skipRequirePasswordChange bool) (*users.User, error) {
if s == nil || s.users == nil {
return nil, fmt.Errorf("auth service not configured")
}
@@ -63,6 +64,9 @@ func (s *Service) AuthenticatePassword(r *http.Request, disableOtp bool) (*users
if getErr != nil {
return nil, fmt.Errorf("unable to get user from store: %v", err)
}
if user.RequirePasswordChange && user.LoginMethod == users.LoginMethodPassword && !skipRequirePasswordChange {
return nil, errors.ErrPasswordChangeRequired
}
if user.TOTPSecret != "" && !disableOtp {
if totpCode == "" {
return nil, errors.ErrNoTotpProvided
@@ -80,9 +84,9 @@ func (s *Service) AuthenticatePassword(r *http.Request, disableOtp bool) (*users
}
// AuthenticatePassword authenticates via the default service.
func AuthenticatePassword(r *http.Request, disableOtp bool) (*users.User, error) {
func AuthenticatePassword(r *http.Request, disableOtp bool, skipRequirePasswordChange bool) (*users.User, error) {
if defaultService != nil {
return defaultService.AuthenticatePassword(r, disableOtp)
return defaultService.AuthenticatePassword(r, disableOtp, skipRequirePasswordChange)
}
return nil, fmt.Errorf("auth service not configured")
}
@@ -0,0 +1,56 @@
package auth
import (
"testing"
"testing/synctest"
"time"
"github.com/golang-jwt/jwt/v4"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func TestSignedTokensAreUniqueAtSameTime(t *testing.T) {
origKey := settings.Config.Auth.Key
settings.Config.Auth.Key = "test-signing-key-for-token-uniqueness"
t.Cleanup(func() { settings.Config.Auth.Key = origKey })
// Freeze the clock so every token has identical issue and expiry times.
synctest.Test(t, func(t *testing.T) {
now := time.Now()
tokens := make(map[string]users.AuthToken)
for _, userID := range []uint64{1, 2} {
user := &users.User{ID: userID}
for _, minimal := range []bool{true, false} {
for range 2 {
tokenString, claim, err := MakeSignedTokenAPI(user, "test-token", time.Hour, users.Permissions{Api: true}, minimal)
if err != nil {
t.Fatal(err)
}
if _, exists := tokens[tokenString]; exists {
t.Fatalf("token collision for user %d with minimal=%v", userID, minimal)
}
tokens[tokenString] = claim
}
}
}
ids := make(map[string]bool)
for tokenString, claim := range tokens {
var parsed users.MinimalAuthToken
if _, err := jwt.ParseWithClaims(tokenString, &parsed, JWTSigningKeyFunc()); err != nil {
t.Fatalf("parse signed token: %v", err)
}
if parsed.ID == "" || ids[parsed.ID] {
t.Fatal("each signed token must have a unique nonempty id")
}
ids[parsed.ID] = true
if parsed.ID != claim.ID {
t.Fatal("signed token id must match returned metadata")
}
if parsed.IssuedAt == nil || !parsed.IssuedAt.Equal(now) || parsed.ExpiresAt == nil || !parsed.ExpiresAt.Equal(now.Add(time.Hour)) {
t.Fatal("token uniqueness must not depend on changing timestamps")
}
assertMinimalJWTClaims(t, tokenString)
}
})
}
+497 -103
View File
@@ -10,10 +10,11 @@ import (
"sync"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/golang-jwt/jwt/v4"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-cache/cache"
"github.com/gtsteffaniak/go-logger/logger"
)
@@ -65,11 +66,37 @@ type GroupMap map[string]StringSet
type HashedTokenInfo struct {
UserID uint64
IsSession bool
// ExpiresAt is the token's exp claim as unix time; 0 means no expiry (e.g.
// non-JWT strings or rows persisted before expiry tracking).
ExpiresAt int64
}
// sessionTokenRetireGrace is how long a rotated session token stays usable after
// renewal. In-flight requests still carrying the previous cookie must not 401.
const sessionTokenRetireGrace = 2 * time.Minute
// BearerTokenGrace is how long a bearer token may still resolve after session
// rotation or after its JWT exp (e.g. share-ACL identity). In-flight requests
// still carrying the previous cookie must not 401 during rotation.
const BearerTokenGrace = 2 * time.Minute
// TokenExpiryUnix extracts the exp claim of a bearer JWT without verifying the
// signature; the value is only used for expiry bookkeeping and tokens are
// verified on the auth path. Returns 0 for non-JWT strings or tokens without
// an expiry claim.
func TokenExpiryUnix(tokenString string) int64 {
claims := jwt.MapClaims{}
if _, _, err := jwt.NewParser().ParseUnverified(tokenString, claims); err != nil {
return 0
}
exp, ok := claims["exp"].(float64)
if !ok {
return 0
}
return int64(exp)
}
// TokenExpiredPastGrace reports whether a token expiry is older than the
// resolution grace window. expiresAt == 0 (unknown/none) never expires.
func TokenExpiredPastGrace(expiresAt int64, now time.Time) bool {
return expiresAt != 0 && now.After(time.Unix(expiresAt, 0).Add(BearerTokenGrace))
}
// Storage manages access rules and group membership.
type Storage struct {
@@ -82,17 +109,30 @@ type Storage struct {
sqlStore SQLPersister // SQL store for persistence
}
// RuleKey identifies one access_rules row.
type RuleKey struct {
Source string
Path string
}
// RuleUpsert is an access_rules row to insert or replace.
type RuleUpsert struct {
RuleKey
Rule *AccessRule
}
// SQLPersister interface for SQL persistence operations
type SQLPersister interface {
SaveAccessRule(source, path string, rule *AccessRule) error
DeleteAccessRule(source, path string) error
SaveGroup(name string, members StringSet) error
DeleteGroup(name string) error
DeleteGroupWithRules(name string, save []RuleUpsert, remove []RuleKey) error
SaveRevokedToken(tokenHash string, revokedAt int64) error
PersistImmediateTokenRevocation(tokenHash string) error
PersistTokenRetirement(tokenHash string, revokedAt int64, pruned []string) error
DeleteRevokedToken(tokenHash string) error
SaveHashedToken(tokenHash string, userID uint64, isSession bool) error
SaveHashedToken(tokenHash string, userID uint64, isSession bool, expiresAt int64) error
DeleteHashedToken(tokenHash string) error
DeleteHashedTokensByUserID(userID uint64) error
}
@@ -109,30 +149,35 @@ func (s *Storage) Flush() error {
// persistGroupSQLNL upserts or deletes one groups row to match in-memory state.
// Caller must hold s.mux. If the group is absent from s.Groups, it is deleted from SQL.
func (s *Storage) persistGroupSQLNL(groupname string) {
func (s *Storage) persistGroupSQLNL(groupname string) error {
if s.sqlStore == nil {
return
return nil
}
members, ok := s.Groups[groupname]
if !ok {
if err := s.sqlStore.DeleteGroup(groupname); err != nil {
logger.Errorf("failed to delete group %q from sql: %v", groupname, err)
return fmt.Errorf("failed to delete group %q from sql: %w", groupname, err)
}
return
return nil
}
if err := s.sqlStore.SaveGroup(groupname, members); err != nil {
logger.Errorf("failed to save group %q: %v", groupname, err)
return fmt.Errorf("failed to save group %q: %w", groupname, err)
}
return nil
}
// ensureGroupExistsNL creates an empty in-memory group if missing and write-through persists it.
// Caller must hold s.mux.
func (s *Storage) ensureGroupExistsNL(groupname string) {
func (s *Storage) ensureGroupExistsNL(groupname string) error {
if _, ok := s.Groups[groupname]; ok {
return
return nil
}
s.Groups[groupname] = make(StringSet)
s.persistGroupSQLNL(groupname)
if err := s.persistGroupSQLNL(groupname); err != nil {
delete(s.Groups, groupname)
return err
}
return nil
}
// NewStorage creates a new Storage instance.
@@ -176,8 +221,14 @@ func accessRuleHasPayload(rule *AccessRule) bool {
// persistRuleSQLNL upserts or deletes one access_rules row to match in-memory state.
// Caller must hold s.mux (write lock). normalizedPath must match rule map keys (see getOrCreateRuleNL).
func (s *Storage) persistRuleSQLNL(sourcePath, normalizedPath string) {
if err := s.persistRuleSQLErrorNL(sourcePath, normalizedPath); err != nil {
logger.Warningf("%v", err)
}
}
func (s *Storage) persistRuleSQLErrorNL(sourcePath, normalizedPath string) error {
if s.sqlStore == nil {
return
return nil
}
var rule *AccessRule
if bySrc, ok := s.AllRules[sourcePath]; ok {
@@ -185,13 +236,30 @@ func (s *Storage) persistRuleSQLNL(sourcePath, normalizedPath string) {
}
if !accessRuleHasPayload(rule) {
if err := s.sqlStore.DeleteAccessRule(sourcePath, normalizedPath); err != nil {
logger.Warningf("access rules SQL delete (source=%s path=%s): %v", sourcePath, normalizedPath, err)
return fmt.Errorf("access rules SQL delete (source=%s path=%s): %w", sourcePath, normalizedPath, err)
}
return
return nil
}
if err := s.sqlStore.SaveAccessRule(sourcePath, normalizedPath, rule); err != nil {
logger.Warningf("access rules SQL save (source=%s path=%s): %v", sourcePath, normalizedPath, err)
return fmt.Errorf("access rules SQL save (source=%s path=%s): %w", sourcePath, normalizedPath, err)
}
return nil
}
func (s *Storage) restoreRuleSQLNL(key RuleKey, rule *AccessRule) error {
if s.sqlStore == nil {
return nil
}
if !accessRuleHasPayload(rule) {
if err := s.sqlStore.DeleteAccessRule(key.Source, key.Path); err != nil {
return fmt.Errorf("access rules SQL delete (source=%s path=%s): %w", key.Source, key.Path, err)
}
return nil
}
if err := s.sqlStore.SaveAccessRule(key.Source, key.Path, rule); err != nil {
return fmt.Errorf("access rules SQL save (source=%s path=%s): %w", key.Source, key.Path, err)
}
return nil
}
// RemoveRuleByPath removes a rule by normalized index path.
@@ -280,7 +348,9 @@ func (s *Storage) AllowUser(sourcePath string, indexPath utils.IndexPath, userna
func (s *Storage) DenyGroup(sourcePath string, indexPath utils.IndexPath, groupname string) error {
s.mux.Lock()
defer s.mux.Unlock()
s.ensureGroupExistsNL(groupname)
if err := s.ensureGroupExistsNL(groupname); err != nil {
return err
}
rule := s.getOrCreateRuleNL(sourcePath, indexPath)
if _, ok := rule.Deny.Groups[groupname]; ok {
return errors.ErrExist
@@ -296,7 +366,9 @@ func (s *Storage) DenyGroup(sourcePath string, indexPath utils.IndexPath, groupn
func (s *Storage) AllowGroup(sourcePath string, indexPath utils.IndexPath, groupname string) error {
s.mux.Lock()
defer s.mux.Unlock()
s.ensureGroupExistsNL(groupname)
if err := s.ensureGroupExistsNL(groupname); err != nil {
return err
}
rule := s.getOrCreateRuleNL(sourcePath, indexPath)
if _, ok := rule.Allow.Groups[groupname]; ok {
return errors.ErrExist
@@ -546,14 +618,22 @@ func (s *Storage) GetAllRules(sourcePath string) (map[string]FrontendAccessRule,
func (s *Storage) AddUserToGroup(group, username string) error {
s.mux.Lock()
defer s.mux.Unlock()
createdGroup := false
if _, ok := s.Groups[group]; !ok {
s.Groups[group] = make(StringSet)
createdGroup = true
}
if _, ok := s.Groups[group][username]; ok {
return nil
}
s.Groups[group][username] = struct{}{}
s.persistGroupSQLNL(group)
if err := s.persistGroupSQLNL(group); err != nil {
delete(s.Groups[group], username)
if createdGroup {
delete(s.Groups, group)
}
return err
}
s.clearAllCaches()
return nil
}
@@ -570,6 +650,98 @@ func (s *Storage) GetAllGroups() []string {
return groups
}
// GetGroupMembers returns every group with its sorted member usernames.
func (s *Storage) GetGroupMembers() map[string][]string {
s.mux.RLock()
defer s.mux.RUnlock()
result := make(map[string][]string, len(s.Groups))
for group, members := range s.Groups {
names := make([]string, 0, len(members))
for name := range members {
names = append(names, name)
}
sort.Strings(names)
result[group] = names
}
return result
}
// SetGroupMembers creates the group if needed and replaces its membership.
func (s *Storage) SetGroupMembers(group string, usernames []string) error {
s.mux.Lock()
defer s.mux.Unlock()
members := make(StringSet, len(usernames))
for _, name := range usernames {
if name != "" {
members[name] = struct{}{}
}
}
previous, existed := s.Groups[group]
s.Groups[group] = members
if err := s.persistGroupSQLNL(group); err != nil {
// Keep memory and SQL consistent: undo the in-memory change.
if existed {
s.Groups[group] = previous
} else {
delete(s.Groups, group)
}
return err
}
s.clearAllCaches()
return nil
}
// DeleteGroup removes a group and every access rule entry that references it.
// The group row and all affected rule rows are written in one SQL transaction, so a
// failure stores nothing. On failure the in-memory groups and rules are restored and the
// error is returned. The lock is held throughout, so no other change can interleave.
func (s *Storage) DeleteGroup(group string) error {
s.mux.Lock()
defer s.mux.Unlock()
previousMembers, existed := s.Groups[group]
delete(s.Groups, group)
touched, originals := s.removeAllRulesForGroupNL(group)
if !existed && len(touched) == 0 {
return nil
}
if err := s.persistGroupDeleteNL(group, touched); err != nil {
if existed {
s.Groups[group] = previousMembers
}
for key, rule := range originals {
if s.AllRules[key.Source] == nil {
s.AllRules[key.Source] = make(RuleMap)
}
s.AllRules[key.Source][key.Path] = rule
}
return err
}
s.clearAllCaches()
return nil
}
// persistGroupDeleteNL writes a group deletion and the resulting rule row changes to SQL
// in one transaction. Caller must hold s.mux.
func (s *Storage) persistGroupDeleteNL(group string, touched []RuleKey) error {
if s.sqlStore == nil {
return nil
}
var save []RuleUpsert
var remove []RuleKey
for _, key := range touched {
var rule *AccessRule
if bySrc, ok := s.AllRules[key.Source]; ok {
rule = bySrc[key.Path]
}
if accessRuleHasPayload(rule) {
save = append(save, RuleUpsert{RuleKey: key, Rule: rule})
} else {
remove = append(remove, key)
}
}
return s.sqlStore.DeleteGroupWithRules(group, save, remove)
}
// GetUserGroups returns all groups for a specific user.
func (s *Storage) GetUserGroups(username string) []string {
s.mux.RLock()
@@ -589,41 +761,72 @@ func (s *Storage) GetUserGroups(username string) []string {
func (s *Storage) SyncUserGroups(username string, newGroups []string) error {
s.mux.Lock()
defer s.mux.Unlock()
affected := make(map[string]struct{})
// Create a set of new groups for efficient lookup
newGroupsSet := make(StringSet, len(newGroups))
for _, g := range newGroups {
if g == "" {
if !IsUsableSyncedGroupName(g) {
continue
}
newGroupsSet[g] = struct{}{}
}
// Iterate over all existing groups to find the user's current memberships
affected := make(map[string]struct{})
for group, members := range s.Groups {
_, userIsInGroup := members[username]
_, groupIsInNewSet := newGroupsSet[group]
// If user is in a group that is not in their new set of groups, remove them.
if userIsInGroup && !groupIsInNewSet {
delete(s.Groups[group], username)
affected[group] = struct{}{}
}
}
for group := range newGroupsSet {
if _, ok := s.Groups[group]; !ok {
affected[group] = struct{}{}
continue
}
if _, ok := s.Groups[group][username]; !ok {
affected[group] = struct{}{}
}
}
// Add user to new groups
before := make(map[string]StringSet, len(affected))
for group := range affected {
before[group] = cloneStringSet(s.Groups[group])
}
for group, members := range s.Groups {
_, userIsInGroup := members[username]
_, groupIsInNewSet := newGroupsSet[group]
if userIsInGroup && !groupIsInNewSet {
delete(s.Groups[group], username)
}
}
for group := range newGroupsSet {
if _, ok := s.Groups[group]; !ok {
s.Groups[group] = make(StringSet)
}
if _, ok := s.Groups[group][username]; !ok {
s.Groups[group][username] = struct{}{}
affected[group] = struct{}{}
}
s.Groups[group][username] = struct{}{}
}
for group := range affected {
s.persistGroupSQLNL(group)
affectedGroups := slices.Sorted(maps.Keys(affected))
var persisted []string
for _, group := range affectedGroups {
if err := s.persistGroupSQLNL(group); err != nil {
for _, g := range persisted {
if restoreErr := s.restoreGroupSQLNL(g, before[g]); restoreErr != nil {
logger.Errorf("failed to restore group %q in sql after sync failure: %v", g, restoreErr)
}
}
for g, snap := range before {
if len(snap) == 0 {
delete(s.Groups, g)
} else {
s.Groups[g] = snap
}
}
return err
}
persisted = append(persisted, group)
}
if len(affected) > 0 {
s.clearAllCaches()
@@ -631,6 +834,31 @@ func (s *Storage) SyncUserGroups(username string, newGroups []string) error {
return nil
}
func (s *Storage) restoreGroupSQLNL(groupname string, members StringSet) error {
if s.sqlStore == nil {
return nil
}
if len(members) == 0 {
if err := s.sqlStore.DeleteGroup(groupname); err != nil {
return fmt.Errorf("failed to delete group %q from sql: %w", groupname, err)
}
return nil
}
if err := s.sqlStore.SaveGroup(groupname, members); err != nil {
return fmt.Errorf("failed to save group %q: %w", groupname, err)
}
return nil
}
func cloneStringSet(src StringSet) StringSet {
if len(src) == 0 {
return nil
}
dst := make(StringSet, len(src))
maps.Copy(dst, src)
return dst
}
// RemoveUserFromGroup removes a username from a group.
func (s *Storage) RemoveUserFromGroup(group, username string) error {
s.mux.Lock()
@@ -643,7 +871,147 @@ func (s *Storage) RemoveUserFromGroup(group, username string) error {
return nil
}
delete(members, username)
s.persistGroupSQLNL(group)
if err := s.persistGroupSQLNL(group); err != nil {
s.Groups[group][username] = struct{}{}
return err
}
s.clearAllCaches()
return nil
}
// RemoveUserFromAllGroups removes a username from every group's member list.
// Group rows themselves are kept, even when left empty. Used when a user is deleted.
func (s *Storage) RemoveUserFromAllGroups(username string) error {
s.mux.Lock()
defer s.mux.Unlock()
affected := make(map[string]StringSet)
for group, members := range s.Groups {
if _, ok := members[username]; ok {
affected[group] = cloneStringSet(members)
}
}
if len(affected) == 0 {
return nil
}
for group := range affected {
delete(s.Groups[group], username)
}
var persisted []string
for _, group := range slices.Sorted(maps.Keys(affected)) {
if err := s.persistGroupSQLNL(group); err != nil {
for _, g := range persisted {
if restoreErr := s.restoreGroupSQLNL(g, affected[g]); restoreErr != nil {
logger.Errorf("failed to restore group %q in sql after failure: %v", g, restoreErr)
}
}
for g, snap := range affected {
s.Groups[g] = snap
}
return err
}
persisted = append(persisted, group)
}
s.clearAllCaches()
return nil
}
// RenameUserInGroups replaces one username with another in every group member list.
// Used when a user's username changes so memberships follow the rename.
func (s *Storage) RenameUserInGroups(oldName, newName string) error {
s.mux.Lock()
defer s.mux.Unlock()
if oldName == newName {
return nil
}
affected := make(map[string]StringSet)
for group, members := range s.Groups {
if _, ok := members[oldName]; ok {
affected[group] = cloneStringSet(members)
}
}
if len(affected) == 0 {
return nil
}
for group := range affected {
delete(s.Groups[group], oldName)
s.Groups[group][newName] = struct{}{}
}
var persisted []string
for _, group := range slices.Sorted(maps.Keys(affected)) {
if err := s.persistGroupSQLNL(group); err != nil {
for _, g := range persisted {
if restoreErr := s.restoreGroupSQLNL(g, affected[g]); restoreErr != nil {
logger.Errorf("failed to restore group %q in sql after failure: %v", g, restoreErr)
}
}
for g, snap := range affected {
s.Groups[g] = snap
}
return err
}
persisted = append(persisted, group)
}
s.clearAllCaches()
return nil
}
// RenameUserInRules replaces one username with another in every allow and deny user list.
// Used when a user's username changes so path access rules follow the rename.
func (s *Storage) RenameUserInRules(oldName, newName string) error {
s.mux.Lock()
defer s.mux.Unlock()
if oldName == newName {
return nil
}
affected := make(map[RuleKey]*AccessRule)
var keys []RuleKey
for sourcePath, rulesBySource := range s.AllRules {
for indexPath, rule := range rulesBySource {
_, inAllow := rule.Allow.Users[oldName]
_, inDeny := rule.Deny.Users[oldName]
if !inAllow && !inDeny {
continue
}
key := RuleKey{Source: sourcePath, Path: indexPath}
affected[key] = cloneAccessRule(rule)
if inAllow {
delete(rule.Allow.Users, oldName)
rule.Allow.Users[newName] = struct{}{}
}
if inDeny {
delete(rule.Deny.Users, oldName)
rule.Deny.Users[newName] = struct{}{}
}
keys = append(keys, key)
}
}
if len(keys) == 0 {
return nil
}
slices.SortFunc(keys, func(a, b RuleKey) int {
if c := strings.Compare(a.Source, b.Source); c != 0 {
return c
}
return strings.Compare(a.Path, b.Path)
})
var persisted []RuleKey
for _, key := range keys {
if err := s.persistRuleSQLErrorNL(key.Source, key.Path); err != nil {
for _, pk := range persisted {
if restoreErr := s.restoreRuleSQLNL(pk, affected[pk]); restoreErr != nil {
logger.Errorf("failed to restore access rule %q/%q in sql after failure: %v", pk.Source, pk.Path, restoreErr)
}
}
for key, snap := range affected {
if s.AllRules[key.Source] == nil {
s.AllRules[key.Source] = make(RuleMap)
}
s.AllRules[key.Source][key.Path] = snap
}
return err
}
persisted = append(persisted, key)
}
s.clearAllCaches()
return nil
}
@@ -794,91 +1162,104 @@ func (s *Storage) RemoveDenyAll(sourcePath string, indexPath utils.IndexPath) (b
}
// RemoveAllRulesForUser removes a user from all allow and deny lists.
// On a persistence failure the in-memory and already-persisted rules are restored and the error is returned.
func (s *Storage) RemoveAllRulesForUser(username string) error {
s.mux.Lock()
defer s.mux.Unlock()
changed := false
dirty := make(map[string]map[string]struct{})
touch := func(sp, ip string) {
if dirty[sp] == nil {
dirty[sp] = make(map[string]struct{})
}
dirty[sp][ip] = struct{}{}
}
affected := make(map[RuleKey]*AccessRule)
for sourcePath, rulesBySource := range s.AllRules {
for indexPath, rule := range rulesBySource {
if _, exists := rule.Allow.Users[username]; exists {
delete(rule.Allow.Users, username)
touch(sourcePath, indexPath)
changed = true
_, inAllow := rule.Allow.Users[username]
_, inDeny := rule.Deny.Users[username]
if !inAllow && !inDeny {
continue
}
if _, exists := rule.Deny.Users[username]; exists {
delete(rule.Deny.Users, username)
touch(sourcePath, indexPath)
changed = true
}
if len(rule.Allow.Users) == 0 && len(rule.Allow.Groups) == 0 && len(rule.Deny.Users) == 0 && len(rule.Deny.Groups) == 0 {
delete(s.AllRules[sourcePath], indexPath)
if len(s.AllRules[sourcePath]) == 0 {
delete(s.AllRules, sourcePath)
affected[RuleKey{Source: sourcePath, Path: indexPath}] = cloneAccessRule(rule)
delete(rule.Allow.Users, username)
delete(rule.Deny.Users, username)
}
}
if len(affected) == 0 {
return nil
}
keys := slices.SortedFunc(maps.Keys(affected), func(a, b RuleKey) int {
if c := strings.Compare(a.Source, b.Source); c != 0 {
return c
}
return strings.Compare(a.Path, b.Path)
})
for _, key := range keys {
rule := s.AllRules[key.Source][key.Path]
if rule.DenyAll || len(rule.Allow.Users) > 0 || len(rule.Allow.Groups) > 0 || len(rule.Deny.Users) > 0 || len(rule.Deny.Groups) > 0 {
continue
}
delete(s.AllRules[key.Source], key.Path)
if len(s.AllRules[key.Source]) == 0 {
delete(s.AllRules, key.Source)
}
}
var persisted []RuleKey
for _, key := range keys {
if err := s.persistRuleSQLErrorNL(key.Source, key.Path); err != nil {
for _, pk := range persisted {
if restoreErr := s.restoreRuleSQLNL(pk, affected[pk]); restoreErr != nil {
logger.Errorf("failed to restore access rule %q/%q in sql after failure: %v", pk.Source, pk.Path, restoreErr)
}
touch(sourcePath, indexPath)
}
}
}
if changed {
s.clearAllCaches()
for sp, paths := range dirty {
for ip := range paths {
s.persistRuleSQLNL(sp, ip)
for k, snap := range affected {
if s.AllRules[k.Source] == nil {
s.AllRules[k.Source] = make(RuleMap)
}
s.AllRules[k.Source][k.Path] = snap
}
return err
}
persisted = append(persisted, key)
}
s.clearAllCaches()
return nil
}
// RemoveAllRulesForGroup removes a group from all allow and deny lists.
func (s *Storage) RemoveAllRulesForGroup(groupname string) error {
s.mux.Lock()
defer s.mux.Unlock()
changed := false
dirty := make(map[string]map[string]struct{})
touch := func(sp, ip string) {
if dirty[sp] == nil {
dirty[sp] = make(map[string]struct{})
}
dirty[sp][ip] = struct{}{}
}
// removeAllRulesForGroupNL removes a group from every allow and deny list in memory and
// returns the rules it touched (rules left empty are deleted) plus a deep copy of each
// touched rule's previous state for rollback. It does not persist; the caller must hold
// s.mux and save the returned rules.
func (s *Storage) removeAllRulesForGroupNL(groupname string) ([]RuleKey, map[RuleKey]*AccessRule) {
var touched []RuleKey
originals := make(map[RuleKey]*AccessRule)
for sourcePath, rulesBySource := range s.AllRules {
for indexPath, rule := range rulesBySource {
if _, exists := rule.Allow.Groups[groupname]; exists {
delete(rule.Allow.Groups, groupname)
touch(sourcePath, indexPath)
changed = true
_, inAllow := rule.Allow.Groups[groupname]
_, inDeny := rule.Deny.Groups[groupname]
if !inAllow && !inDeny {
continue
}
if _, exists := rule.Deny.Groups[groupname]; exists {
delete(rule.Deny.Groups, groupname)
touch(sourcePath, indexPath)
changed = true
}
if len(rule.Allow.Users) == 0 && len(rule.Allow.Groups) == 0 && len(rule.Deny.Users) == 0 && len(rule.Deny.Groups) == 0 {
key := RuleKey{Source: sourcePath, Path: indexPath}
originals[key] = cloneAccessRule(rule)
delete(rule.Allow.Groups, groupname)
delete(rule.Deny.Groups, groupname)
// A rule left with only DenyAll must stay, otherwise removing the group would open the path.
if !accessRuleHasPayload(rule) {
delete(s.AllRules[sourcePath], indexPath)
if len(s.AllRules[sourcePath]) == 0 {
delete(s.AllRules, sourcePath)
}
touch(sourcePath, indexPath)
}
touched = append(touched, key)
}
}
if changed {
s.clearAllCaches()
for sp, paths := range dirty {
for ip := range paths {
s.persistRuleSQLNL(sp, ip)
}
}
return touched, originals
}
func cloneAccessRule(rule *AccessRule) *AccessRule {
if rule == nil {
return nil
}
return &AccessRule{
DenyAll: rule.DenyAll,
Deny: RuleSet{Users: cloneStringSet(rule.Deny.Users), Groups: cloneStringSet(rule.Deny.Groups)},
Allow: RuleSet{Users: cloneStringSet(rule.Allow.Users), Groups: cloneStringSet(rule.Allow.Groups)},
}
return nil
}
// GetRulesForUser returns all rules for a specific user for a given sourcePath.
@@ -1338,7 +1719,7 @@ func (s *Storage) RevokeToken(tokenString string) error {
}
// RetireToken schedules a rotated session token for revocation after
// sessionTokenRetireGrace. The owner mapping is kept during the grace window so
// BearerTokenGrace. The owner mapping is kept during the grace window so
// in-flight requests still carrying the previous cookie stay authenticated.
func (s *Storage) RetireToken(tokenString string) error {
tokenHash := utils.HashSHA256(tokenString)
@@ -1415,7 +1796,7 @@ func (rb *tokenRetireRollback) capture(s *Storage, tokenHash string, now time.Ti
rb.prunedRevoked = make(map[string]int64)
rb.prunedHashed = make(map[string]HashedTokenInfo)
for hash, revokedAt := range revokedAfter {
if revokedAt != 0 && now.Sub(time.Unix(revokedAt, 0)) < sessionTokenRetireGrace {
if revokedAt != 0 && now.Sub(time.Unix(revokedAt, 0)) < BearerTokenGrace {
continue
}
if at, ok := s.RevokedTokens[hash]; ok {
@@ -1448,7 +1829,7 @@ func (rb *tokenRetireRollback) apply(s *Storage) {
func (s *Storage) pruneRevocationsNL(now time.Time) []string {
var pruned []string
for hash, revokedAt := range s.RevokedTokens {
if revokedAt == 0 || now.Sub(time.Unix(revokedAt, 0)) >= sessionTokenRetireGrace {
if revokedAt == 0 || now.Sub(time.Unix(revokedAt, 0)) >= BearerTokenGrace {
delete(s.RevokedTokens, hash)
delete(s.HashedTokens, hash)
pruned = append(pruned, hash)
@@ -1470,7 +1851,7 @@ func (s *Storage) IsTokenRevoked(tokenString string) bool {
if revokedAt == 0 {
return true
}
return time.Since(time.Unix(revokedAt, 0)) >= sessionTokenRetireGrace
return time.Since(time.Unix(revokedAt, 0)) >= BearerTokenGrace
}
// AddApiToken maps a named API token string hash to an owner user id.
@@ -1484,13 +1865,19 @@ func (s *Storage) AddSessionToken(tokenString string, userID uint64) error {
}
func (s *Storage) addHashedToken(tokenString string, userID uint64, isSession bool) error {
expiresAt := TokenExpiryUnix(tokenString)
// Tokens already past the expiry grace window are never registered; expired
// mappings are pruned at startup load instead of on each registration.
if TokenExpiredPastGrace(expiresAt, time.Now()) {
return nil
}
tokenHash := utils.HashSHA256(tokenString)
s.mux.Lock()
s.HashedTokens[tokenHash] = HashedTokenInfo{UserID: userID, IsSession: isSession}
s.HashedTokens[tokenHash] = HashedTokenInfo{UserID: userID, IsSession: isSession, ExpiresAt: expiresAt}
sqlStore := s.sqlStore
s.mux.Unlock()
if sqlStore != nil {
return sqlStore.SaveHashedToken(tokenHash, userID, isSession)
return sqlStore.SaveHashedToken(tokenHash, userID, isSession, expiresAt)
}
return nil
}
@@ -1502,12 +1889,19 @@ func (s *Storage) GetUserIDFromToken(tokenString string) (uint64, bool) {
}
// GetHashedTokenInfo retrieves owner/type metadata for a given token string.
// Tokens past their expiry grace window resolve as unknown.
func (s *Storage) GetHashedTokenInfo(tokenString string) (HashedTokenInfo, bool) {
s.mux.RLock()
defer s.mux.RUnlock()
tokenHash := utils.HashSHA256(tokenString)
info, exists := s.HashedTokens[tokenHash]
return info, exists
if !exists {
return info, false
}
if TokenExpiredPastGrace(info.ExpiresAt, time.Now()) {
return HashedTokenInfo{}, false
}
return info, true
}
// GetRevokedTokens returns a copy of the revocation map (hash → revocation unix time).
@@ -1,6 +1,7 @@
package access_test
import (
stderrors "errors"
"os"
"path/filepath"
"strings"
@@ -1851,3 +1852,200 @@ func TestAddUserToGroup_WriteThrough(t *testing.T) {
t.Fatal("bob should be removed from editors in SQL")
}
}
func TestSetGroupMembers_ReplacesMembership(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
if err := s.SetGroupMembers("editors", []string{"alice", "bob", ""}); err != nil {
t.Fatalf("SetGroupMembers: %v", err)
}
if err := s.SetGroupMembers("editors", []string{"bob", "carol"}); err != nil {
t.Fatalf("SetGroupMembers replace: %v", err)
}
got := s.GetGroupMembers()["editors"]
if len(got) != 2 || got[0] != "bob" || got[1] != "carol" {
t.Fatalf("unexpected members: %v", got)
}
sqlGroups, err := sqlStore.GetAllGroups()
if err != nil {
t.Fatalf("GetAllGroups: %v", err)
}
if _, ok := sqlGroups["editors"]["alice"]; ok {
t.Fatal("alice should have been removed in SQL")
}
if _, ok := sqlGroups["editors"]["carol"]; !ok {
t.Fatal("carol should be persisted in SQL")
}
}
func TestDeleteGroup_RemovesGroupAndRules(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
if err := s.SetGroupMembers("acme", []string{"alice"}); err != nil {
t.Fatalf("SetGroupMembers: %v", err)
}
if err := s.AllowGroup("mnt/storage", idxPath("/tenant"), "acme"); err != nil {
t.Fatalf("AllowGroup: %v", err)
}
if err := s.DeleteGroup("acme"); err != nil {
t.Fatalf("DeleteGroup: %v", err)
}
if _, ok := s.GetGroupMembers()["acme"]; ok {
t.Fatal("acme should be gone from memory")
}
sqlGroups, err := sqlStore.GetAllGroups()
if err != nil {
t.Fatalf("GetAllGroups: %v", err)
}
if _, ok := sqlGroups["acme"]; ok {
t.Fatal("acme should be gone from SQL")
}
if rules := s.GetRulesForGroup("mnt/storage", "acme"); len(rules) != 0 {
t.Fatalf("expected no rules for deleted group, got %v", rules)
}
// Deleting a missing group is a no-op.
if err := s.DeleteGroup("acme"); err != nil {
t.Fatalf("DeleteGroup missing: %v", err)
}
}
// failingGroupDeletePersister wraps a real store but fails the transactional group delete.
type failingGroupDeletePersister struct {
access.SQLPersister
}
func (failingGroupDeletePersister) DeleteGroupWithRules(string, []access.RuleUpsert, []access.RuleKey) error {
return stderrors.New("simulated sql failure")
}
func TestDeleteGroup_FailureLeavesGroupAndRulesIntact(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
if err := s.SetGroupMembers("acme", []string{"alice"}); err != nil {
t.Fatalf("SetGroupMembers: %v", err)
}
if err := s.AllowGroup("mnt/storage", idxPath("/tenant"), "acme"); err != nil {
t.Fatalf("AllowGroup: %v", err)
}
if err := s.DenyGroup("mnt/storage", idxPath("/private"), "acme"); err != nil {
t.Fatalf("DenyGroup: %v", err)
}
rulesBefore := len(s.GetRulesForGroup("mnt/storage", "acme"))
if rulesBefore != 2 {
t.Fatalf("expected 2 rules before delete, got %d", rulesBefore)
}
s.SetSQLStore(failingGroupDeletePersister{sqlStore})
if err := s.DeleteGroup("acme"); err == nil {
t.Fatal("expected DeleteGroup to return the SQL error")
}
if got := s.GetGroupMembers()["acme"]; len(got) != 1 || got[0] != "alice" {
t.Fatalf("group should be restored after a failed delete, got %v", got)
}
if got := len(s.GetRulesForGroup("mnt/storage", "acme")); got != rulesBefore {
t.Fatalf("rules should be restored after a failed delete: want %d, got %d", rulesBefore, got)
}
// SQL was never changed.
sqlGroups, err := sqlStore.GetAllGroups()
if err != nil {
t.Fatal(err)
}
if _, ok := sqlGroups["acme"]; !ok {
t.Fatal("group row should still be in SQL")
}
sqlRules, err := sqlStore.GetAllAccessRules()
if err != nil {
t.Fatal(err)
}
if len(sqlRules["mnt/storage"]) != 2 {
t.Fatalf("rule rows should still be in SQL, got %v", sqlRules["mnt/storage"])
}
}
func TestDeleteGroup_RemovesGroupAndRuleRowsFromSQL(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
if err := s.SetGroupMembers("acme", []string{"alice"}); err != nil {
t.Fatalf("SetGroupMembers: %v", err)
}
if err := s.AllowGroup("mnt/storage", idxPath("/tenant"), "acme"); err != nil {
t.Fatalf("AllowGroup: %v", err)
}
if err := s.AllowUser("mnt/storage", idxPath("/tenant"), "alice"); err != nil {
t.Fatalf("AllowUser: %v", err)
}
if err := s.DeleteGroup("acme"); err != nil {
t.Fatalf("DeleteGroup: %v", err)
}
sqlGroups, err := sqlStore.GetAllGroups()
if err != nil {
t.Fatal(err)
}
if _, ok := sqlGroups["acme"]; ok {
t.Fatal("group row should be gone from SQL")
}
sqlRules, err := sqlStore.GetAllAccessRules()
if err != nil {
t.Fatal(err)
}
for _, rule := range sqlRules["mnt/storage"] {
if _, has := rule.Allow.Groups["acme"]; has {
t.Fatal("no stored rule should still reference the deleted group")
}
}
// The rule that also allowed a user must survive with only the group removed.
if len(sqlRules["mnt/storage"]) != 1 {
t.Fatalf("expected the user rule to remain, got %v", sqlRules["mnt/storage"])
}
}
func TestDeleteGroup_KeepsDenyAllRule(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
// "Deny everyone except group acme": a DenyAll rule whose only allow entry is the group.
if err := s.SetGroupMembers("acme", []string{"alice"}); err != nil {
t.Fatalf("SetGroupMembers: %v", err)
}
if err := s.DenyAll("mnt/storage", idxPath("/tenant")); err != nil {
t.Fatalf("DenyAll: %v", err)
}
if err := s.AllowGroup("mnt/storage", idxPath("/tenant"), "acme"); err != nil {
t.Fatalf("AllowGroup: %v", err)
}
if !s.Permitted("mnt/storage", idxPath("/tenant"), "alice") {
t.Fatal("alice should be permitted through the group before it is deleted")
}
if err := s.DeleteGroup("acme"); err != nil {
t.Fatalf("DeleteGroup: %v", err)
}
// Deleting the group must not remove the deny-all rule and open the path to everyone.
if s.Permitted("mnt/storage", idxPath("/tenant"), "alice") || s.Permitted("mnt/storage", idxPath("/tenant"), "bob") {
t.Fatal("path must stay denied for everyone after the allowed group is deleted")
}
rules := s.GetRulesForGroup("mnt/storage", "acme")
if len(rules) != 0 {
t.Fatalf("no rule should reference the deleted group, got %v", rules)
}
sqlRules, err := sqlStore.GetAllAccessRules()
if err != nil {
t.Fatal(err)
}
stored := 0
for _, r := range sqlRules["mnt/storage"] {
if r.DenyAll {
stored++
}
}
if stored != 1 {
t.Fatalf("expected the deny-all rule to remain stored in SQL, found %d", stored)
}
}
@@ -0,0 +1,174 @@
package access_test
import (
"errors"
"maps"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
)
type stubGroupSQL struct {
saveErr error
deleteErr error
}
func (s *stubGroupSQL) SaveAccessRule(string, string, *access.AccessRule) error { return nil }
func (s *stubGroupSQL) DeleteAccessRule(string, string) error { return nil }
func (s *stubGroupSQL) SaveGroup(string, access.StringSet) error {
return s.saveErr
}
func (s *stubGroupSQL) DeleteGroup(string) error { return s.deleteErr }
func (s *stubGroupSQL) DeleteGroupWithRules(string, []access.RuleUpsert, []access.RuleKey) error {
return s.deleteErr
}
func (s *stubGroupSQL) SaveRevokedToken(string, int64) error {
return nil
}
func (s *stubGroupSQL) PersistImmediateTokenRevocation(string) error { return nil }
func (s *stubGroupSQL) PersistTokenRetirement(string, int64, []string) error { return nil }
func (s *stubGroupSQL) DeleteRevokedToken(string) error { return nil }
func (s *stubGroupSQL) SaveHashedToken(string, uint64, bool, int64) error { return nil }
func (s *stubGroupSQL) DeleteHashedToken(string) error { return nil }
func (s *stubGroupSQL) DeleteHashedTokensByUserID(uint64) error {
return nil
}
type recordingGroupSQL struct {
groups map[string]access.StringSet
saveCalls int
failOn int
}
func (s *recordingGroupSQL) SaveAccessRule(string, string, *access.AccessRule) error { return nil }
func (s *recordingGroupSQL) DeleteAccessRule(string, string) error { return nil }
func (s *recordingGroupSQL) SaveGroup(name string, members access.StringSet) error {
if s.groups == nil {
s.groups = make(map[string]access.StringSet)
}
s.saveCalls++
if s.failOn > 0 && s.saveCalls == s.failOn {
return errors.New("save failed")
}
s.groups[name] = cloneStringSet(members)
return nil
}
func (s *recordingGroupSQL) DeleteGroup(name string) error {
if s.groups == nil {
s.groups = make(map[string]access.StringSet)
}
delete(s.groups, name)
return nil
}
func (s *recordingGroupSQL) DeleteGroupWithRules(name string, _ []access.RuleUpsert, _ []access.RuleKey) error {
return s.DeleteGroup(name)
}
func (s *recordingGroupSQL) SaveRevokedToken(string, int64) error { return nil }
func (s *recordingGroupSQL) PersistImmediateTokenRevocation(string) error { return nil }
func (s *recordingGroupSQL) PersistTokenRetirement(string, int64, []string) error { return nil }
func (s *recordingGroupSQL) DeleteRevokedToken(string) error { return nil }
func (s *recordingGroupSQL) SaveHashedToken(string, uint64, bool, int64) error { return nil }
func (s *recordingGroupSQL) DeleteHashedToken(string) error { return nil }
func (s *recordingGroupSQL) DeleteHashedTokensByUserID(uint64) error { return nil }
func cloneStringSet(src access.StringSet) access.StringSet {
if len(src) == 0 {
return nil
}
dst := make(access.StringSet, len(src))
maps.Copy(dst, src)
return dst
}
func TestAddUserToGroup_RollsBackOnSQLFailure(t *testing.T) {
userStore := users.NewStorage(nil)
store := access.NewStorage(userStore)
store.SetSQLStore(&stubGroupSQL{saveErr: errors.New("save failed")})
err := store.AddUserToGroup("editors", "alice")
if err == nil {
t.Fatal("expected SQL save error")
}
groups := store.GetUserGroups("alice")
if len(groups) != 0 {
t.Fatalf("expected in-memory rollback, alice groups = %#v", groups)
}
}
func TestRemoveUserFromGroup_RollsBackOnSQLFailure(t *testing.T) {
userStore := users.NewStorage(nil)
store := access.NewStorage(userStore)
if err := store.AddUserToGroup("editors", "alice"); err != nil {
t.Fatalf("seed group: %v", err)
}
store.SetSQLStore(&stubGroupSQL{saveErr: errors.New("save failed")})
err := store.RemoveUserFromGroup("editors", "alice")
if err == nil {
t.Fatal("expected SQL save error")
}
groups := store.GetUserGroups("alice")
if len(groups) != 1 || groups[0] != "editors" {
t.Fatalf("expected in-memory rollback, alice groups = %#v", groups)
}
}
func TestAllowGroup_ReturnsSQLSaveError(t *testing.T) {
setupTestSources()
store := access.NewStorage(users.NewStorage(nil))
store.SetSQLStore(&stubGroupSQL{saveErr: errors.New("save failed")})
err := store.AllowGroup("mnt/storage", idxPath("/tenant"), "acme")
if err == nil {
t.Fatal("expected SQL save error from AllowGroup")
}
}
func TestEnsureGroupExistsNL_RollsBackOnSQLFailure(t *testing.T) {
setupTestSources()
store := access.NewStorage(users.NewStorage(nil))
store.SetSQLStore(&stubGroupSQL{saveErr: errors.New("save failed")})
err := store.AllowGroup("mnt/storage", idxPath("/tenant"), "newgroup")
if err == nil {
t.Fatal("expected SQL save error")
}
if groups := store.GetAllGroups(); len(groups) != 0 {
t.Fatalf("expected no in-memory group after failed ensure, got %#v", groups)
}
}
func TestSyncUserGroups_RollsBackOnPartialPersistFailure(t *testing.T) {
userStore := users.NewStorage(nil)
store := access.NewStorage(userStore)
store.Groups["groupA"] = access.StringSet{"alice": {}}
rec := &recordingGroupSQL{
failOn: 2,
groups: map[string]access.StringSet{
"groupA": {"alice": {}},
},
}
store.SetSQLStore(rec)
err := store.SyncUserGroups("alice", []string{"groupB"})
if err == nil {
t.Fatal("expected partial persist failure")
}
groups := store.GetUserGroups("alice")
if len(groups) != 1 || groups[0] != "groupA" {
t.Fatalf("expected in-memory rollback to groupA only, got %#v", groups)
}
if _, ok := store.Groups["groupB"]; ok {
t.Fatal("groupB should not remain in memory after rollback")
}
if rec.saveCalls != 3 {
t.Fatalf("saveCalls = %d, want 3 (groupA persist, groupB fail, groupA restore)", rec.saveCalls)
}
if _, ok := rec.groups["groupA"]["alice"]; !ok {
t.Fatalf("expected SQL rollback for groupA, got %#v", rec.groups["groupA"])
}
if _, ok := rec.groups["groupB"]; ok {
t.Fatalf("expected SQL groupB to be absent after rollback, got %#v", rec.groups["groupB"])
}
}
@@ -0,0 +1,79 @@
package access
import (
"fmt"
"strings"
"unicode"
"unicode/utf8"
)
// MaxGroupNameLength bounds names for manually created groups. Existing names
// (e.g. LDAP DNs synced from an IdP) may exceed it and are never rejected on read.
const MaxGroupNameLength = 128
// MinGroupNameLength bounds manually created group names. Existing records are
// never rejected so legacy single-character names stay usable.
const MinGroupNameLength = 2
// NormalizeGroupName trims and validates a group name supplied by an admin API.
// It is only used when manually creating or referencing a group; names that
// already exist in the database are always accepted on read, list, edit and
// delete paths so legacy records stay usable.
func NormalizeGroupName(name string) (string, error) {
name = strings.TrimSpace(name)
if name == "" {
return "", fmt.Errorf("group name is required")
}
if utf8.RuneCountInString(name) < MinGroupNameLength && !isSingleLetterName(name) {
return "", fmt.Errorf("group name must be at least %d characters", MinGroupNameLength)
}
if utf8.RuneCountInString(name) > MaxGroupNameLength {
return "", fmt.Errorf("group name exceeds %d characters", MaxGroupNameLength)
}
for _, r := range name {
if unicode.IsControl(r) {
return "", fmt.Errorf("group name must not contain control characters")
}
}
return name, nil
}
// isSingleLetterName reports whether name is one non-ASCII letter (e.g. a CJK ideograph),
// which is a meaningful group name on its own.
func isSingleLetterName(name string) bool {
r, size := utf8.DecodeRuneInString(name)
return size == len(name) && r > unicode.MaxASCII && unicode.IsLetter(r)
}
// NormalizeMembers trims, dedupes and drops empty usernames while preserving order.
func NormalizeMembers(usernames []string) []string {
seen := make(map[string]struct{}, len(usernames))
out := make([]string, 0, len(usernames))
for _, name := range usernames {
name = strings.TrimSpace(name)
if name == "" {
continue
}
if _, ok := seen[name]; ok {
continue
}
seen[name] = struct{}{}
out = append(out, name)
}
return out
}
// IsUsableSyncedGroupName reports whether a group name coming from an identity
// provider can be stored. Only the minimum needed to keep rows usable is
// rejected; everything else is kept so IdP naming conventions keep working.
func IsUsableSyncedGroupName(name string) bool {
if strings.TrimSpace(name) == "" {
return false
}
for _, r := range name {
if unicode.IsControl(r) {
return false
}
}
return true
}
@@ -0,0 +1,115 @@
package access_test
import (
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
)
func TestNormalizeGroupName(t *testing.T) {
for _, tc := range []struct {
name string
in string
want string
wantErr bool
}{
{"trims", " editors ", "editors", false},
{"spaces inside", "cn=admins with spaces,ou=x", "cn=admins with spaces,ou=x", false},
{"empty", "", "", true},
{"blank", " ", "", true},
{"newline", "a\nb", "", true},
{"too short", "a", "", true},
{"single wide char", "界", "界", false},
{"two chars", "ab", "ab", false},
{"single CJK char", "组", "组", false},
{"single non-letter non-ascii", "€", "", true},
{"too long", string(make([]byte, access.MaxGroupNameLength+1)), "", true},
} {
got, err := access.NormalizeGroupName(tc.in)
if tc.wantErr {
if err == nil {
t.Fatalf("%s: expected error, got %q", tc.name, got)
}
continue
}
if err != nil || got != tc.want {
t.Fatalf("%s: got %q, err=%v", tc.name, got, err)
}
}
// Long names of valid characters fail on length, not on content.
if _, err := access.NormalizeGroupName(string(make([]byte, access.MaxGroupNameLength+1))); err == nil {
t.Fatal("expected too-long name to fail")
}
if got := access.NormalizeMembers([]string{" a ", "b", "", "a", " "}); len(got) != 2 || got[0] != "a" || got[1] != "b" {
t.Fatalf("unexpected members: %v", got)
}
}
func TestRemoveUserFromAllGroups(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
if err := s.SetGroupMembers("editors", []string{"alice", "bob"}); err != nil {
t.Fatal(err)
}
if err := s.SetGroupMembers("staff", []string{"alice"}); err != nil {
t.Fatal(err)
}
if err := s.RemoveUserFromAllGroups("alice"); err != nil {
t.Fatal(err)
}
if groups := s.GetUserGroups("alice"); len(groups) != 0 {
t.Fatalf("alice should be in no groups, got %v", groups)
}
if got := s.GetGroupMembers()["editors"]; len(got) != 1 || got[0] != "bob" {
t.Fatalf("expected [bob], got %v", got)
}
// The emptied group row still exists.
if _, ok := s.GetGroupMembers()["staff"]; !ok {
t.Fatal("empty group should remain")
}
sqlGroups, err := sqlStore.GetAllGroups()
if err != nil {
t.Fatal(err)
}
if _, ok := sqlGroups["staff"]["alice"]; ok {
t.Fatal("alice should be removed from staff in SQL")
}
// Removing a user who is in no group is a no-op.
if err := s.RemoveUserFromAllGroups("nobody"); err != nil {
t.Fatal(err)
}
}
func TestRenameUserInGroups(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
if err := s.SetGroupMembers("editors", []string{"alice", "bob"}); err != nil {
t.Fatal(err)
}
if err := s.RenameUserInGroups("alice", "alicia"); err != nil {
t.Fatal(err)
}
got := s.GetGroupMembers()["editors"]
if len(got) != 2 || got[0] != "alicia" || got[1] != "bob" {
t.Fatalf("unexpected members: %v", got)
}
sqlGroups, err := sqlStore.GetAllGroups()
if err != nil {
t.Fatal(err)
}
if _, ok := sqlGroups["editors"]["alicia"]; !ok {
t.Fatal("alicia should be persisted in SQL")
}
if _, ok := sqlGroups["editors"]["alice"]; ok {
t.Fatal("alice should be removed in SQL")
}
// Renaming a non-member or same name is a no-op.
if err := s.RenameUserInGroups("nobody", "other"); err != nil {
t.Fatal(err)
}
if err := s.RenameUserInGroups("alicia", "alicia"); err != nil {
t.Fatal(err)
}
}
@@ -0,0 +1,12 @@
package access
import (
"os"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/testutil"
)
func TestMain(m *testing.M) {
os.Exit(testutil.Main(m))
}
+40 -18
View File
@@ -5,6 +5,7 @@ import (
"testing"
"time"
"github.com/golang-jwt/jwt/v4"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
)
@@ -15,7 +16,10 @@ func (failingRevokePersister) SaveAccessRule(string, string, *access.AccessRule)
func (failingRevokePersister) DeleteAccessRule(string, string) error { return nil }
func (failingRevokePersister) SaveGroup(string, access.StringSet) error { return nil }
func (failingRevokePersister) DeleteGroup(string) error { return nil }
func (failingRevokePersister) SaveRevokedToken(string, int64) error { return nil }
func (failingRevokePersister) DeleteGroupWithRules(string, []access.RuleUpsert, []access.RuleKey) error {
return nil
}
func (failingRevokePersister) SaveRevokedToken(string, int64) error { return nil }
func (failingRevokePersister) PersistImmediateTokenRevocation(string) error {
return errors.New("simulated revocation persistence failure")
}
@@ -23,7 +27,7 @@ func (failingRevokePersister) PersistTokenRetirement(string, int64, []string) er
return errors.New("simulated retirement persistence failure")
}
func (failingRevokePersister) DeleteRevokedToken(string) error { return nil }
func (failingRevokePersister) SaveHashedToken(string, uint64, bool) error { return nil }
func (failingRevokePersister) SaveHashedToken(string, uint64, bool, int64) error { return nil }
func (failingRevokePersister) DeleteHashedToken(string) error { return nil }
func (failingRevokePersister) DeleteHashedTokensByUserID(uint64) error { return nil }
@@ -109,25 +113,43 @@ func TestRevokeTokenIsImmediate(t *testing.T) {
}
}
func TestRetireTokenHonorsGraceWindow(t *testing.T) {
// signedTestJWT mints a bare HS256 JWT carrying only an exp claim. The registry
// only reads claims unverified, so any key works.
func signedTestJWT(t *testing.T, exp time.Time) string {
t.Helper()
token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{"exp": exp.Unix()})
tokenString, err := token.SignedString([]byte("test-key"))
if err != nil {
t.Fatal(err)
}
return tokenString
}
func TestExpiredTokenResolvesWithinGrace(t *testing.T) {
store, _ := createTestStorage(t)
if err := store.AddSessionToken("tok", 1); err != nil {
token := signedTestJWT(t, time.Now().Add(-time.Minute))
if err := store.AddSessionToken(token, 7); err != nil {
t.Fatal(err)
}
if err := store.RetireToken("tok"); err != nil {
t.Fatal(err)
info, ok := store.GetHashedTokenInfo(token)
if !ok || info.UserID != 7 || !info.IsSession {
t.Fatalf("recently expired token must still resolve within grace, got %+v ok=%v", info, ok)
}
if store.IsTokenRevoked("tok") {
t.Fatal("retired token must remain valid during the grace window")
}
if _, ok := store.GetHashedTokenInfo("tok"); !ok {
t.Fatal("retired token mapping must remain during the grace window")
}
// Simulate the grace window elapsing.
store.RevokedTokens[utils.HashSHA256("tok")] = time.Now().Add(-3 * time.Minute).Unix()
if !store.IsTokenRevoked("tok") {
t.Fatal("retired token must be revoked once the grace window elapses")
if info.ExpiresAt == 0 {
t.Fatal("expiry must be recorded from the token's exp claim")
}
}
func TestExpiredTokenPastGraceNeverRegistered(t *testing.T) {
store, _ := createTestStorage(t)
token := signedTestJWT(t, time.Now().Add(-access.BearerTokenGrace-time.Minute))
if err := store.AddSessionToken(token, 7); err != nil {
t.Fatal(err)
}
if _, ok := store.GetHashedTokenInfo(token); ok {
t.Fatal("token expired past grace must not resolve")
}
if _, ok := store.HashedTokens[utils.HashSHA256(token)]; ok {
t.Fatal("token expired past grace must not be registered")
}
}
@@ -0,0 +1,129 @@
package access_test
import (
"slices"
"testing"
)
func TestRenameUserInRules(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
src := "test_source"
if err := s.AllowUser(src, idxPath("/allowed"), "alice"); err != nil {
t.Fatal(err)
}
if err := s.DenyUser(src, idxPath("/denied"), "alice"); err != nil {
t.Fatal(err)
}
if err := s.AllowUser(src, idxPath("/both"), "alice"); err != nil {
t.Fatal(err)
}
if err := s.DenyUser(src, idxPath("/both"), "alice"); err != nil {
t.Fatal(err)
}
// New name already listed on another path (set semantics: old entry removed, new unchanged).
if err := s.AllowUser(src, idxPath("/existing"), "alicia"); err != nil {
t.Fatal(err)
}
if err := s.RenameUserInRules("alice", "alicia"); err != nil {
t.Fatal(err)
}
if rules := s.GetRulesForUser(src, "alice"); len(rules) != 0 {
t.Fatalf("expected no rules for old name alice, got %d", len(rules))
}
aliciaRules := s.GetRulesForUser(src, "alicia")
if len(aliciaRules) != 4 {
t.Fatalf("expected 4 paths for alicia, got %d", len(aliciaRules))
}
for _, path := range []string{"/allowed/", "/denied/", "/both/", "/existing/"} {
if _, ok := aliciaRules[path]; !ok {
t.Fatalf("missing rule at %q: %v", path, aliciaRules)
}
}
sqlRules, err := sqlStore.GetAllAccessRules()
if err != nil {
t.Fatal(err)
}
for _, rule := range sqlRules[src] {
if _, ok := rule.Allow.Users["alice"]; ok {
t.Fatal("alice should not appear in allow lists in SQL")
}
if _, ok := rule.Deny.Users["alice"]; ok {
t.Fatal("alice should not appear in deny lists in SQL")
}
}
if err := s.RenameUserInRules("nobody", "other"); err != nil {
t.Fatal(err)
}
if err := s.RenameUserInRules("alicia", "alicia"); err != nil {
t.Fatal(err)
}
}
func TestRemoveAllRulesForUserOnDelete(t *testing.T) {
setupTestSources()
s, _, sqlStore := createTestStorageWithSQL(t)
src := "test_source"
if err := s.AllowUser(src, idxPath("/a"), "bob"); err != nil {
t.Fatal(err)
}
if err := s.DenyUser(src, idxPath("/b"), "bob"); err != nil {
t.Fatal(err)
}
if err := s.RemoveAllRulesForUser("bob"); err != nil {
t.Fatal(err)
}
if rules := s.GetRulesForUser(src, "bob"); len(rules) != 0 {
t.Fatalf("expected no in-memory rules for bob, got %d", len(rules))
}
all, err := s.GetAllRules(src)
if err != nil {
t.Fatal(err)
}
if len(all) != 0 {
t.Fatalf("expected no rules left on source, got %d", len(all))
}
sqlRules, err := sqlStore.GetAllAccessRules()
if err != nil {
t.Fatal(err)
}
if len(sqlRules[src]) != 0 {
t.Fatalf("expected no SQL rules for source, got %v", sqlRules[src])
}
}
func TestRenameUserInRulesBothAllowAndDenySamePath(t *testing.T) {
setupTestSources()
s, _, _ := createTestStorageWithSQL(t)
src := "test_source"
if err := s.AllowUser(src, idxPath("/shared"), "old"); err != nil {
t.Fatal(err)
}
if err := s.DenyUser(src, idxPath("/shared"), "old"); err != nil {
t.Fatal(err)
}
if err := s.RenameUserInRules("old", "new"); err != nil {
t.Fatal(err)
}
rules, ok := s.GetFrontendRules(src, idxPath("/shared"))
if !ok {
t.Fatal("expected rule at /shared")
}
if !slices.Contains(rules.Allow.Users, "new") || slices.Contains(rules.Allow.Users, "old") {
t.Fatalf("allow users: %v", rules.Allow.Users)
}
if !slices.Contains(rules.Deny.Users, "new") || slices.Contains(rules.Deny.Users, "old") {
t.Fatalf("deny users: %v", rules.Deny.Users)
}
}
@@ -24,6 +24,9 @@ const (
EventAccessUpdate EventType = "accessUpdate"
EventAccessCreate EventType = "accessCreate"
EventAccessDelete EventType = "accessDelete"
EventQuotaCreate EventType = "quotaCreate"
EventQuotaUpdate EventType = "quotaUpdate"
EventQuotaDelete EventType = "quotaDelete"
EventLogin EventType = "login"
EventLogout EventType = "logout"
EventSignup EventType = "signup"
@@ -54,6 +57,9 @@ var AllEventTypes = []EventType{
EventAccessUpdate,
EventAccessCreate,
EventAccessDelete,
EventQuotaCreate,
EventQuotaUpdate,
EventQuotaDelete,
EventLogin,
EventLogout,
EventSignup,
@@ -84,6 +90,13 @@ var AccessEventTypes = []EventType{
EventAccessDelete,
}
// QuotaEventTypes are folder quota mutations (scope=quotas).
var QuotaEventTypes = []EventType{
EventQuotaCreate,
EventQuotaUpdate,
EventQuotaDelete,
}
// ShareEventTypes are share lifecycle events (scope=shares). Share downloads are
// EventDownload rows with a non-empty details.shareHash.
var ShareEventTypes = []EventType{
@@ -108,13 +121,15 @@ func ResolveScopeEventTypes(scope string, explicit []EventType) ([]EventType, er
allowed = FileEventTypes
case "access":
allowed = AccessEventTypes
case "quotas":
allowed = QuotaEventTypes
case "shares":
if len(explicit) == 0 {
return nil, nil
}
allowed = ShareScopeEventTypes
default:
return nil, fmt.Errorf("scope must be all, files, access, or shares")
return nil, fmt.Errorf("scope must be all, files, access, quotas, or shares")
}
if len(explicit) == 0 {
return allowed, nil
@@ -144,6 +159,7 @@ func (e EventType) Valid() bool {
EventArchive, EventUnarchive,
EventShareCreate, EventShareUpdate, EventShareDelete,
EventUserCreate, EventUserUpdate, EventUserDelete, EventAccessUpdate, EventAccessCreate, EventAccessDelete,
EventQuotaCreate, EventQuotaUpdate, EventQuotaDelete,
EventLogin, EventLogout, EventSignup,
EventPasskeyRegister, EventPasskeyDelete,
EventTokenCreate, EventTokenDelete,
@@ -44,6 +44,18 @@ func TestResolveScopeEventTypes(t *testing.T) {
t.Fatalf("access default missing %q in %v", et, got)
}
}
got, err = ResolveScopeEventTypes("quotas", nil)
if err != nil {
t.Fatal(err)
}
if len(got) != len(QuotaEventTypes) {
t.Fatalf("quotas default: got %d types, want %d", len(got), len(QuotaEventTypes))
}
for _, et := range QuotaEventTypes {
if !containsEventType(got, et) {
t.Fatalf("quotas default missing %q in %v", et, got)
}
}
}
func containsEventType(list []EventType, target EventType) bool {
@@ -21,6 +21,9 @@ type IndexInfo struct {
UsedAsIndexed uint64 `json:"used"`
UsedDisk uint64 `json:"usedAlt"`
DiskTotal uint64 `json:"total"`
// Root-filesystem-only view (no nested mount aggregation)
UsedDiskRoot uint64 `json:"usedAltRoot"`
DiskTotalRoot uint64 `json:"totalRoot"`
// Scanner information - map of scanner path to scanner stats
Scanners map[string]*PersistedScannerInfo `json:"scanners"`
+91
View File
@@ -0,0 +1,91 @@
package quota
import (
"fmt"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
)
// MeasurementStatusReady means enforcement uses the configured meter.
const MeasurementStatusReady = "ready"
// MeasurementStatusAccountedFallback means index meter configured but accounted counter used.
const MeasurementStatusAccountedFallback = "accounted_fallback"
// IsIndexMeter reports whether meter counts usage from the search index.
func IsIndexMeter(meter string) bool {
return meter == MeterIndexSize || meter == MeterIndexScope
}
// IndexingEnabled reports whether folder sizes are available from the index on this source.
func IndexingEnabled(sourceName string) bool {
idx := indexing.GetIndex(sourceName)
if idx == nil {
return false
}
return !idx.Config.ResolvedRules.IndexingDisabled
}
// IndexSizeAvailable reports whether GetFolderSize succeeds for the quota root path.
func IndexSizeAvailable(sourceName, indexPath string) bool {
idx := indexing.GetIndex(sourceName)
if idx == nil || idx.Config.ResolvedRules.IndexingDisabled {
return false
}
path := normalizeMeterPath(indexPath)
_, ok := idx.GetFolderSizeForIndexPath(path)
return ok
}
// EffectiveMeter resolves runtime meter: accounted when indexing off, configured index when size known, else accounted fallback.
func EffectiveMeter(configuredMeter, sourceName, quotaRootPath string) string {
if configuredMeter == MeterAccounted || !IsIndexMeter(configuredMeter) {
return MeterAccounted
}
if !IndexingEnabled(sourceName) {
return MeterAccounted
}
if IndexSizeAvailable(sourceName, quotaRootPath) {
return configuredMeter
}
return MeterAccounted
}
// MeasurementStatus compares configured vs effective meter for API snapshots.
func MeasurementStatus(configuredMeter, effectiveMeter string) string {
if IsIndexMeter(configuredMeter) && effectiveMeter == MeterAccounted {
return MeasurementStatusAccountedFallback
}
return MeasurementStatusReady
}
// ValidateConfiguredMeter rejects index meters when indexing is disabled on the source.
func ValidateConfiguredMeter(sourceName, meter string) error {
meter = strings.TrimSpace(meter)
if meter == "" {
return nil
}
if meter == MeterAccounted {
return nil
}
if IsIndexMeter(meter) {
if !IndexingEnabled(sourceName) {
return fmt.Errorf("index meter not allowed when indexing is disabled on source %q", sourceName)
}
return nil
}
return fmt.Errorf("invalid quota meter %q", meter)
}
func normalizeMeterPath(path string) string {
path = strings.TrimSpace(path)
path = strings.TrimSuffix(path, "/")
if path == "" {
return "/"
}
if !strings.HasPrefix(path, "/") {
path = "/" + path
}
return path
}
@@ -0,0 +1,116 @@
package quota_test
import (
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/quota"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func setupMeterTestSource(t *testing.T, name, path string, indexingDisabled bool) {
t.Helper()
prevSourceMap := settings.Config.Server.SourceMap
prevNameToSource := settings.Config.Server.NameToSource
t.Cleanup(func() {
indexing.ClearTestIndices()
settings.Config.Server.SourceMap = prevSourceMap
settings.Config.Server.NameToSource = prevNameToSource
})
indexing.SetTestIndex(name, path)
idx := indexing.GetIndex(name)
if idx == nil {
t.Fatal("expected test index")
}
idx.Config.ResolvedRules.IndexingDisabled = indexingDisabled
settings.Config.Server.SourceMap = map[string]*settings.Source{
path: {Path: path, Name: name},
}
settings.Config.Server.NameToSource = map[string]*settings.Source{
name: settings.Config.Server.SourceMap[path],
}
}
func TestValidateConfiguredMeter_IndexingDisabled(t *testing.T) {
root := t.TempDir()
name := "meter-src"
setupMeterTestSource(t, name, root, true)
if err := quota.ValidateConfiguredMeter(name, quota.MeterIndexSize); err == nil {
t.Fatal("expected error for index_size when indexing disabled")
}
if err := quota.ValidateConfiguredMeter(name, quota.MeterIndexScope); err == nil {
t.Fatal("expected error for index_scope when indexing disabled")
}
if err := quota.ValidateConfiguredMeter(name, quota.MeterAccounted); err != nil {
t.Fatalf("accounted should be allowed: %v", err)
}
}
func TestEffectiveMeter_IndexingDisabledForcesAccounted(t *testing.T) {
root := t.TempDir()
name := "meter-src2"
setupMeterTestSource(t, name, root, true)
got := quota.EffectiveMeter(quota.MeterIndexSize, name, "/")
if got != quota.MeterAccounted {
t.Fatalf("expected accounted, got %q", got)
}
}
func TestEffectiveMeter_ExplicitAccountedWhenIndexOn(t *testing.T) {
root := t.TempDir()
name := "meter-src3"
setupMeterTestSource(t, name, root, false)
idx := indexing.GetIndex(name)
idx.SetFolderSize("/", 1024)
got := quota.EffectiveMeter(quota.MeterAccounted, name, "/")
if got != quota.MeterAccounted {
t.Fatalf("expected accounted, got %q", got)
}
}
func TestEffectiveMeter_IndexWhenSizeAvailable(t *testing.T) {
root := t.TempDir()
name := "meter-src4"
setupMeterTestSource(t, name, root, false)
idx := indexing.GetIndex(name)
idx.SetFolderSize("/", 2048)
got := quota.EffectiveMeter(quota.MeterIndexSize, name, "/")
if got != quota.MeterIndexSize {
t.Fatalf("expected index_size, got %q", got)
}
status := quota.MeasurementStatus(quota.MeterIndexSize, got)
if status != quota.MeasurementStatusReady {
t.Fatalf("expected ready, got %q", status)
}
}
func TestEffectiveMeter_IndexPathTrailingSlash(t *testing.T) {
root := t.TempDir()
name := "meter-src6"
setupMeterTestSource(t, name, root, false)
idx := indexing.GetIndex(name)
idx.SetFolderSize("/projects/", 4096)
got := quota.EffectiveMeter(quota.MeterIndexSize, name, "/projects")
if got != quota.MeterIndexSize {
t.Fatalf("expected index_size for path without trailing slash, got %q", got)
}
}
func TestEffectiveMeter_RuntimeFallback(t *testing.T) {
root := t.TempDir()
name := "meter-src5"
setupMeterTestSource(t, name, root, false)
got := quota.EffectiveMeter(quota.MeterIndexScope, name, "/missing")
if got != quota.MeterAccounted {
t.Fatalf("expected accounted fallback, got %q", got)
}
status := quota.MeasurementStatus(quota.MeterIndexScope, got)
if status != quota.MeasurementStatusAccountedFallback {
t.Fatalf("expected accounted_fallback, got %q", status)
}
}
+43
View File
@@ -0,0 +1,43 @@
package quota
// Meter names for scope and folder quotas.
const (
MeterIndexScope = "index_scope"
MeterIndexSize = "index_size"
MeterAccounted = "accounted"
)
// FolderQuota is a path-bound storage cap with configurable meter.
type FolderQuota struct {
ID string `json:"id"`
Source string `json:"source"` // backend source path
Path string `json:"path"` // index path within source
UserID uint64 `json:"userId,omitempty"` // 0 = all users
LimitBytes int64 `json:"limitBytes"`
Meter string `json:"meter,omitempty"` // index_size | index_scope (indexed size) | accounted (tracked usage)
}
// Counter holds accounted usage for a quota_id.
type Counter struct {
QuotaID string `json:"quotaId"`
UsedBytes int64 `json:"usedBytes"`
ReservedBytes int64 `json:"reservedBytes"`
}
// Snapshot is a read model for API and admission checks.
type Snapshot struct {
QuotaID string `json:"quotaId"`
Kind string `json:"quotaKind"` // folder, scope, share
LimitBytes int64 `json:"limitBytes"`
UsedBytes int64 `json:"usedBytes"`
ReservedBytes int64 `json:"reservedBytes"`
Meter string `json:"meter,omitempty"` // effective meter (backward compatible)
ConfiguredMeter string `json:"configuredMeter,omitempty"`
EffectiveMeter string `json:"effectiveMeter,omitempty"`
MeasurementStatus string `json:"measurementStatus,omitempty"` // ready, accounted_fallback
}
// ShareQuotaID returns the counter key for a share quota.
func ShareQuotaID(hash string) string {
return "share:" + hash
}
+19 -2
View File
@@ -48,6 +48,7 @@ type ShareLimits struct {
PerUserDownloadLimit bool `json:"perUserDownloadLimit,omitempty"`
ExtractEmbeddedSubtitles bool `json:"extractEmbeddedSubtitles,omitempty"`
DownloadsLimit int `json:"downloadsLimit,omitempty"`
QuotaLimitBytes int64 `json:"quotaLimitBytes,omitempty"`
HideFileExt string `json:"hideFileExt,omitempty"` // show hidden files based on extensions in shares
Banner string `json:"banner,omitempty"`
SourceName string `json:"source,omitempty"` // source display name for API; backend path is Share.SourcePath
@@ -85,8 +86,11 @@ type ShareEditable struct {
type ShareFrontend struct {
ShareEditable
ShareColumns
Username string `json:"username,omitempty"`
PathExists bool `json:"pathExists,omitempty"`
Username string `json:"username,omitempty"`
PathExists bool `json:"pathExists,omitempty"`
QuotaUsedBytes int64 `json:"quotaUsedBytes,omitempty"`
QuotaReservedBytes int64 `json:"quotaReservedBytes,omitempty"`
QuotaAvailableBytes int64 `json:"quotaAvailableBytes,omitempty"`
}
// SharePostBody is POST /api/share JSON. Plaintext password is hashed to Share.PasswordHash before persist.
@@ -98,6 +102,19 @@ type SharePostBody struct {
Path string `json:"path,omitempty"`
}
// EditableFromShare returns the client-editable subset of a persisted share.
// ShareExpiryInput is omitted because expiration is stored in ShareColumns.Expire;
// callers merging partial updates must preserve Expire when expires/unit are absent.
func EditableFromShare(s *Share) ShareEditable {
if s == nil {
return ShareEditable{}
}
return ShareEditable{
FrontendShareInfo: s.FrontendShareInfo,
ShareLimits: s.ShareLimits,
}
}
// ApplyPostBodyUpdate copies client-editable fields onto link.
// Caller must preserve path, sourcePath, pinnedItems, version, download counters, and secrets.
func ApplyPostBodyUpdate(link *Share, req *SharePostBody, expire int64) {
@@ -0,0 +1,200 @@
package sql
import (
"testing"
"time"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
)
// TestCreatedTime_InsertAndReadBack checks that an inserted Created value is returned unchanged.
func TestCreatedTime_InsertAndReadBack(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(""))
defer pop()
db, _, err := NewIndexDB("created_time_set", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db.Close()
created := time.Unix(1700000000, 0)
info := &iteminfo.FileInfo{
Path: "/a.txt",
ItemInfo: iteminfo.ItemInfo{
Name: "a.txt",
Size: 12,
Created: &created,
ModTime: time.Unix(1, 0),
Type: "text/plain",
},
}
if insertErr := db.InsertItem("src1", "/a.txt", info); insertErr != nil {
t.Fatal(insertErr)
}
got, err := db.GetItem("src1", "/a.txt")
if err != nil {
t.Fatal(err)
}
if got.Created == nil {
t.Fatal("expected non-nil Created")
}
if !got.Created.Equal(created.Truncate(time.Second)) {
t.Errorf("Created = %v, want %v", got.Created, created.Truncate(time.Second))
}
}
// TestCreatedTime_NilStaysNil checks that an item inserted without Created reads back with nil.
func TestCreatedTime_NilStaysNil(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(""))
defer pop()
db, _, err := NewIndexDB("created_time_nil", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db.Close()
info := &iteminfo.FileInfo{
Path: "/b.txt",
ItemInfo: iteminfo.ItemInfo{
Name: "b.txt",
Size: 12,
ModTime: time.Unix(1, 0),
Type: "text/plain",
},
}
if insertErr := db.InsertItem("src1", "/b.txt", info); insertErr != nil {
t.Fatal(insertErr)
}
got, err := db.GetItem("src1", "/b.txt")
if err != nil {
t.Fatal(err)
}
if got.Created != nil {
t.Errorf("expected nil Created, got %v", got.Created)
}
}
// TestCreatedTime_UpsertSetsCreated checks that an upsert fills in a previously nil Created.
func TestCreatedTime_UpsertSetsCreated(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(""))
defer pop()
db, _, err := NewIndexDB("created_time_upsert", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db.Close()
info := &iteminfo.FileInfo{
Path: "/c.txt",
ItemInfo: iteminfo.ItemInfo{
Name: "c.txt",
Size: 12,
ModTime: time.Unix(1, 0),
Type: "text/plain",
},
}
if insertErr := db.InsertItem("src1", "/c.txt", info); insertErr != nil {
t.Fatal(insertErr)
}
created := time.Unix(1700000000, 0)
info.Created = &created
if updateErr := db.InsertItem("src1", "/c.txt", info); updateErr != nil {
t.Fatal(updateErr)
}
got, err := db.GetItem("src1", "/c.txt")
if err != nil {
t.Fatal(err)
}
if got.Created == nil {
t.Fatal("expected non-nil Created after upsert")
}
if !got.Created.Equal(created.Truncate(time.Second)) {
t.Errorf("Created = %v, want %v", got.Created, created.Truncate(time.Second))
}
}
// TestCreatedTime_MigrationAddsColumn checks that CreateIndexTable adds created_time to an
// old-schema table and can be run repeatedly.
func TestCreatedTime_MigrationAddsColumn(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(""))
defer pop()
db, _, err := NewIndexDB("created_time_migration", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db.Close()
if _, err := db.Exec(`DROP TABLE index_items`); err != nil {
t.Fatal(err)
}
oldSchema := `
CREATE TABLE index_items (
source TEXT NOT NULL,
path TEXT NOT NULL,
parent_path TEXT NOT NULL,
name TEXT NOT NULL,
size INTEGER NOT NULL,
mod_time INTEGER NOT NULL,
type TEXT NOT NULL,
is_dir BOOLEAN NOT NULL,
is_hidden BOOLEAN NOT NULL,
has_preview BOOLEAN NOT NULL,
last_updated INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (source, path)
)`
if _, err := db.Exec(oldSchema); err != nil {
t.Fatal(err)
}
if err := db.CreateIndexTable(); err != nil {
t.Fatal(err)
}
if !hasCreatedTimeColumn(t, db) {
t.Fatal("expected created_time column after migration")
}
// Running again must not error (repeatable migration).
if err := db.CreateIndexTable(); err != nil {
t.Fatal(err)
}
if !hasCreatedTimeColumn(t, db) {
t.Fatal("expected created_time column after second CreateIndexTable call")
}
}
// hasCreatedTimeColumn reports whether index_items currently has a created_time column.
func hasCreatedTimeColumn(t *testing.T, db *IndexDB) bool {
t.Helper()
rows, err := db.Query(`PRAGMA table_info(index_items)`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var cid, notNull, pk int
var name, columnType string
var defaultValue interface{}
if err := rows.Scan(&cid, &name, &columnType, &notNull, &defaultValue, &pk); err != nil {
t.Fatal(err)
}
if name == "created_time" {
return true
}
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
return false
}
+115 -106
View File
@@ -9,6 +9,7 @@ import (
"strings"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/sqlitebusy"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/go-logger/logger"
@@ -244,6 +245,7 @@ func (db *IndexDB) CreateIndexTable() error {
parent_path TEXT NOT NULL,
name TEXT NOT NULL,
size INTEGER NOT NULL,
created_time INTEGER NOT NULL DEFAULT 0,
mod_time INTEGER NOT NULL,
type TEXT NOT NULL,
is_dir BOOLEAN NOT NULL,
@@ -258,17 +260,63 @@ func (db *IndexDB) CreateIndexTable() error {
CREATE INDEX IF NOT EXISTS idx_last_updated ON index_items(source, last_updated);
`
_, err := db.Exec(query)
if err != nil {
return err
}
return db.ensureCreatedTimeColumn()
}
// ensureCreatedTimeColumn adds the created_time column to index_items when it is
// missing, so databases created before the column existed are upgraded in place.
func (db *IndexDB) ensureCreatedTimeColumn() error {
rows, err := db.Query(`PRAGMA table_info(index_items)`)
if err != nil {
return err
}
found := false
for rows.Next() {
var cid, notNull, pk int
var name, columnType string
var defaultValue interface{}
scanErr := rows.Scan(&cid, &name, &columnType, &notNull, &defaultValue, &pk)
if scanErr != nil {
rows.Close()
return scanErr
}
if name == "created_time" {
found = true
}
}
if rowsErr := rows.Err(); rowsErr != nil {
rows.Close()
return rowsErr
}
rows.Close()
if found {
return nil
}
_, err = db.Exec(`ALTER TABLE index_items ADD COLUMN created_time INTEGER NOT NULL DEFAULT 0`)
return err
}
// createdUnix converts a creation time to unix seconds for storage, returning 0
// when the time is nil or zero (stored as "unavailable").
func createdUnix(created *time.Time) int64 {
if created == nil || created.IsZero() {
return 0
}
return created.Unix()
}
func (db *IndexDB) InsertItem(source, path string, info *iteminfo.FileInfo) error {
query := `
INSERT INTO index_items (source, path, parent_path, name, size, mod_time, type, is_dir, is_hidden, has_preview, last_updated)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
INSERT INTO index_items (source, path, parent_path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview, last_updated)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(source, path) DO UPDATE SET
parent_path = excluded.parent_path,
name = excluded.name,
size = excluded.size,
created_time = excluded.created_time,
mod_time = excluded.mod_time,
type = excluded.type,
is_dir = excluded.is_dir,
@@ -283,6 +331,7 @@ func (db *IndexDB) InsertItem(source, path string, info *iteminfo.FileInfo) erro
parentPath,
info.Name,
info.Size,
createdUnix(info.Created),
info.ModTime.Unix(),
info.Type,
info.Type == "directory",
@@ -291,11 +340,12 @@ func (db *IndexDB) InsertItem(source, path string, info *iteminfo.FileInfo) erro
time.Now().Unix(),
)
if err != nil {
if !isBusyError(err) && !isTransactionError(err) {
if !sqlitebusy.IsBusyOrLocked(err) {
logger.Errorf("InsertItem failed for source=%s path=%s: %v", source, path, err)
}
return sqlitebusy.Wrap(err)
}
return err
return nil
}
// dbTransientOpAttempts is how many times to run a bulk DB op that may hit SQLITE_BUSY before giving up.
@@ -304,9 +354,7 @@ const dbTransientOpAttempts = 2
const dbTransientRetryBackoff = 50 * time.Millisecond
// BulkInsertItems inserts multiple items in a single transaction for a specific source.
// Database errors (busy/locked) are treated as soft failures - the filesystem is the source of truth.
// Returns nil on success or soft failure (busy/locked), error only on hard failures.
// Transient busy/locked errors retry once before giving up (see dbTransientOpAttempts).
// Transient busy/locked errors retry before giving up (see dbTransientOpAttempts); persistent busy returns an error.
func (db *IndexDB) BulkInsertItems(source string, items []*iteminfo.FileInfo) error {
if len(items) == 0 {
return nil
@@ -332,8 +380,10 @@ func (db *IndexDB) BulkInsertItems(source string, items []*iteminfo.FileInfo) er
continue
}
if transient {
logger.Warningf("[DB_TX] BulkInsertItems: still failing after %d attempts (%d items, %v) — batch dropped",
logger.Warningf("[DB_TX] BulkInsertItems: still failing after %d attempts (%d items, %v)",
dbTransientOpAttempts, len(items), time.Since(startTime))
return fmt.Errorf("bulk insert failed after %d attempts (%d items): %w",
dbTransientOpAttempts, len(items), sqlitebusy.ErrBusy)
}
return nil
}
@@ -345,7 +395,7 @@ func (db *IndexDB) BulkInsertItems(source string, items []*iteminfo.FileInfo) er
func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo) (success bool, transient bool, err error) {
tx, err := db.BeginTransaction()
if err != nil {
if isBusyError(err) || isTransactionError(err) {
if sqlitebusy.IsBusyOrLocked(err) {
db.mu.Unlock()
logger.Debugf("[DB_TX] BulkInsertItems: BeginTransaction failed (DB busy/locked): %v", err)
return false, true, nil
@@ -363,12 +413,13 @@ func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo)
}()
stmt, err := tx.Prepare(`
INSERT INTO index_items (source, path, parent_path, name, size, mod_time, type, is_dir, is_hidden, has_preview, last_updated)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
INSERT INTO index_items (source, path, parent_path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview, last_updated)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(source, path) DO UPDATE SET
parent_path = excluded.parent_path,
name = excluded.name,
size = excluded.size,
created_time = excluded.created_time,
mod_time = excluded.mod_time,
type = excluded.type,
is_dir = excluded.is_dir,
@@ -377,7 +428,7 @@ func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo)
last_updated = excluded.last_updated
`)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_TX] BulkInsertItems: Prepare busy/locked: %v", err)
return false, true, nil
}
@@ -394,6 +445,7 @@ func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo)
parentPath,
info.Name,
info.Size,
createdUnix(info.Created),
info.ModTime.Unix(),
info.Type,
info.Type == "directory",
@@ -402,7 +454,7 @@ func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo)
nowUnix,
)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_TX] BulkInsertItems: Exec busy/locked during insert: %v", err)
return false, true, nil
}
@@ -412,7 +464,7 @@ func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo)
}
if err := tx.Commit(); err != nil {
if isBusyError(err) || isTransactionError(err) {
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_TX] BulkInsertItems: Commit busy/locked: %v", err)
return false, true, nil
}
@@ -422,51 +474,23 @@ func (db *IndexDB) attemptBulkInsert(source string, items []*iteminfo.FileInfo)
return true, false, nil
}
// isBusyError checks if an error is SQLITE_BUSY (error code 5)
func isBusyError(err error) bool {
if err == nil {
return false
}
errStr := err.Error()
// SQLITE_BUSY is error code 5, and modernc.org/sqlite returns it as "database is locked (5)"
return strings.Contains(errStr, "database is locked") || strings.Contains(errStr, "SQLITE_BUSY") || strings.Contains(errStr, "(5)")
}
// isTransactionError checks if an error is related to nested transactions (error code 1)
func isTransactionError(err error) bool {
if err == nil {
return false
}
errStr := err.Error()
// SQLITE_ERROR for transaction issues is error code 1
return strings.Contains(errStr, "cannot start a transaction within a transaction") ||
strings.Contains(errStr, "cannot commit") ||
strings.Contains(errStr, "(1)")
}
// GetItem retrieves a single item by path for a specific source.
// Returns nil on database busy/lock errors (non-fatal).
// Busy/locked errors are returned so callers can fall back to the filesystem.
func (db *IndexDB) GetItem(source, path string) (*iteminfo.FileInfo, error) {
query := `
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items WHERE source = ? AND path = ?
`
row := db.QueryRow(query, source, path)
item, err := scanItem(row)
if err != nil {
// Soft failure: DB is busy or locked, return nil
// Caller will handle missing data by fetching from filesystem
if isBusyError(err) || isTransactionError(err) {
return nil, nil
}
return nil, err
return nil, sqlitebusy.Wrap(err)
}
return item, nil
}
// GetItemsByPaths retrieves multiple items by their paths in a single query for a specific source.
// This is more efficient than calling GetItem multiple times.
// Returns empty map on database busy/lock errors (non-fatal).
func (db *IndexDB) GetItemsByPaths(source string, paths []string) (map[string]*iteminfo.FileInfo, error) {
if len(paths) == 0 {
return make(map[string]*iteminfo.FileInfo), nil
@@ -482,18 +506,13 @@ func (db *IndexDB) GetItemsByPaths(source string, paths []string) (map[string]*i
}
query := fmt.Sprintf(`
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items WHERE source = ? AND path IN (%s)
`, strings.Join(placeholders, ","))
rows, err := db.Query(query, args...)
if err != nil {
// Soft failure: DB is busy or locked, return empty map
// Caller will handle missing data (e.g., skip size updates, fetch from filesystem)
if isBusyError(err) || isTransactionError(err) {
return make(map[string]*iteminfo.FileInfo), nil
}
return nil, err
return nil, sqlitebusy.Wrap(err)
}
defer rows.Close()
@@ -512,16 +531,16 @@ func (db *IndexDB) GetItemsByPaths(source string, paths []string) (map[string]*i
// GetDirectoryChildren retrieves all children of a directory for a specific source.
func (db *IndexDB) GetDirectoryChildren(source, dirPath string) ([]*iteminfo.FileInfo, error) {
query := `
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items WHERE source = ? AND parent_path = ?
ORDER BY is_dir DESC, name ASC
`
rows, err := db.Query(query, source, dirPath)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
logger.Warningf("[DB_TX] GetDirectoryChildren: DB busy/locked, skipping query")
return []*iteminfo.FileInfo{}, nil
if sqlitebusy.IsBusyOrLocked(err) {
logger.Warningf("[DB_TX] GetDirectoryChildren: DB busy/locked: %v", err)
return nil, sqlitebusy.Wrap(err)
}
logger.Errorf("GetDirectoryChildren: Query failed for source=%s, parent_path=%s, error=%v", source, dirPath, err)
return nil, err
@@ -671,9 +690,9 @@ func (db *IndexDB) DeleteStaleFolders(source string, pathPrefix string, scanStar
}
if err != nil {
if isBusyError(err) || isTransactionError(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleEntriesQuick Phase 1: DB busy, skipping cleanup")
return totalDeleted, nil
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleEntriesQuick Phase 1: DB busy: %v", err)
return totalDeleted, sqlitebusy.Wrap(err)
}
return totalDeleted, err
}
@@ -738,9 +757,9 @@ func (db *IndexDB) DeleteStaleFilesInDirs(source string, updatedDirs []string, s
result, err := db.Exec(query, queryArgs...)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleFilesInDirs: DB busy, skipping batch")
break // Skip this batch and continue with next
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleFilesInDirs: DB busy: %v", err)
return totalDeleted, sqlitebusy.Wrap(err)
}
return totalDeleted, err
}
@@ -802,9 +821,9 @@ func (db *IndexDB) DeleteStaleEntries(source string, pathPrefix string, scanStar
}
if err != nil {
if isBusyError(err) || isTransactionError(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleEntries: DB busy, skipping cleanup")
return totalDeleted, nil
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleEntries: DB busy: %v", err)
return totalDeleted, sqlitebusy.Wrap(err)
}
return totalDeleted, err
}
@@ -832,9 +851,9 @@ func (db *IndexDB) DeleteStaleItemsOlderThan(olderThan time.Duration) (int, erro
result, err := db.Exec(query, cutoffTime)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleItemsOlderThan: DB busy, skipping cleanup")
return 0, nil
if sqlitebusy.IsBusyOrLocked(err) {
logger.Debugf("[DB_MAINTENANCE] DeleteStaleItemsOlderThan: DB busy: %v", err)
return 0, sqlitebusy.Wrap(err)
}
return 0, err
}
@@ -859,8 +878,8 @@ func (db *IndexDB) GetRecursiveCount(source string, pathPrefix string) (dirs uin
var dirCount, fileCount sql.NullInt64
err = db.QueryRow(query, source, pathPrefix, nextPrefix).Scan(&dirCount, &fileCount)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
return 0, 0, nil
if sqlitebusy.IsBusyOrLocked(err) {
return 0, 0, sqlitebusy.Wrap(err)
}
return 0, 0, err
}
@@ -883,8 +902,8 @@ func (db *IndexDB) GetDirectFileCount(source string, pathPrefix string) (files u
var fileCount sql.NullInt64
err = db.QueryRow(query, source, pathPrefix).Scan(&fileCount)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
return 0, nil
if sqlitebusy.IsBusyOrLocked(err) {
return 0, sqlitebusy.Wrap(err)
}
return 0, err
}
@@ -920,9 +939,8 @@ func (db *IndexDB) GetTypeGroupsForSize(source string, size int64, pathPrefix st
rows, err := db.Query(query, args...)
if err != nil {
// Soft failure: DB is busy or locked, return empty results
if isBusyError(err) || isTransactionError(err) {
return []string{}, nil
if sqlitebusy.IsBusyOrLocked(err) {
return nil, sqlitebusy.Wrap(err)
}
return nil, err
}
@@ -959,7 +977,7 @@ func (db *IndexDB) GetFilesForMultipleSizes(source string, sizes []int64, pathPr
if pathPrefix != "" {
nextPrefix := getNextPathPrefix(pathPrefix)
query = fmt.Sprintf(`
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items
WHERE source = ? AND size IN (%s) AND is_dir = 0 AND path >= ? AND path < ?
ORDER BY size, type, name
@@ -967,7 +985,7 @@ func (db *IndexDB) GetFilesForMultipleSizes(source string, sizes []int64, pathPr
args = append(args, pathPrefix, nextPrefix)
} else {
query = fmt.Sprintf(`
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items
WHERE source = ? AND size IN (%s) AND is_dir = 0
ORDER BY size, type, name
@@ -976,9 +994,8 @@ func (db *IndexDB) GetFilesForMultipleSizes(source string, sizes []int64, pathPr
rows, err := db.Query(query, args...)
if err != nil {
// Soft failure: DB is busy or locked, return empty results
if isBusyError(err) || isTransactionError(err) {
return make(map[int64][]*iteminfo.FileInfo), nil
if sqlitebusy.IsBusyOrLocked(err) {
return nil, sqlitebusy.Wrap(err)
}
return nil, err
}
@@ -1008,10 +1025,7 @@ func (db *IndexDB) GetAllDirectories(source string) ([]string, error) {
rows, err := db.Query(query, source)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
return []string{}, nil
}
return nil, err
return nil, sqlitebusy.Wrap(err)
}
defer rows.Close()
@@ -1071,16 +1085,17 @@ func (db *IndexDB) UpdateFolderSizesIfChanged(source string, pathSizes map[strin
for attempt := 1; attempt <= dbTransientOpAttempts; attempt++ {
result, err := db.Exec(query, args...)
if err != nil {
if isBusyError(err) || isTransactionError(err) {
if sqlitebusy.IsBusyOrLocked(err) {
if attempt < dbTransientOpAttempts {
logger.Warningf("[DB_TX] UpdateFolderSizesIfChanged: busy/locked (attempt %d/%d), retrying after %v: %v",
attempt, dbTransientOpAttempts, dbTransientRetryBackoff, err)
time.Sleep(dbTransientRetryBackoff)
continue
}
logger.Warningf("[DB_TX] UpdateFolderSizesIfChanged: still busy after %d attempts (%d paths, %v) — skipped",
logger.Warningf("[DB_TX] UpdateFolderSizesIfChanged: still busy after %d attempts (%d paths, %v)",
dbTransientOpAttempts, len(pathSizes), time.Since(startTime))
return 0, nil
return 0, fmt.Errorf("update folder sizes failed after %d attempts (%d paths): %w",
dbTransientOpAttempts, len(pathSizes), sqlitebusy.ErrBusy)
}
return 0, err
}
@@ -1130,7 +1145,7 @@ func (db *IndexDB) GetFilesBySizeAndType(source string, size int64, fileType str
if pathPrefix != "" {
nextPrefix := getNextPathPrefix(pathPrefix)
query = `
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items
WHERE source = ? AND size = ? AND type = ? AND is_dir = 0 AND path >= ? AND path < ?
ORDER BY name
@@ -1138,7 +1153,7 @@ func (db *IndexDB) GetFilesBySizeAndType(source string, size int64, fileType str
args = []interface{}{source, size, fileType, pathPrefix, nextPrefix}
} else {
query = `
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items
WHERE source = ? AND size = ? AND type = ? AND is_dir = 0
ORDER BY name
@@ -1148,11 +1163,7 @@ func (db *IndexDB) GetFilesBySizeAndType(source string, size int64, fileType str
rows, err := db.Query(query, args...)
if err != nil {
// Soft failure: DB is busy or locked, return empty results
if isBusyError(err) || isTransactionError(err) {
return []*iteminfo.FileInfo{}, nil
}
return nil, err
return nil, sqlitebusy.Wrap(err)
}
defer rows.Close()
@@ -1176,7 +1187,7 @@ func (db *IndexDB) GetFilesBySize(source string, size int64, pathPrefix string)
if pathPrefix != "" {
nextPrefix := getNextPathPrefix(pathPrefix)
query = `
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items
WHERE source = ? AND size = ? AND is_dir = 0 AND path >= ? AND path < ?
ORDER BY name
@@ -1184,7 +1195,7 @@ func (db *IndexDB) GetFilesBySize(source string, size int64, pathPrefix string)
args = []interface{}{source, size, pathPrefix, nextPrefix}
} else {
query = `
SELECT path, name, size, mod_time, type, is_dir, is_hidden, has_preview
SELECT path, name, size, created_time, mod_time, type, is_dir, is_hidden, has_preview
FROM index_items
WHERE source = ? AND size = ? AND is_dir = 0
ORDER BY name
@@ -1194,11 +1205,7 @@ func (db *IndexDB) GetFilesBySize(source string, size int64, pathPrefix string)
rows, err := db.Query(query, args...)
if err != nil {
// Soft failure: DB is busy or locked, return empty slice
if isBusyError(err) || isTransactionError(err) {
return []*iteminfo.FileInfo{}, nil
}
return nil, err
return nil, sqlitebusy.Wrap(err)
}
defer rows.Close()
@@ -1244,11 +1251,7 @@ func (db *IndexDB) GetSizeGroupsForDuplicates(source string, minSize int64, path
rows, err := db.Query(query, args...)
if err != nil {
// Soft failure: DB is busy or locked, return empty results
if isBusyError(err) || isTransactionError(err) {
return []int64{}, make(map[int64]int), nil
}
return nil, nil, err
return nil, nil, sqlitebusy.Wrap(err)
}
defer rows.Close()
@@ -1307,6 +1310,7 @@ func getParentPath(path string) string {
func scanItem(scanner interface{ Scan(...interface{}) error }) (*iteminfo.FileInfo, error) {
var info iteminfo.FileInfo
var createdTime int64
var modTime int64
var isDir bool
@@ -1314,6 +1318,7 @@ func scanItem(scanner interface{ Scan(...interface{}) error }) (*iteminfo.FileIn
&info.Path,
&info.Name,
&info.Size,
&createdTime,
&modTime,
&info.Type,
&isDir,
@@ -1323,6 +1328,10 @@ func scanItem(scanner interface{ Scan(...interface{}) error }) (*iteminfo.FileIn
if err != nil {
return nil, err
}
if createdTime > 0 {
t := time.Unix(createdTime, 0)
info.Created = &t
}
info.ModTime = time.Unix(modTime, 0)
info.IsDir = isDir
return &info, nil
+13 -12
View File
@@ -8,14 +8,15 @@ import (
// SearchItem represents a single search result row from the database
type SearchItem struct {
Source string
Path string
Name string
Size int64
ModTime int64
Type string
IsDir bool
HasPreview bool
Source string
Path string
Name string
Size int64
CreatedTime int64
ModTime int64
Type string
IsDir bool
HasPreview bool
}
// appendNameGlobSQL adds AND (name GLOB ? OR ...) or AND (name GLOB ? AND ...) when joinAnd is true.
@@ -44,8 +45,8 @@ func appendNameGlobSQL(query string, args []interface{}, patterns []string, join
// Returns rows that can be iterated to scan search results.
func (db *IndexDB) SearchItems(source string, scope string, largest bool, nameGlobPatterns []string, nameGlobPatternsAnd, caseExact bool) (*sql.Rows, error) {
query := `
SELECT path, name, size, mod_time, type, is_dir, has_preview
FROM index_items
SELECT path, name, size, created_time, mod_time, type, is_dir, has_preview
FROM index_items
WHERE source = ?
`
args := []interface{}{source}
@@ -77,8 +78,8 @@ func (db *IndexDB) SearchItemsMultiSource(sources []string, sourceScopes map[str
}
query := `
SELECT source, path, name, size, mod_time, type, is_dir, has_preview
FROM index_items
SELECT source, path, name, size, created_time, mod_time, type, is_dir, has_preview
FROM index_items
`
args := []interface{}{}
whereClauses := []string{}
+5 -3
View File
@@ -9,6 +9,7 @@ import (
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/sqlitebusy"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-logger/logger"
// SQLite driver is imported in driver_cgo.go or driver_nocgo.go based on build tags
@@ -225,9 +226,10 @@ func NewTempDB(id string, config ...*TempDBConfig) (*TempDB, error) {
tmpFile.Close()
}
// Open SQLite database with basic connection string
// Driver is selected at compile time: "sqlite3" (CGO) or "sqlite" (pure Go)
db, err := sql.Open(SqliteDriver, tmpPath)
// Open SQLite database with basic connection string.
// Driver is selected at compile time: "sqlite3" (CGO) or "sqlite" (pure Go).
// busy_timeout is set in the DSN so every pooled connection honors it.
db, err := sql.Open(SqliteDriver, sqlitebusy.WithBusyTimeout(tmpPath))
if err != nil {
os.Remove(tmpPath)
return nil, fmt.Errorf("failed to open SQLite database: %w", err)
+37
View File
@@ -101,6 +101,43 @@ func (s *SQLStore) SaveGroup(name string, members access.StringSet) error {
return nil
}
// DeleteGroupWithRules deletes a group and applies access rule row changes in a single
// transaction. If any statement fails nothing is written. A group row that is already
// absent is not an error, so the call is safe to retry.
func (s *SQLStore) DeleteGroupWithRules(name string, save []access.RuleUpsert, remove []access.RuleKey) error {
tx, err := s.BeginTx()
if err != nil {
return fmt.Errorf("failed to begin transaction: %w", err)
}
rollback := func(cause error) error {
if rbErr := tx.Rollback(); rbErr != nil {
return fmt.Errorf("%w (rollback also failed: %v)", cause, rbErr)
}
return cause
}
for _, u := range save {
ruleDataJSON, err := json.Marshal(u.Rule)
if err != nil {
return rollback(fmt.Errorf("failed to marshal rule data: %w", err))
}
if _, err := tx.Exec(`INSERT OR REPLACE INTO access_rules (source, path, rule_data) VALUES (?, ?, ?)`, u.Source, u.Path, ruleDataJSON); err != nil {
return rollback(fmt.Errorf("failed to save access rule: %w", err))
}
}
for _, k := range remove {
if _, err := tx.Exec(`DELETE FROM access_rules WHERE source = ? AND path = ?`, k.Source, k.Path); err != nil {
return rollback(fmt.Errorf("failed to delete access rule: %w", err))
}
}
if _, err := tx.Exec(`DELETE FROM groups WHERE name = ?`, name); err != nil {
return rollback(fmt.Errorf("failed to delete group: %w", err))
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("failed to commit group delete: %w", err)
}
return nil
}
// DeleteGroup deletes a group by name
func (s *SQLStore) DeleteGroup(name string) error {
query := `DELETE FROM groups WHERE name = ?`
@@ -0,0 +1,98 @@
package sqldb
import (
"path/filepath"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/access"
)
// newGroupTestStore opens a fresh SQLStore in a temp dir.
func newGroupTestStore(t *testing.T) *SQLStore {
t.Helper()
store, _, err := NewSQLStore(filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatalf("failed to create SQL store: %v", err)
}
t.Cleanup(func() { store.Close() })
return store
}
// groupRule returns a rule that allows the given group.
func groupRule(group string) *access.AccessRule {
return &access.AccessRule{Allow: access.RuleSet{
Users: access.StringSet{},
Groups: access.StringSet{group: {}},
}}
}
func TestDeleteGroupWithRules_CommitsAllChanges(t *testing.T) {
store := newGroupTestStore(t)
keep := access.RuleKey{Source: "src", Path: "/keep"}
drop := access.RuleKey{Source: "src", Path: "/drop"}
if err := store.SaveGroup("acme", access.StringSet{"alice": {}}); err != nil {
t.Fatal(err)
}
if err := store.SaveAccessRule(keep.Source, keep.Path, groupRule("acme")); err != nil {
t.Fatal(err)
}
if err := store.SaveAccessRule(drop.Source, drop.Path, groupRule("acme")); err != nil {
t.Fatal(err)
}
// keep is rewritten (now allows "other"), drop is removed, the group row is deleted.
err := store.DeleteGroupWithRules("acme", []access.RuleUpsert{{RuleKey: keep, Rule: groupRule("other")}}, []access.RuleKey{drop})
if err != nil {
t.Fatalf("DeleteGroupWithRules: %v", err)
}
groups, err := store.GetAllGroups()
if err != nil {
t.Fatal(err)
}
if _, ok := groups["acme"]; ok {
t.Fatal("group row should be deleted")
}
rules, err := store.GetAllAccessRules()
if err != nil {
t.Fatal(err)
}
if _, ok := rules["src"]["/drop"]; ok {
t.Fatal("dropped rule row should be deleted")
}
kept, ok := rules["src"]["/keep"]
if !ok {
t.Fatal("kept rule row should still exist")
}
if _, has := kept.Allow.Groups["other"]; !has {
t.Fatalf("kept rule should have been rewritten, got %+v", kept.Allow.Groups)
}
// Deleting again (group row already absent) is not an error.
if err := store.DeleteGroupWithRules("acme", nil, nil); err != nil {
t.Fatalf("repeat delete should succeed: %v", err)
}
}
func TestDeleteGroupWithRules_RollsBackOnFailure(t *testing.T) {
store := newGroupTestStore(t)
if err := store.SaveGroup("acme", access.StringSet{"alice": {}}); err != nil {
t.Fatal(err)
}
// Make the rule statements fail so the transaction has to roll back.
if _, err := store.db.Exec(`DROP TABLE access_rules`); err != nil {
t.Fatal(err)
}
err := store.DeleteGroupWithRules("acme", []access.RuleUpsert{{RuleKey: access.RuleKey{Source: "src", Path: "/x"}, Rule: groupRule("other")}}, nil)
if err == nil {
t.Fatal("expected an error when the rule write fails")
}
groups, gErr := store.GetAllGroups()
if gErr != nil {
t.Fatal(gErr)
}
if _, ok := groups["acme"]; !ok {
t.Fatal("group row must survive a failed transaction")
}
}
@@ -0,0 +1,12 @@
package sqldb
import (
"os"
"testing"
"github.com/gtsteffaniak/filebrowser/backend/internal/testutil"
)
func TestMain(m *testing.M) {
os.Exit(testutil.Main(m))
}
+91 -2
View File
@@ -6,7 +6,7 @@ import (
)
// currentSchemaVersion is the SQLite schema marker for this codebase.
const currentSchemaVersion = 3
const currentSchemaVersion = 4
// Schema creates all tables for the SQLite database
func createSchema(db *sql.DB) error {
@@ -78,7 +78,8 @@ func createSchema(db *sql.DB) error {
CREATE TABLE IF NOT EXISTS hashed_tokens (
token_hash TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
is_session INTEGER NOT NULL DEFAULT 0
is_session INTEGER NOT NULL DEFAULT 0,
expires_at INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_hashed_tokens_user_id ON hashed_tokens(user_id);
@@ -123,6 +124,28 @@ func createSchema(db *sql.DB) error {
CREATE INDEX IF NOT EXISTS idx_activity_user_created ON activity_log(user_id, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_activity_event_created ON activity_log(event_type, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_activity_user_event_created ON activity_log(user_id, event_type, created_at DESC);
-- Folder quotas (path-bound caps; meter: index_size | accounted)
CREATE TABLE IF NOT EXISTS quotas (
id TEXT PRIMARY KEY NOT NULL,
source TEXT NOT NULL,
path TEXT NOT NULL,
user_id TEXT,
limit_bytes INTEGER NOT NULL DEFAULT 0,
meter TEXT NOT NULL DEFAULT 'index_size',
created_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_quotas_source ON quotas(source);
CREATE INDEX IF NOT EXISTS idx_quotas_source_path ON quotas(source, path);
-- Accounted usage counters (scope, share, optional folder accounted in future)
CREATE TABLE IF NOT EXISTS quota_counters (
quota_id TEXT PRIMARY KEY NOT NULL,
used_bytes INTEGER NOT NULL DEFAULT 0,
reserved_bytes INTEGER NOT NULL DEFAULT 0,
version INTEGER NOT NULL DEFAULT 0,
updated_at INTEGER NOT NULL
);
`
_, err := db.Exec(schema)
@@ -183,6 +206,9 @@ func runMigrations(db *sql.DB, fromVersion int) error {
case 1:
// Canonical schema is createSchema + Bolt import; no step migrations.
case 2:
if err := migrateV2(db); err != nil {
return err
}
if err := normalizeLegacyShareTokens(db); err != nil {
return err
}
@@ -190,6 +216,10 @@ func runMigrations(db *sql.DB, fromVersion int) error {
if err := addHashedTokenSessionColumn(db); err != nil {
return err
}
case 4:
if err := addHashedTokenExpiryColumn(db); err != nil {
return err
}
default:
return fmt.Errorf("unknown schema version: %d", v)
}
@@ -204,6 +234,48 @@ func runMigrations(db *sql.DB, fromVersion int) error {
return nil
}
func migrateV2(db *sql.DB) error {
schema := `
CREATE TABLE IF NOT EXISTS quotas (
id TEXT PRIMARY KEY NOT NULL,
source TEXT NOT NULL,
path TEXT NOT NULL,
user_id TEXT,
limit_bytes INTEGER NOT NULL DEFAULT 0,
meter TEXT NOT NULL DEFAULT 'index_size',
created_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_quotas_source ON quotas(source);
CREATE INDEX IF NOT EXISTS idx_quotas_source_path ON quotas(source, path);
CREATE TABLE IF NOT EXISTS quota_counters (
quota_id TEXT PRIMARY KEY NOT NULL,
used_bytes INTEGER NOT NULL DEFAULT 0,
reserved_bytes INTEGER NOT NULL DEFAULT 0,
version INTEGER NOT NULL DEFAULT 0,
updated_at INTEGER NOT NULL
);
`
if _, err := db.Exec(schema); err != nil {
return fmt.Errorf("migrate v2: %w", err)
}
return migrateV2AddMeterColumn(db)
}
func migrateV2AddMeterColumn(db *sql.DB) error {
var count int
if err := db.QueryRow(`SELECT COUNT(*) FROM pragma_table_info('quotas') WHERE name = 'meter'`).Scan(&count); err != nil {
return fmt.Errorf("migrate v2 meter column check: %w", err)
}
if count > 0 {
return nil
}
if _, err := db.Exec(`ALTER TABLE quotas ADD COLUMN meter TEXT NOT NULL DEFAULT 'index_size'`); err != nil {
return fmt.Errorf("migrate v2 add meter column: %w", err)
}
return nil
}
// addHashedTokenSessionColumn adds the is_session flag to pre-existing
// hashed_tokens tables. Existing rows are treated as non-session (API) tokens.
func addHashedTokenSessionColumn(db *sql.DB) error {
@@ -220,6 +292,23 @@ func addHashedTokenSessionColumn(db *sql.DB) error {
return nil
}
// addHashedTokenExpiryColumn adds the expires_at timestamp to pre-existing
// hashed_tokens tables. Existing rows get 0 (unknown expiry) and are left
// untouched; expiry is recorded for newly registered tokens.
func addHashedTokenExpiryColumn(db *sql.DB) error {
hasColumn, err := tableHasColumn(db, "hashed_tokens", "expires_at")
if err != nil {
return err
}
if hasColumn {
return nil
}
if _, err := db.Exec(`ALTER TABLE hashed_tokens ADD COLUMN expires_at INTEGER NOT NULL DEFAULT 0`); err != nil {
return fmt.Errorf("failed to add hashed_tokens.expires_at: %w", err)
}
return nil
}
// tableHasColumn reports whether table/column exists in the current schema.
func tableHasColumn(db *sql.DB, table, column string) (bool, error) {
rows, err := db.Query(fmt.Sprintf("PRAGMA table_info(%s)", table))
+221
View File
@@ -0,0 +1,221 @@
package sqldb
import (
"database/sql"
"fmt"
"strconv"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/quota"
)
func scanQuotaUserID(s string, dest *uint64) error {
if s == "" {
*dest = 0
return nil
}
u, err := strconv.ParseUint(s, 10, 64)
if err != nil {
return fmt.Errorf("parse quota user_id: %w", err)
}
*dest = u
return nil
}
func quotaUserIDDB(id uint64) string {
if id == 0 {
return ""
}
return strconv.FormatUint(id, 10)
}
// GetAllFolderQuotas loads every folder quota row.
func (s *SQLStore) GetAllFolderQuotas() ([]quota.FolderQuota, error) {
rows, err := s.db.Query(`SELECT id, source, path, user_id, limit_bytes, meter FROM quotas`)
if err != nil {
return nil, fmt.Errorf("query quotas: %w", err)
}
defer rows.Close()
var out []quota.FolderQuota
for rows.Next() {
var q quota.FolderQuota
var userIDStr string
if err := rows.Scan(&q.ID, &q.Source, &q.Path, &userIDStr, &q.LimitBytes, &q.Meter); err != nil {
return nil, fmt.Errorf("scan quota: %w", err)
}
if err := scanQuotaUserID(userIDStr, &q.UserID); err != nil {
return nil, err
}
out = append(out, q)
}
return out, rows.Err()
}
// GetFolderQuotaByID returns one folder quota.
func (s *SQLStore) GetFolderQuotaByID(id string) (*quota.FolderQuota, error) {
var q quota.FolderQuota
var userIDStr string
err := s.db.QueryRow(`SELECT id, source, path, user_id, limit_bytes, meter FROM quotas WHERE id = ?`, id).
Scan(&q.ID, &q.Source, &q.Path, &userIDStr, &q.LimitBytes, &q.Meter)
if err == sql.ErrNoRows {
return nil, fmt.Errorf("quota not found")
}
if err != nil {
return nil, fmt.Errorf("get quota: %w", err)
}
if err := scanQuotaUserID(userIDStr, &q.UserID); err != nil {
return nil, err
}
return &q, nil
}
// GetFolderQuotasForSourcePath returns quotas for a source and optional path prefix.
func (s *SQLStore) GetFolderQuotasForSourcePath(source, path string) ([]quota.FolderQuota, error) {
path = strings.TrimSuffix(path, "/")
query := `SELECT id, source, path, user_id, limit_bytes, meter FROM quotas WHERE source = ?`
args := []interface{}{source}
if path != "" {
query += ` AND (path = ? OR path LIKE ?)`
args = append(args, path, path+"/%")
}
rows, err := s.db.Query(query, args...)
if err != nil {
return nil, fmt.Errorf("query quotas for source: %w", err)
}
defer rows.Close()
var out []quota.FolderQuota
for rows.Next() {
var q quota.FolderQuota
var userIDStr string
if err := rows.Scan(&q.ID, &q.Source, &q.Path, &userIDStr, &q.LimitBytes, &q.Meter); err != nil {
return nil, fmt.Errorf("scan quota: %w", err)
}
if err := scanQuotaUserID(userIDStr, &q.UserID); err != nil {
return nil, err
}
out = append(out, q)
}
return out, rows.Err()
}
// SaveFolderQuota inserts or updates a folder quota.
func (s *SQLStore) SaveFolderQuota(q *quota.FolderQuota) error {
meter := q.Meter
if meter == "" {
meter = quota.MeterIndexSize
}
_, err := s.db.Exec(
`INSERT INTO quotas (id, source, path, user_id, limit_bytes, meter, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET source=excluded.source, path=excluded.path, user_id=excluded.user_id, limit_bytes=excluded.limit_bytes, meter=excluded.meter`,
q.ID, q.Source, q.Path, quotaUserIDDB(q.UserID), q.LimitBytes, meter, currentTimestamp(),
)
if err != nil {
return fmt.Errorf("save quota: %w", err)
}
return nil
}
// DeleteFolderQuota removes a folder quota and its counter.
func (s *SQLStore) DeleteFolderQuota(id string) error {
if _, err := s.db.Exec(`DELETE FROM quotas WHERE id = ?`, id); err != nil {
return fmt.Errorf("delete quota: %w", err)
}
if _, err := s.db.Exec(`DELETE FROM quota_counters WHERE quota_id = ?`, id); err != nil {
return fmt.Errorf("delete quota counter: %w", err)
}
return nil
}
// GetAllQuotaCounters loads all quota counters.
func (s *SQLStore) GetAllQuotaCounters() ([]quota.Counter, error) {
rows, err := s.db.Query(`SELECT quota_id, used_bytes, reserved_bytes FROM quota_counters`)
if err != nil {
return nil, fmt.Errorf("query quota counters: %w", err)
}
defer rows.Close()
var out []quota.Counter
for rows.Next() {
var c quota.Counter
if err := rows.Scan(&c.QuotaID, &c.UsedBytes, &c.ReservedBytes); err != nil {
return nil, fmt.Errorf("scan quota counter: %w", err)
}
out = append(out, c)
}
return out, rows.Err()
}
// GetQuotaCounter returns one counter row.
func (s *SQLStore) GetQuotaCounter(quotaID string) (*quota.Counter, error) {
var c quota.Counter
err := s.db.QueryRow(`SELECT quota_id, used_bytes, reserved_bytes FROM quota_counters WHERE quota_id = ?`, quotaID).
Scan(&c.QuotaID, &c.UsedBytes, &c.ReservedBytes)
if err == sql.ErrNoRows {
return nil, fmt.Errorf("quota counter not found")
}
if err != nil {
return nil, fmt.Errorf("get quota counter: %w", err)
}
return &c, nil
}
// EnsureQuotaCounter creates a counter row if missing.
func (s *SQLStore) EnsureQuotaCounter(quotaID string) error {
_, err := s.db.Exec(
`INSERT INTO quota_counters (quota_id, used_bytes, reserved_bytes, version, updated_at)
VALUES (?, 0, 0, 0, ?)
ON CONFLICT(quota_id) DO NOTHING`,
quotaID, currentTimestamp(),
)
if err != nil {
return fmt.Errorf("ensure quota counter: %w", err)
}
return nil
}
// DeleteQuotaCounter removes a counter row.
func (s *SQLStore) DeleteQuotaCounter(quotaID string) error {
_, err := s.db.Exec(`DELETE FROM quota_counters WHERE quota_id = ?`, quotaID)
if err != nil {
return fmt.Errorf("delete quota counter: %w", err)
}
return nil
}
// BulkUpsertQuotaCounters persists dirty counters.
func (s *SQLStore) BulkUpsertQuotaCounters(counters []quota.Counter) error {
if len(counters) == 0 {
return nil
}
tx, err := s.db.Begin()
if err != nil {
return fmt.Errorf("begin quota counter tx: %w", err)
}
defer func() {
_ = tx.Rollback()
}()
stmt, err := tx.Prepare(
`INSERT INTO quota_counters (quota_id, used_bytes, reserved_bytes, version, updated_at)
VALUES (?, ?, ?, 0, ?)
ON CONFLICT(quota_id) DO UPDATE SET
used_bytes = excluded.used_bytes,
reserved_bytes = excluded.reserved_bytes,
updated_at = excluded.updated_at`,
)
if err != nil {
return fmt.Errorf("prepare quota counter upsert: %w", err)
}
defer stmt.Close()
now := currentTimestamp()
for _, c := range counters {
if _, err := stmt.Exec(c.QuotaID, c.UsedBytes, c.ReservedBytes, now); err != nil {
return fmt.Errorf("upsert quota counter %s: %w", c.QuotaID, err)
}
}
return tx.Commit()
}
+22 -6
View File
@@ -32,20 +32,36 @@ func (s *SQLStore) GetSetting(key string) ([]byte, error) {
// SaveSetting inserts or updates a setting
func (s *SQLStore) SaveSetting(key string, value interface{}) error {
query := `INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)`
if err := s.saveSettingQuery(s.db.Exec, query, key, value); err != nil {
return fmt.Errorf("failed to save setting: %w", err)
}
return nil
}
type settingExec func(query string, args ...interface{}) (sql.Result, error)
func (s *SQLStore) saveSettingQuery(exec settingExec, query, key string, value interface{}) error {
valueJSON, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("failed to marshal setting value: %w", err)
}
query := `INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)`
_, err = s.db.Exec(query, key, valueJSON)
if err != nil {
return fmt.Errorf("failed to save setting: %w", err)
if _, err := exec(query, key, valueJSON); err != nil {
return err
}
return nil
}
func (s *SQLStore) saveSettingInTx(tx *sql.Tx, key string, value interface{}) error {
query := `INSERT OR REPLACE INTO settings (key, value) VALUES (?, ?)`
return s.saveSettingQuery(tx.Exec, query, key, value)
}
// SaveSettingTx inserts or updates a setting within an existing transaction.
func (s *SQLStore) SaveSettingTx(tx *sql.Tx, key string, value interface{}) error {
return s.saveSettingInTx(tx, key, value)
}
// DeleteSetting deletes a setting by key
func (s *SQLStore) DeleteSetting(key string) error {
query := `DELETE FROM settings WHERE key = ?`
+8 -2
View File
@@ -7,6 +7,7 @@ import (
"path/filepath"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/sqlitebusy"
"github.com/gtsteffaniak/go-logger/logger"
// SQLite driver is imported in driver_cgo.go or driver_nocgo.go based on build tags
)
@@ -38,8 +39,13 @@ func NewSQLStoreWithOptions(dbPath string, opts NewSQLStoreOpts) (*SQLStore, boo
return nil, existingDb, fmt.Errorf("failed to create database directory: %w", err)
}
// Open SQLite database with the appropriate driver
db, err := sql.Open(SqliteDriver, fmt.Sprintf("file:%s?cache=shared&mode=rwc&_journal_mode=WAL", dbPath))
// Open SQLite database with the appropriate driver.
// busy_timeout is set in the DSN so every pooled connection honors it.
// Shared cache is intentionally not used: its table-lock conflicts surface as
// SQLITE_LOCKED_SHAREDCACHE, which busy_timeout does not wait on, and SQLite
// discourages the mode. WAL already provides concurrent readers.
dsn := fmt.Sprintf("file:%s?mode=rwc&_journal_mode=WAL", dbPath)
db, err := sql.Open(SqliteDriver, sqlitebusy.WithBusyTimeout(dsn))
if err != nil {
return nil, existingDb, fmt.Errorf("failed to open database: %w", err)
}
+9 -6
View File
@@ -41,6 +41,7 @@ type sqlExecutor interface {
type HashedTokenRecord struct {
UserID uint64
IsSession bool
ExpiresAt int64 // unix expiry from the token's exp claim; 0 = unknown/none
}
// PersistImmediateTokenRevocation records an immediate revocation and removes the
@@ -145,9 +146,10 @@ func (s *SQLStore) DeleteRevokedToken(tokenHash string) error {
// SaveHashedToken saves a token hash to owner user_id mapping (decimal text).
// isSession records that the bearer token is a FileBrowser session token rather
// than a named API token, so auth can reject non-session tokens without caps.
func (s *SQLStore) SaveHashedToken(tokenHash string, userID uint64, isSession bool) error {
query := `INSERT OR REPLACE INTO hashed_tokens (token_hash, user_id, is_session) VALUES (?, ?, ?)`
_, err := s.db.Exec(query, tokenHash, strconv.FormatUint(userID, 10), isSession)
// expiresAt is the token's exp claim as unix time; 0 means no expiry.
func (s *SQLStore) SaveHashedToken(tokenHash string, userID uint64, isSession bool, expiresAt int64) error {
query := `INSERT OR REPLACE INTO hashed_tokens (token_hash, user_id, is_session, expires_at) VALUES (?, ?, ?, ?)`
_, err := s.db.Exec(query, tokenHash, strconv.FormatUint(userID, 10), isSession, expiresAt)
if err != nil {
return fmt.Errorf("failed to save hashed token: %w", err)
}
@@ -170,7 +172,7 @@ func (s *SQLStore) GetUserIDByTokenHash(tokenHash string) (uint64, error) {
// GetAllHashedTokens retrieves all token hash → owner mappings.
func (s *SQLStore) GetAllHashedTokens() (map[string]HashedTokenRecord, error) {
query := `SELECT token_hash, user_id, is_session FROM hashed_tokens`
query := `SELECT token_hash, user_id, is_session, expires_at FROM hashed_tokens`
rows, err := s.db.Query(query)
if err != nil {
return nil, fmt.Errorf("failed to get hashed tokens: %w", err)
@@ -181,14 +183,15 @@ func (s *SQLStore) GetAllHashedTokens() (map[string]HashedTokenRecord, error) {
for rows.Next() {
var tokenHash, idStr string
var isSession bool
if err := rows.Scan(&tokenHash, &idStr, &isSession); err != nil {
var expiresAt int64
if err := rows.Scan(&tokenHash, &idStr, &isSession, &expiresAt); err != nil {
return nil, fmt.Errorf("failed to scan hashed token: %w", err)
}
uid, err := strconv.ParseUint(idStr, 10, 64)
if err != nil {
return nil, fmt.Errorf("invalid user_id in hashed_tokens: %w", err)
}
hashedTokens[tokenHash] = HashedTokenRecord{UserID: uid, IsSession: isSession}
hashedTokens[tokenHash] = HashedTokenRecord{UserID: uid, IsSession: isSession, ExpiresAt: expiresAt}
}
if err := rows.Err(); err != nil {
@@ -32,7 +32,7 @@ func TestMigrationAddsHashedTokenSessionColumn(t *testing.T) {
t.Fatalf("runMigrations: %v", err)
}
if err = store.SaveHashedToken("new-hash", 8, true); err != nil {
if err = store.SaveHashedToken("new-hash", 8, true, 12345); err != nil {
t.Fatalf("SaveHashedToken after migration: %v", err)
}
records, err := store.GetAllHashedTokens()
@@ -46,9 +46,15 @@ func TestMigrationAddsHashedTokenSessionColumn(t *testing.T) {
if legacy.IsSession {
t.Fatal("legacy rows must migrate as non-session tokens")
}
if legacy.ExpiresAt != 0 {
t.Fatalf("legacy rows must migrate with no expiry, got %d", legacy.ExpiresAt)
}
if !records["new-hash"].IsSession {
t.Fatal("session flag was not persisted")
}
if records["new-hash"].ExpiresAt != 12345 {
t.Fatalf("expires_at not persisted, got %d", records["new-hash"].ExpiresAt)
}
}
func TestGetSettingMissingReturnsSentinel(t *testing.T) {
@@ -0,0 +1,42 @@
package sqlitebusy
import (
"errors"
"fmt"
"strings"
)
// DefaultBusyTimeoutMs is how long SQLite waits on SQLITE_BUSY before failing an operation.
const DefaultBusyTimeoutMs = 5000
// ErrBusy is returned when an operation could not complete because the database was busy or locked.
var ErrBusy = errors.New("sqlite database is busy or locked")
// WithBusyTimeout appends the busy_timeout connection parameter to dsn so that
// every connection in the pool honors the timeout. Both supported drivers
// (mattn/go-sqlite3 and modernc.org/sqlite) apply the parameter at
// connection-open time, unlike a one-off PRAGMA which only configures the
// single connection that database/sql happens to pick.
func WithBusyTimeout(dsn string) string {
separator := "?"
if strings.Contains(dsn, "?") {
separator = "&"
}
return fmt.Sprintf("%s%s_busy_timeout=%d", dsn, separator, DefaultBusyTimeoutMs)
}
// IsBusyOrLocked reports whether err is SQLITE_BUSY or SQLITE_LOCKED from the
// configured SQLite driver, classified by the driver's result code rather than
// error text.
func IsBusyOrLocked(err error) bool {
return err != nil && isBusyOrLocked(err)
}
// Wrap returns err wrapped with ErrBusy when the underlying error is busy/locked.
// The original error is retained so callers can still inspect the driver result code.
func Wrap(err error) error {
if err == nil || !IsBusyOrLocked(err) {
return err
}
return fmt.Errorf("%w: %w", ErrBusy, err)
}
@@ -0,0 +1,199 @@
package sqlitebusy
import (
"context"
"database/sql"
"errors"
"path/filepath"
"testing"
)
// sqliteBusySnapshot is the extended result code returned when a WAL read
// transaction tries to upgrade to a write on a stale snapshot.
const sqliteBusySnapshot = 517
func TestIsBusyOrLocked(t *testing.T) {
if IsBusyOrLocked(nil) {
t.Fatal("expected nil not to be busy")
}
if IsBusyOrLocked(newGenericError(t)) {
t.Fatal("expected non-busy SQLITE_ERROR not to be classified as busy")
}
if !IsBusyOrLocked(newBusyError(t)) {
t.Fatal("expected SQLITE_BUSY to be classified as busy")
}
if !IsBusyOrLocked(newBusySnapshotError(t)) {
t.Fatal("expected SQLITE_BUSY_SNAPSHOT to be classified as busy")
}
}
func TestWrap(t *testing.T) {
for name, makeErr := range map[string]func(*testing.T) error{
"SQLITE_BUSY": newBusyError,
"SQLITE_BUSY_SNAPSHOT": newBusySnapshotError,
} {
t.Run(name, func(t *testing.T) {
wrapped := Wrap(makeErr(t))
if !errors.Is(wrapped, ErrBusy) {
t.Fatalf("expected ErrBusy, got %v", wrapped)
}
})
}
generic := newGenericError(t)
if err := Wrap(generic); errors.Is(err, ErrBusy) {
t.Fatalf("expected generic error to pass through unwrapped, got %v", err)
}
if err := Wrap(nil); err != nil {
t.Fatalf("expected nil, got %v", err)
}
}
// TestWithBusyTimeoutAppliesToEveryConnection verifies the DSN parameter is
// applied at connection-open time, so a second pooled connection does not
// bypass the timeout like a one-off PRAGMA would.
func TestWithBusyTimeoutAppliesToEveryConnection(t *testing.T) {
dbPath := filepath.Join(t.TempDir(), "pool.db")
db, err := sql.Open(testDriver, WithBusyTimeout("file:"+dbPath))
if err != nil {
t.Fatalf("open: %v", err)
}
defer db.Close()
db.SetMaxOpenConns(2)
ctx := context.Background()
conn1, err := db.Conn(ctx)
if err != nil {
t.Fatalf("conn1: %v", err)
}
defer conn1.Close()
conn2, err := db.Conn(ctx)
if err != nil {
t.Fatalf("conn2: %v", err)
}
defer conn2.Close()
for i, conn := range []*sql.Conn{conn1, conn2} {
var timeout int
if err := conn.QueryRowContext(ctx, "PRAGMA busy_timeout").Scan(&timeout); err != nil {
t.Fatalf("conn %d: read busy_timeout: %v", i+1, err)
}
if timeout != DefaultBusyTimeoutMs {
t.Fatalf("conn %d: busy_timeout = %d, want %d", i+1, timeout, DefaultBusyTimeoutMs)
}
}
}
// newGenericError returns a real SQLITE_ERROR (result code 1) from the driver.
func newGenericError(t *testing.T) error {
t.Helper()
db, err := sql.Open(testDriver, filepath.Join(t.TempDir(), "generic.db"))
if err != nil {
t.Fatalf("open: %v", err)
}
defer db.Close()
_, err = db.Exec("SELECT * FROM table_that_does_not_exist")
if err == nil {
t.Fatal("expected error from invalid query")
}
return err
}
// newBusyError returns a real SQLITE_BUSY (result code 5) from the driver by
// holding a write lock on one connection and writing from a second with a zero
// busy timeout.
func newBusyError(t *testing.T) error {
t.Helper()
dbPath := filepath.Join(t.TempDir(), "busy.db")
holder, err := sql.Open(testDriver, WithBusyTimeout("file:"+dbPath))
if err != nil {
t.Fatalf("open holder: %v", err)
}
defer holder.Close()
holder.SetMaxOpenConns(1)
if _, err = holder.Exec("CREATE TABLE t (id INTEGER)"); err != nil {
t.Fatalf("create table: %v", err)
}
tx, err := holder.Begin()
if err != nil {
t.Fatalf("begin: %v", err)
}
defer func() { _ = tx.Rollback() }()
if _, err = tx.Exec("INSERT INTO t VALUES (1)"); err != nil {
t.Fatalf("holder insert: %v", err)
}
contender, err := sql.Open(testDriver, "file:"+dbPath+"?_busy_timeout=0")
if err != nil {
t.Fatalf("open contender: %v", err)
}
defer contender.Close()
_, err = contender.Exec("INSERT INTO t VALUES (2)")
if err == nil {
t.Fatal("expected SQLITE_BUSY from contended write")
}
return err
}
// newBusySnapshotError returns a real extended-busy error (SQLITE_BUSY_SNAPSHOT)
// from the driver. In WAL mode, a transaction that has already taken a read
// snapshot cannot upgrade to a write after another connection committed —
// SQLite returns SQLITE_BUSY_SNAPSHOT instead of waiting on busy_timeout.
func newBusySnapshotError(t *testing.T) error {
t.Helper()
dbPath := filepath.Join(t.TempDir(), "snapshot.db")
dsn := "file:" + dbPath + "?_journal_mode=WAL&_busy_timeout=0"
db, err := sql.Open(testDriver, dsn)
if err != nil {
t.Fatalf("open: %v", err)
}
defer db.Close()
db.SetMaxOpenConns(2)
if _, err = db.Exec("CREATE TABLE t (id INTEGER); INSERT INTO t VALUES (1)"); err != nil {
t.Fatalf("setup: %v", err)
}
ctx := context.Background()
reader, err := db.Conn(ctx)
if err != nil {
t.Fatalf("reader conn: %v", err)
}
defer reader.Close()
tx, err := reader.BeginTx(ctx, nil)
if err != nil {
t.Fatalf("begin: %v", err)
}
defer func() { _ = tx.Rollback() }()
// Establish the read snapshot.
var count int
if err = tx.QueryRow("SELECT COUNT(*) FROM t").Scan(&count); err != nil {
t.Fatalf("snapshot read: %v", err)
}
// Commit a write from a different connection, invalidating the snapshot.
writer, err := db.Conn(ctx)
if err != nil {
t.Fatalf("writer conn: %v", err)
}
defer writer.Close()
if _, err = writer.ExecContext(ctx, "INSERT INTO t VALUES (2)"); err != nil {
t.Fatalf("writer insert: %v", err)
}
// Upgrading the stale snapshot to a write must fail with SQLITE_BUSY_SNAPSHOT.
_, err = tx.Exec("INSERT INTO t VALUES (3)")
if err == nil {
t.Fatal("expected SQLITE_BUSY_SNAPSHOT from write on stale snapshot")
}
if code := driverErrorCode(err); code != sqliteBusySnapshot {
t.Fatalf("expected extended code %d (SQLITE_BUSY_SNAPSHOT), got %d", sqliteBusySnapshot, code)
}
return err
}
@@ -0,0 +1,23 @@
//go:build cgosql
// +build cgosql
package sqlitebusy
import (
"errors"
sqlite3 "github.com/mattn/go-sqlite3"
)
const testDriver = "sqlite3"
// driverErrorCode returns the driver's extended result code for err, so tests
// can assert which busy/locked variant was produced. mattn reports the
// extended code in Error.ExtendedCode.
func driverErrorCode(err error) int {
var sqliteErr sqlite3.Error
if errors.As(err, &sqliteErr) {
return int(sqliteErr.ExtendedCode)
}
return -1
}
@@ -0,0 +1,23 @@
//go:build !cgosql
// +build !cgosql
package sqlitebusy
import (
"errors"
sqlite "modernc.org/sqlite"
)
const testDriver = "sqlite"
// driverErrorCode returns the driver's extended result code for err, so tests
// can assert which busy/locked variant was produced. modernc reports the
// (possibly extended) code via Error.Code.
func driverErrorCode(err error) int {
var sqliteErr *sqlite.Error
if errors.As(err, &sqliteErr) {
return sqliteErr.Code()
}
return -1
}
@@ -0,0 +1,21 @@
//go:build cgosql
// +build cgosql
package sqlitebusy
import (
"errors"
sqlite3 "github.com/mattn/go-sqlite3"
)
// isBusyOrLocked reports whether err is a SQLITE_BUSY or SQLITE_LOCKED error
// from the CGO (mattn/go-sqlite3) driver. Error.Code is already the primary
// result code; extended codes are reported separately in Error.ExtendedCode.
func isBusyOrLocked(err error) bool {
var sqliteErr sqlite3.Error
if !errors.As(err, &sqliteErr) {
return false
}
return sqliteErr.Code == sqlite3.ErrBusy || sqliteErr.Code == sqlite3.ErrLocked
}
@@ -0,0 +1,28 @@
//go:build !cgosql
// +build !cgosql
package sqlitebusy
import (
"errors"
sqlite "modernc.org/sqlite"
sqlite3 "modernc.org/sqlite/lib"
)
// isBusyOrLocked reports whether err is a SQLITE_BUSY or SQLITE_LOCKED error
// from the pure-Go (modernc.org/sqlite) driver. The driver enables extended
// result codes, so Error.Code() may carry extra detail in the high bits (e.g.
// SQLITE_BUSY_SNAPSHOT); only the lowest byte holds the primary code.
func isBusyOrLocked(err error) bool {
var sqliteErr *sqlite.Error
if !errors.As(err, &sqliteErr) {
return false
}
switch sqliteErr.Code() & 0xff {
case sqlite3.SQLITE_BUSY, sqlite3.SQLITE_LOCKED:
return true
default:
return false
}
}
@@ -64,6 +64,7 @@ func APIScopesToBackend(apiScopes []FrontendScope) ([]BackendScope, error) {
Path: source.Path,
Scope: scope.Scope,
Permissions: frontendScopePermissions(scope),
Quota: scope.Quota,
})
}
return newScopes, nil
@@ -86,6 +87,7 @@ func (u *User) GetFrontendScopes() []FrontendScope {
Name: source.Name,
Scope: scope.Scope,
Permissions: &perms,
Quota: scope.Quota,
})
}
return newScopes
@@ -11,8 +11,14 @@ const (
SidebarLinkSourceAlt SidebarLinkCategory = "source-alt"
SidebarLinkSourceHybrid SidebarLinkCategory = "source-hybrid"
SidebarLinkSourceHybrid2 SidebarLinkCategory = "source-hybrid-2"
SidebarLinkTool SidebarLinkCategory = "tool"
SidebarLinkCustom SidebarLinkCategory = "custom"
// "-root" variants display the root-filesystem-only disk usage (no nested
// mount aggregation) instead of the aggregated multi-mount totals.
SidebarLinkSourceRoot SidebarLinkCategory = "source-root"
SidebarLinkSourceAltRoot SidebarLinkCategory = "source-alt-root"
SidebarLinkSourceHybridRoot SidebarLinkCategory = "source-hybrid-root"
SidebarLinkSourceHybrid2Root SidebarLinkCategory = "source-hybrid-2-root"
SidebarLinkTool SidebarLinkCategory = "tool"
SidebarLinkCustom SidebarLinkCategory = "custom"
)
// NormalizeSidebarLinkCategory returns a known category string, preserving source-* variants.
@@ -23,7 +29,10 @@ func NormalizeSidebarLinkCategory(category string) string {
}
switch SidebarLinkCategory(c) {
case SidebarLinkSource, SidebarLinkSourceMinimal, SidebarLinkSourceAlt,
SidebarLinkSourceHybrid, SidebarLinkSourceHybrid2, SidebarLinkTool, SidebarLinkCustom:
SidebarLinkSourceHybrid, SidebarLinkSourceHybrid2,
SidebarLinkSourceRoot, SidebarLinkSourceAltRoot,
SidebarLinkSourceHybridRoot, SidebarLinkSourceHybrid2Root,
SidebarLinkTool, SidebarLinkCustom:
return c
}
if strings.HasPrefix(c, "source") {
@@ -36,3 +45,14 @@ func NormalizeSidebarLinkCategory(category string) string {
func IsSourceSidebarCategory(category string) bool {
return strings.HasPrefix(NormalizeSidebarLinkCategory(category), "source")
}
// SidebarLinkDiskScopeRoot reports whether the category shows the
// root-filesystem-only disk usage rather than the aggregated nested-mount usage.
func SidebarLinkDiskScopeRoot(category string) bool {
return strings.HasSuffix(NormalizeSidebarLinkCategory(category), "-root")
}
// BaseSidebarLinkCategory strips the root-only "-root" suffix from a category.
func BaseSidebarLinkCategory(category string) string {
return strings.TrimSuffix(NormalizeSidebarLinkCategory(category), "-root")
}
@@ -0,0 +1,164 @@
package users
import (
"encoding/json"
"testing"
)
func assertBoolMapEntries(t *testing.T, name string, got, want map[string]bool) {
t.Helper()
if len(got) != len(want) {
t.Fatalf("%s length = %d, want %d (got=%v want=%v)", name, len(got), len(want), got, want)
}
for key, wantVal := range want {
gotVal, ok := got[key]
if !ok {
t.Fatalf("%s missing key %q", name, key)
}
if gotVal != wantVal {
t.Fatalf("%s[%q] = %v, want %v", name, key, gotVal, wantVal)
}
}
}
func TestFrontendUserEffectiveToolAccessJSON(t *testing.T) {
t.Parallel()
u := FrontendUser{
Username: "alice",
EffectiveToolAccess: map[string]bool{
"sizeViewer": true,
"fileWatcher": false,
"activityViewer": true,
"duplicateFinder": false,
},
}
encoded, err := json.Marshal(u)
if err != nil {
t.Fatal(err)
}
var payload map[string]json.RawMessage
if err := json.Unmarshal(encoded, &payload); err != nil {
t.Fatal(err)
}
if _, ok := payload["effectiveToolAccess"]; !ok {
t.Fatalf("expected effectiveToolAccess key in JSON, got keys: %v", mapKeys(payload))
}
var decoded FrontendUser
if err := json.Unmarshal(encoded, &decoded); err != nil {
t.Fatal(err)
}
assertBoolMapEntries(t, "effectiveToolAccess", decoded.EffectiveToolAccess, u.EffectiveToolAccess)
}
func TestToolAccessMapRoundTripValidIDs(t *testing.T) {
t.Parallel()
original := ToolAccessMap{
ToolSizeViewer: true,
ToolFileWatcher: false,
ToolActivityViewer: true,
ToolDuplicateFinder: false,
ToolAdvancedSearch: true,
ToolMaterialIconPicker: false,
}
encoded, err := json.Marshal(original)
if err != nil {
t.Fatal(err)
}
var payload map[string]bool
if err := json.Unmarshal(encoded, &payload); err != nil {
t.Fatal(err)
}
if len(payload) != len(original) {
t.Fatalf("toolAccess JSON length = %d, want %d", len(payload), len(original))
}
var decoded ToolAccessMap
if err := json.Unmarshal(encoded, &decoded); err != nil {
t.Fatal(err)
}
if len(decoded) != len(original) {
t.Fatalf("toolAccess length = %d, want %d", len(decoded), len(original))
}
for id, want := range original {
got, ok := decoded[id]
if !ok {
t.Fatalf("toolAccess missing key %q", id)
}
if got != want {
t.Fatalf("toolAccess[%q] = %v, want %v", id, got, want)
}
}
}
func TestToolAccessMapRejectsUnknownToolID(t *testing.T) {
t.Parallel()
var m ToolAccessMap
err := json.Unmarshal([]byte(`{"sizeViewer":true,"notATool":false}`), &m)
if err == nil {
t.Fatal("expected unknown tool id to be rejected")
}
}
func TestUserToolAccessJSONRoundTrip(t *testing.T) {
t.Parallel()
wantToolAccess := ToolAccessMap{
ToolSizeViewer: true,
ToolFileWatcher: false,
}
u := User{
FrontendUser: FrontendUser{
Username: "bob",
ToolAccess: wantToolAccess,
},
}
encoded, err := json.Marshal(u)
if err != nil {
t.Fatal(err)
}
var payload map[string]json.RawMessage
if err := json.Unmarshal(encoded, &payload); err != nil {
t.Fatal(err)
}
if _, ok := payload["toolAccess"]; !ok {
t.Fatalf("expected toolAccess key in JSON, got keys: %v", mapKeys(payload))
}
var decoded User
if err := json.Unmarshal(encoded, &decoded); err != nil {
t.Fatal(err)
}
if len(decoded.ToolAccess) != len(wantToolAccess) {
t.Fatalf("toolAccess length = %d, want %d", len(decoded.ToolAccess), len(wantToolAccess))
}
for id, want := range wantToolAccess {
got, ok := decoded.ToolAccess[id]
if !ok {
t.Fatalf("toolAccess missing key %q", id)
}
if got != want {
t.Fatalf("toolAccess[%q] = %v, want %v", id, got, want)
}
}
}
func mapKeys(m map[string]json.RawMessage) []string {
keys := make([]string, 0, len(m))
for key := range m {
keys = append(keys, key)
}
return keys
}
@@ -0,0 +1,91 @@
package users
import (
"encoding/json"
"fmt"
)
// ToolID identifies a built-in application tool (matches frontend route segments).
type ToolID string
const (
ToolSizeViewer ToolID = "sizeViewer"
ToolDuplicateFinder ToolID = "duplicateFinder"
ToolAdvancedSearch ToolID = "advancedSearch"
ToolMaterialIconPicker ToolID = "materialIconPicker"
ToolFileWatcher ToolID = "fileWatcher"
ToolActivityViewer ToolID = "activityViewer"
)
// AllToolIDs lists every defined tool in stable catalog order.
var AllToolIDs = []ToolID{
ToolSizeViewer,
ToolDuplicateFinder,
ToolAdvancedSearch,
ToolMaterialIconPicker,
ToolFileWatcher,
ToolActivityViewer,
}
// Valid reports whether id is a known tool constant.
func (id ToolID) Valid() bool {
switch id {
case ToolSizeViewer, ToolDuplicateFinder, ToolAdvancedSearch,
ToolMaterialIconPicker, ToolFileWatcher, ToolActivityViewer:
return true
default:
return false
}
}
// ParseToolID parses s into a ToolID when it names a catalog tool.
func ParseToolID(s string) (ToolID, error) {
id := ToolID(s)
if !id.Valid() {
return "", fmt.Errorf("unknown tool id %q", s)
}
return id, nil
}
// ToolAccessMap stores per-user tool grants keyed by ToolID.
// JSON uses string keys for API compatibility.
type ToolAccessMap map[ToolID]bool
// MarshalJSON encodes the map with string keys.
func (m ToolAccessMap) MarshalJSON() ([]byte, error) {
if m == nil {
return []byte("null"), nil
}
out := make(map[string]bool, len(m))
for id, granted := range m {
if id.Valid() {
out[string(id)] = granted
}
}
return json.Marshal(out)
}
// UnmarshalJSON decodes string-keyed JSON into typed ToolID keys.
func (m *ToolAccessMap) UnmarshalJSON(data []byte) error {
if m == nil {
return fmt.Errorf("ToolAccessMap: UnmarshalJSON on nil pointer")
}
if string(data) == "null" {
*m = nil
return nil
}
var raw map[string]bool
if err := json.Unmarshal(data, &raw); err != nil {
return err
}
out := make(ToolAccessMap, len(raw))
for key, granted := range raw {
id, err := ParseToolID(key)
if err != nil {
return err
}
out[id] = granted
}
*m = out
return nil
}
@@ -21,3 +21,16 @@ func TestValidateUsernameAcceptsNormalName(t *testing.T) {
t.Fatalf("expected valid username, got %v", err)
}
}
func TestValidateUsernameRejectsUnsafeNames(t *testing.T) {
for _, username := range []string{"..", "....", ".", "-", "al..ice", "bob/", "./bob", `bob\`, "alice/docs", " alice", "alice "} {
if err := ValidateUsername(username); err == nil {
t.Fatalf("expected %q to be rejected", username)
}
}
for _, username := range []string{"alice", "first.last", "j.r.r", "a-b_c", "user@example.com", "José", "李雷", ".alice", "users"} {
if err := ValidateUsername(username); err != nil {
t.Fatalf("expected %q to be accepted, got %v", username, err)
}
}
}
+71 -31
View File
@@ -8,10 +8,27 @@ import (
)
// ProfileStorageVersion is the user version after nested profile JSON in user_data.
// It marks a storage-shape boundary (used by the persist layer), not the newest version.
const ProfileStorageVersion = 5
// NewestUserVersion is the single reference for the latest user version.
// Per-migration watermarks below keep their historical values; only this moves forward.
const NewestUserVersion = 6
// PermMigrationVersion is the user version after the legacy perm-key to permissions migration.
const PermMigrationVersion = 1
// TokenMigrationVersion is the user version after the legacy apiKeys to tokens migration.
const TokenMigrationVersion = 2
// TokenHashBackfillVersion is the user version that guarantees every stored API
// token (Tokens and legacy ApiKeys) has a SHA256(raw) → owner row in
// hashed_tokens. Strict server-side auth ignores BelongsTo and rejects any
// bearer JWT without a hash mapping, so this migration must run on startup.
const TokenHashBackfillVersion = 6
// CurrentUserMigrationVersion is persisted for newly created accounts and after legacy migrations finish.
const CurrentUserMigrationVersion = ProfileStorageVersion
const CurrentUserMigrationVersion = NewestUserVersion
type LoginMethod string
@@ -30,8 +47,10 @@ var reservedUsernames = map[string]struct{}{
strings.ToLower(AnonymousUserName): {},
}
// ValidateUsername rejects empty and reserved login names.
// ValidateUsername rejects empty and reserved login names, and names that do not map to their
// own createUserDir home directory.
func ValidateUsername(username string) error {
raw := username
username = strings.TrimSpace(username)
if username == "" {
return fmt.Errorf("username is empty")
@@ -39,6 +58,11 @@ func ValidateUsername(username string) error {
if _, reserved := reservedUsernames[strings.ToLower(username)]; reserved {
return fmt.Errorf("username %q is reserved", username)
}
// The per-user home directory is derived from CleanUsername(username), so reject names
// that it would rewrite, the names "." and "-", and names containing path separators.
if CleanUsername(raw) != raw || raw == "." || raw == "-" || strings.ContainsAny(raw, `/\`) {
return fmt.Errorf("username %q contains characters that are not allowed", raw)
}
return nil
}
@@ -77,7 +101,7 @@ type AuthToken struct {
Token string `json:"token,omitempty"`
Name string `json:"name,omitempty"`
Username string `json:"username,omitempty"`
BelongsTo uint64 `json:"belongsTo,omitempty"` // legacy stored metadata only; identity is hashed_tokens lookup
BelongsTo uint64 `json:"belongsTo,omitempty"` // legacy stored metadata only, never trusted for auth; identity is always the hashed_tokens lookup
IssuedAt int64 `json:"issuedAt,omitempty"`
ExpiresAt int64 `json:"expiresAt,omitempty"`
Permissions Permissions `json:"Permissions,omitempty"`
@@ -89,11 +113,11 @@ type MinimalAuthToken struct {
}
type Permissions struct {
Api bool `json:"api"` // allow api access
Admin bool `json:"admin"` // allow admin access
Api bool `json:"api"` // allow api access
Admin bool `json:"admin"` // allow admin access
Modify bool `json:"modify,omitempty"` // deprecated: legacy user migration only; not used for API tokens
Share bool `json:"share"` // allow sharing files
Realtime bool `json:"realtime"` // allow realtime updates
Share bool `json:"share"` // allow sharing files
Realtime bool `json:"realtime"` // allow realtime updates
Delete bool `json:"delete,omitempty"` // deprecated: legacy user migration only; not used for API tokens
Create bool `json:"create,omitempty"` // deprecated: legacy user migration only; not used for API tokens
Download bool `json:"download,omitempty"` // deprecated: legacy user migration only; not used for API tokens
@@ -102,12 +126,12 @@ type Permissions struct {
// SourceFilePermissions holds per-source file operation permissions (v4+).
type SourceFilePermissions struct {
View bool `json:"view"`
Download bool `json:"download"`
Modify bool `json:"modify"`
Delete bool `json:"delete"`
Create bool `json:"create"`
Configured bool `json:"configured,omitempty"` // true when explicitly set (allows intentional deny-all)
View bool `json:"view"`
Download bool `json:"download"`
Modify bool `json:"modify"`
Delete bool `json:"delete"`
Create bool `json:"create"`
Configured bool `json:"configured,omitempty"` // true when explicitly set (allows intentional deny-all)
}
// MarkSourceFilePermissionsConfigured marks permissions as explicitly set (including deny-all).
@@ -156,16 +180,18 @@ type Preview struct {
// FrontendUser holds fields safe to return from user APIs (embedded on User).
type FrontendUser struct {
NonAdminEditable
DisableSettings bool `json:"disableSettings"`
Username string `json:"username"`
FrontendScopes []FrontendScope `json:"scopes"`
LockPassword bool `json:"lockPassword"`
Permissions Permissions `json:"permissions"` // global: admin, api, share, realtime
SourcePermissions map[string]SourceFilePermissions `json:"sourcePermissions,omitempty"` // deprecated: use scopes[].permissions
LoginMethod LoginMethod `json:"loginMethod"`
OtpEnabled bool `json:"otpEnabled"`
ShowFirstLogin bool `json:"showFirstLogin"`
Perm Permissions `json:"perm,omitzero"`
DisableSettings bool `json:"disableSettings"`
Username string `json:"username"`
FrontendScopes []FrontendScope `json:"scopes"`
LockPassword bool `json:"lockPassword"`
RequirePasswordChange bool `json:"requirePasswordChange"`
Permissions Permissions `json:"permissions"` // global: admin, api, share, realtime
SourcePermissions map[string]SourceFilePermissions `json:"sourcePermissions,omitempty"` // deprecated: use scopes[].permissions
LoginMethod LoginMethod `json:"loginMethod"`
OtpEnabled bool `json:"otpEnabled"`
ShowFirstLogin bool `json:"showFirstLogin"`
EffectiveToolAccess map[string]bool `json:"effectiveToolAccess,omitempty"`
Perm Permissions `json:"perm,omitzero"`
}
// PinnedItems maps source filesystem path -> index directory path -> pinned item names.
@@ -183,6 +209,7 @@ type User struct {
// BackendScopes is the authoritative, persisted access list (SourceScope.Name = backend source path).
// SQLite stores this inside user_data JSON under the key "scopes" (see sqldb.UserData).
BackendScopes []BackendScope `json:"backendScopes,omitempty"`
DeclinedDefaultSources []string `json:"declinedDefaultSources,omitempty"`
BackendSourcePermissions map[string]SourceFilePermissions `json:"backendSourcePermissions,omitempty"` // key = source path
Tokens map[string]AuthToken `json:"tokens,omitempty"`
TOTPSecret string `json:"totpSecret,omitempty"`
@@ -193,21 +220,28 @@ type User struct {
UserLegacyFields `json:",inline"`
}
type ScopeQuota struct {
ID string `json:"id"`
LimitBytes int64 `json:"limitBytes"` // 0 = unlimited
Meter string `json:"meter,omitempty"` // index_scope (indexed size) | accounted (tracked usage)
}
type FrontendScope struct {
Name string `json:"name"` // Bolt: filesystem path; JSON API: display name after prepForFrontend
Scope string `json:"scope"` // index path within that source
Name string `json:"name"` // Bolt: filesystem path; JSON API: display name after prepForFrontend
Scope string `json:"scope"` // index path within that source
Permissions *SourceFilePermissions `json:"permissions,omitempty"`
Quota *ScopeQuota `json:"quota,omitempty"`
}
type BackendScope struct {
Path string `json:"path"` // real path for the source
Scope string `json:"scope"` // index path within that source
Path string `json:"path"` // real path for the source
Scope string `json:"scope"` // index path within that source
Permissions SourceFilePermissions `json:"permissions,omitempty"`
Quota *ScopeQuota `json:"quota,omitempty"`
}
// json tags must match variable name with smaller case first letter
type NonAdminEditable struct {
EditorQuickSave bool `json:"editorQuickSave"` // show quick save button in editor
HideSidebarFileActions bool `json:"hideSidebarFileActions"` // hide the file actions in the sidebar
DisableQuickToggles bool `json:"disableQuickToggles"` // disable the quick toggles in the sidebar
DisableSearchOptions bool `json:"disableSearchOptions"` // disable the search options in the search bar
@@ -216,9 +250,10 @@ type NonAdminEditable struct {
StickySidebar bool `json:"stickySidebar"` // keep sidebar open when navigating
DarkMode bool `json:"darkMode"` // should dark mode be enabled
Password string `json:"password,omitempty"`
Locale string `json:"locale"` // language to use: eg. de, en, or fr
ViewMode string `json:"viewMode"` // view mode to use: eg. normal, list, grid, or compact
SingleClick bool `json:"singleClick"` // open directory on single click, also enables middle click to open in new tab
Locale string `json:"locale"` // language to use: eg. de, en, or fr
ViewMode string `json:"viewMode"` // view mode to use: eg. normal, list, grid, or compact
NewFileTemplate []string `json:"newFileTemplate"` // list of custom filenames that will be used as template for new files
SingleClick bool `json:"singleClick"` // open directory on single click, also enables middle click to open in new tab
Sorting Sorting `json:"sorting"`
ShowHidden bool `json:"showHidden"` // show hidden files in the UI. On windows this includes files starting with a dot and windows hidden files
HideFileExt string `json:"hideFileExt"` // space separated list of file extensions to hide in UI and API
@@ -235,14 +270,19 @@ type NonAdminEditable struct {
CustomTheme string `json:"customTheme"` // Name of theme to use chosen from custom themes config.
ShowSelectMultiple bool `json:"showSelectMultiple"` // show select multiple files on desktop
ShowCopyPath bool `json:"showCopyPath"` // show copy path action in the context menu
ShowTypeColumn bool `json:"showTypeColumn"` // show the Type column in list and compact views
ShowCreationDateColumn bool `json:"showCreationDateColumn"` // show the Creation date column in list and compact views
ShowToolsInSidebar bool `json:"showToolsInSidebar"` // when false, sidebar hides links with category "tool" (default: true)
DebugOffice bool `json:"debugOffice"` // debug onlyoffice editor
SidebarLinks []SidebarLink `json:"sidebarLinks"` // customizable sidebar links
ToolAccess ToolAccessMap `json:"toolAccess,omitempty"` // per-tool access overrides
HideFilesInTree bool `json:"hideFilesInTree"` // hide files in the sidebar tree navigation, when true, will show only directories.
DeleteAfterArchive bool `json:"deleteAfterArchive"` // delete source files after successful creation/extraction of archives
PreferEditorForMarkdown bool `json:"preferEditorForMarkdown"` // prefer editor first for markdown files instead of the Markdown Viewer
PromptRightCloseButton bool `json:"promptRightCloseButton"` // show the prompts close button on the right
ShowFirstLogin bool `json:"showFirstLogin"`
PasskeyCredentials []WebAuthnCredential `json:"passkeyCredentials,omitempty"`
ShowAdvancedProfile bool `json:"showAdvancedProfile"`
}
type FileLoading struct {
Loaded 100 of 670 files, more files were not shown because too many files have changed in this diff. Show more