Compare commits

..
Author SHA1 Message Date
Graham Steffaniak 0d30433739 Merge branch 'dev/v2.2.0' into office-file-guards 2026-11-04 15:07:10 -06:00
Graham Steffaniak b45e3498b8 improve text file guards 2026-11-04 15:05:18 -06:00
Kevin B. 6aa4ffb5f8 add virtual scroll and reverse order in upload/download prompts (#3328) 2026-11-04 15:01:40 -06:00
Graham Steffaniak 609163735f added better office file previews (#3335) 2026-11-03 15:40:03 -06:00
Graham Steffaniak 7fb372623a Jpeg xl support (#3334) 2026-11-03 11:51:41 -06:00
Kevin B. 10b3c985b6 use virtual scroll for remaining lists (#3326) 2026-11-03 10:12:46 -06:00
Graham Steffaniak a6833724da make build pure go (#3329) 2026-11-03 10:08:23 -06:00
Kevin B.andGraham Steffaniak b383221a20 add virtual scrolling to listings (#3320)
Co-authored-by: Graham Steffaniak <graham.steffaniak@autodesk.com>
2026-11-02 14:06:08 -06:00
Graham Steffaniak e98aa6fc25 phase 2 build optimizations (#3319) 2026-11-02 13:42:53 -06:00
Graham Steffaniak e0730e8c7a Build optimizations (#3318) 2026-11-01 12:55:28 -06:00
Kevin B. 435ded56c6 share ListingView logic with adv search using composables (#3299) 2026-11-01 11:14:18 -06:00
Graham Steffaniak 6f54f77d64 Merge v2.1.4 v2.2.0 (#3317) 2026-11-01 11:03:25 -06:00
Graham Steffaniak 77a70cde6b introduce go-media package for faster processing (#3298) 2026-11-01 10:19:36 -06:00
Graham Steffaniak 77f5e2fe36 move ListingView and advancedSearch to composition api + TS (#3294) 2026-10-30 13:20:31 -05:00
Kevin B. 9766eee0d0 update url.js 2026-10-29 20:41:08 -05:00
Kevin B. fdb107e2bb Merge branch 'dev/v2.2.0' into adv-search-and-list-view-to-composition 2026-10-29 20:07:16 -05:00
Graham Steffaniak f9dddb117c add separate tsconfig for tests files (#3295) 2026-10-29 19:26:12 -05:00
Graham Steffaniak e17f544920 improve navigation in search results (#3275) 2026-10-29 19:16:44 -05:00
Graham Steffaniak c6cc8d56b4 feat: add wrap method option to editor config (#3282) 2026-10-29 19:10:48 -05:00
Kevin B. 802bcd8911 update vite config 2026-10-29 09:21:34 -05:00
Kevin B. db13271bf3 fix slow typecheck reruns 2026-10-29 01:24:51 -05:00
Kevin B. bb8c5f9aa8 add typecheck to CI 2026-10-29 00:04:41 -05:00
Kevin B. 968ce8d066 add separate tsconfig for tests
This to avoid tests files from failing the build if some has a type error.
2026-10-28 23:44:51 -05:00
Kevin B. 356013a137 tests to ts 2026-10-28 22:29:12 -05:00
Graham Steffaniak 7c68c38479 Add HLS media transcoding support (#3168) 2026-10-28 16:38:23 -05:00
Kevin B. dcca322f48 move ListingView and advancedSearch to composition api + TS
This is also part of the changes to create composables for adv search and listing view.

- Updated the tests that were calling functions directly, which don’t exist with <script setup>
- Added missing types.

All should still behaving the same as before.
2026-10-28 15:58:31 -05:00
Kevin B. 4a6e8edc28 use single wrap dropdown 2026-10-28 12:44:02 -05:00
Kevin B. d9c91e1125 Merge branch 'dev/v2.2.0' into feat/editor-wrap-method 2026-10-28 11:23:58 -05:00
Kevin B. 4cdf1e71e4 address coderabbit comment 2026-10-27 22:50:03 -05:00
Kevin B. 9e6c2cd0a3 add playwright tests 2026-10-27 22:19:42 -05:00
Kevin B. c102fe7556 add natural ordering to quick search 2026-10-27 21:28:42 -05:00
Kevin B. d0591b9de4 address coderabbit review 2026-10-27 21:28:42 -05:00
Kevin B. ebe47ce643 improve navigation in search results
- sort header sticky in adv search
- next/previous navigation follows search results of both quick and adv search
- closing preview from quick or adv search shows the previous results and scrolls to the item we just closed
  like the normal listing
- fix previous sorting don't persisting in adv search
2026-10-27 21:28:42 -05:00
Graham Steffaniak fbbf40f877 Fix styling v2.2.0 (#3257) 2026-10-27 19:30:37 -05:00
Kevin B. 1508d9ff21 Improve caches and listing navigation (#3193) 2026-10-27 19:30:03 -05:00
Kevin B. 0e318d6eaa feat: add wrap method option to editor config 2026-10-26 11:47:18 -05:00
Emmanuel Salomon fd64a470d3 fix: sign Collabora sessions with a key derived from auth.key only (#3173) 2026-10-21 18:50:56 -05:00
+5 751cc56809 Merge main v2.2.0 v2.1.3 (#3248)
Co-authored-by: Paolo Valletta <11146190+paoValle@users.noreply.github.com>
Co-authored-by: Jay Hemnani <193022578+jayhemnani9910@users.noreply.github.com>
Co-authored-by: vulragrag-star <320132805+vulragrag-star@users.noreply.github.com>
Co-authored-by: Anatolii Prylutskyi <5327697+anpryl@users.noreply.github.com>
Co-authored-by: Kevin B. <178341547+Kurami32@users.noreply.github.com>
Co-authored-by: Som Samantray <92726151+SomSamantray@users.noreply.github.com>
Co-authored-by: Kevin B. <kurami32_kbm@proton.me>
Co-authored-by: HawkForge <75696486+HawkForge@users.noreply.github.com>
Co-authored-by: JUSHUANGHUI LI <71743926+oodadoudou@users.noreply.github.com>
Co-authored-by: anonymous-writer <35840715+anonymous-writer@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Graham Steffaniak <gtsteffaniak@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-21 12:55:25 -05:00
Graham Steffaniak c5112cd51c feat: edit office files in Collabora Online through WOPI (#3108) 2026-10-15 19:18:05 -05:00
Graham Steffaniak 90354e468d Feat/quick search fts5 (#3146) 2026-10-14 16:37:19 -05:00
Graham Steffaniak c35e4f33c8 updated config message 2026-10-13 18:47:50 -05:00
Graham SteffaniakandDevin 474cd90e74 Invalidate in-flight search as soon as the query changes
During the 250ms debounce window the sequence counter was not yet bumped,
so a response for the previous query could still pass the stale-response
guard and briefly repopulate results. abortSearch() now bumps the sequence
so any late response is ignored, and onInput() calls it before scheduling
the debounced search.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 18:44:59 -05:00
Graham SteffaniakandDevin a483504426 Fix the "no results" message after debouncing quick search
The debounce refactor moved the in-flight counter decrement into a
finally block, so the "no results" check ran while the request was still
counted as in flight and the message never appeared. Evaluate it after
the counter drops, and only for a request that actually completed.

Fixes the settings access.spec.ts search assertion.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 18:43:59 -05:00
Graham SteffaniakandDevin 36dba16fd6 Make the FTS5 search index opt-in via indexSqlConfig.fts5
FTS5 speeds up substring searches but costs roughly 40% more indexing
time and 15% more index file size, so it now defaults to off.

- Add indexSqlConfig.fts5 (default false). When off, search uses the
  existing unindexed query and no FTS table or triggers are created.
- When enabled, the index is built from existing rows on startup (no
  rescan); disabling it drops the table and triggers.
- Regenerate config.yaml and swagger docs.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 18:24:14 -05:00
Graham SteffaniakandDevin 5f3049ebc6 Debounce quick search and add an FTS5 trigram index
Quick search sent a request on every keystroke and each one ran an
unindexed lower(name) GLOB scan, so searches piled up and the last one
finished last. This:

- debounces the search input by 250ms, aborts the superseded request, and
  ignores late responses so the UI cannot show stale results;
- passes the request context into the query (15s timeout) so an abandoned
  search is interrupted instead of holding the index DB connection;
- removes the session-cancellation map, which could not interrupt the scan;
- adds one global FTS5 trigram index over (name, source:path) so substring
  searches use an index. It is derived data: a schema or health mismatch
  deletes and rebuilds the index DB, and it falls back to the old query
  when FTS5 is unavailable.

Also fixes GLOB escaping: SQLite GLOB has no backslash escape, so literal
* ? [ in a search term and in the scope prefix now use bracket classes.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 16:51:35 -05:00
manu 74923fbe1b test: view-only session can still read the file through GetFile 2026-10-12 10:48:37 +02:00
manu 59bfdbcb54 fix: allow printing with view permission, as OnlyOffice does
The OnlyOffice integration lets a user print with View alone; the WOPI
editor disabled printing without Download. Align them, so the choice of
editor does not change what a user may do. Export and copy still need
Download.
2026-10-10 04:21:20 +02:00
manu b079dd19e7 fix: match only advertised office extensions
Object.hasOwn instead of `in`, which also matched inherited properties:
a file named report.constructor would have been routed to the editor.

Also correct the serveWopi comment and test name: a moved file is
rejected, a file replaced in place is reported as changed in storage.
2026-10-10 04:21:20 +02:00
manu b3cbd0925e refactor: configure Collabora through integrations.office
Review follow-up on #3108.

- There is no integrations.wopi block any more: integrations.office gains
  product (onlyoffice, the default, or collabora) and tokenExpirationHours.
  secret stays required for OnlyOffice and becomes optional for Collabora.
  Existing OnlyOffice configs are unchanged. With product collabora, the
  OnlyOffice-specific paths (document keys, previews via /converter, CSP
  script origins, onlyOfficeUrl) stand down.
- The SPA receives officeProduct and officeExtensions instead of the wopi*
  globals. WOPI stays the name of the protocol code and routes.
- Discovery is refreshed in the background until shutdown: every 30s while
  it has never loaded (the editor often starts after FileBrowser), then every
  10 minutes. Before, a failed first fetch left Collabora disabled until
  restart.
- WopiEditor shows its close button when the session request fails, instead
  of leaving the user on a blank view.
- Fix the router import broken by the upstream switch to a named export.
2026-10-10 04:21:20 +02:00
manu 9753cfbf8b feat: edit office files in Collabora Online through WOPI
FileBrowser gains a WOPI host, so office documents can open in Collabora
Online (or any WOPI editor, OnlyOffice Docs included) next to the existing
OnlyOffice integration. Refs #641.

- integrations.wopi: url, internalUrl, secret, product, tokenExpirationHours,
  viewOnly. Discovery is fetched from the editor and cached; extensions it
  declares open in the editor, except formats FileBrowser already views
  itself (images, plain text).
- GET /api/wopi/session returns the editor URL and a signed access token
  bound to one file, one user and a mode. The token has its own audience and
  a key derived from the auth key (or integrations.wopi.secret), so it never
  doubles as a session token.
- /wopi/files/{id}[/contents]: CheckFileInfo, GetFile, PutFile and the lock
  operations. Permissions are re-checked on every call, so revoking access
  takes effect mid-session. PutFile receives the whole body before writing,
  and answers Collabora's COOLStatusCode 1010 when the file changed in
  storage since it was loaded, letting the user overwrite or reload.
- access_token is redacted from request logs.
- WopiEditor.vue form-posts the token into an iframe and handles UI_Close.
  Public shares keep using OnlyOffice for now.
2026-10-10 04:21:20 +02:00
559 changed files with 43402 additions and 11552 deletions

No files matched your search

-1
View File
@@ -9,7 +9,6 @@ backend/filebrowser.sqlite*
backend/tmp
backend/dist
backend/backend*
backend/internal/web/dist/**
backend/vendor/**
frontend/dist/**
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
# Builds the comma-separated Docker tag list for a release image.
#
# Usage: docker-release-tags.sh <metadata.json> <variant> <update_track> <update_major>
# metadata.json JSON output of docker/metadata-action (uses .tags)
# variant default | slim | media
# default and media: floating tags include latest
# slim: every tag gets a -slim suffix and latest is not added
# update_track true when the semver policy allows floating beta/stable/latest tags
# update_major true when the semver policy allows major-only version tags
#
# Image repositories are fixed per variant so that each image only gets its own tags.
set -euo pipefail
if [[ $# -ne 4 ]]; then
echo "usage: $0 <metadata.json> <default|slim|media> <true|false> <true|false>" >&2
exit 2
fi
metadata_file=$1
variant=$2
update_track=$3
update_major=$4
case "$variant" in
default)
suffix=""
add_latest=true
repos=(gtstef/filebrowser ghcr.io/gtsteffaniak/filebrowser)
;;
slim)
suffix="-slim"
add_latest=false
repos=(gtstef/filebrowser ghcr.io/gtsteffaniak/filebrowser)
;;
media)
suffix=""
add_latest=true
repos=(gtstef/filebrowser-media ghcr.io/gtsteffaniak/filebrowser-media)
;;
*)
echo "unknown variant: $variant" >&2
exit 2
;;
esac
jq -r \
--arg suffix "$suffix" \
--argjson addLatest "$add_latest" \
--argjson updateTrack "$update_track" \
--argjson updateMajor "$update_major" \
--arg dh "${repos[0]}" \
--arg gh "${repos[1]}" '
# Strip the v prefix, drop metadata latest, and append the variant suffix.
.tags |= map(
sub(":v"; ":")
| select(test(":latest") | not)
| . + $suffix
) |
# Version tags with a prerelease suffix (X.Y.Z-suffix) get major.minor, and major when allowed.
.tags += [
.tags[]
| select(test(":[0-9]+\\.[0-9]+\\.[0-9]+-"))
| split(":")
| .[-1] as $version
| ($version | split("-")) as $version_split
| ($version_split[0] | split(".")) as $version_parts
| ($version_split[1:] | join("-")) as $tag_suffix
| ($version_parts | "\(.[0]).\(.[1])") as $major_minor
| ($version_parts[0]) as $major
| "\($dh):\($major_minor)-\($tag_suffix)",
"\($gh):\($major_minor)-\($tag_suffix)",
if $updateMajor then
"\($dh):\($major)-\($tag_suffix)",
"\($gh):\($major)-\($tag_suffix)"
else empty end
] |
# Floating beta tag.
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-beta" + $suffix)) then
.tags += ["\($dh):beta\($suffix)", "\($gh):beta\($suffix)"]
else . end |
# Floating stable tag, plus latest for variants that have it.
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-stable" + $suffix)) then
.tags += (
["\($dh):stable\($suffix)", "\($gh):stable\($suffix)"]
+ (if $addLatest then ["\($dh):latest", "\($gh):latest"] else [] end)
)
else . end |
.tags | join(",")
' "$metadata_file"
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
with:
node-version-file: frontend/.nvmrc
- working-directory: frontend
run: npm i && npm run build
run: npm ci && npm run build
- uses: actions/setup-go@v5
with:
go-version: 'stable'
+29 -6
View File
@@ -37,7 +37,7 @@ jobs:
node-version-file: frontend/.nvmrc
- name: Build frontend
working-directory: frontend
run: npm i && npm run build
run: npm ci && npm run build
- name: Upload frontend build artifact
uses: actions/upload-artifact@v4
with:
@@ -97,7 +97,7 @@ jobs:
uses: actions/download-artifact@v4
with:
name: dist
path: frontend/dist
path: backend/internal/web/dist
- name: Download backend build
uses: actions/download-artifact@v4
with:
@@ -105,10 +105,6 @@ jobs:
path: backend/
- name: Make backend executable
run: chmod +x backend/filebrowser
- name: Copy frontend build to backend
run: |
mkdir -p backend/internal/web/embed
cp -r frontend/dist/* backend/internal/web/embed
- name: Set up QEMU
uses: docker/setup-qemu-action@v3.0.0
- name: Set up Docker Buildx
@@ -166,3 +162,30 @@ jobs:
REVISION=${{ steps.meta.outputs.revision }}
cache-from: type=gha
cache-to: type=gha,mode=max
build_media_docker_image:
name: Build media docker image
runs-on: ubuntu-24.04-arm
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3.0.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3.0.0
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./_docker/Dockerfile.media
push: false # Do not push the image for now
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
VERSION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.version'] }}
REVISION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
cache-from: type=gha,scope=pr-media
cache-to: type=gha,mode=max,scope=pr-media
+14 -4
View File
@@ -47,7 +47,7 @@ jobs:
with:
node-version-file: frontend/.nvmrc
- working-directory: frontend
run: npm i && npm run lint
run: npm ci && npm run lint
test-frontend:
runs-on: ubuntu-24.04-arm
timeout-minutes: 5
@@ -57,7 +57,17 @@ jobs:
with:
node-version-file: frontend/.nvmrc
- working-directory: frontend
run: npm i && npm run test
run: npm ci && npm run test
typecheck-frontend:
runs-on: ubuntu-24.04-arm
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: frontend/.nvmrc
- working-directory: frontend
run: npm ci && npm run typecheck
check-translations:
runs-on: ubuntu-24.04-arm
timeout-minutes: 5
@@ -67,7 +77,7 @@ jobs:
with:
node-version-file: frontend/.nvmrc
- working-directory: frontend
run: npm i && npm run i18n:check
run: npm ci && npm run i18n:check
check-docs:
runs-on: ubuntu-24.04-arm
timeout-minutes: 5
@@ -87,4 +97,4 @@ jobs:
with:
node-version-file: frontend/.nvmrc
- working-directory: frontend
run: npm i && npm run icons:check
run: npm ci && npm run icons:check
+70 -90
View File
@@ -15,16 +15,20 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: backend/go.mod
cache-dependency-path: backend/go.sum
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version-file: frontend/.nvmrc
cache: npm
cache-dependency-path: frontend/package-lock.json
- name: Install dependencies and build frontend
run: npm i && npm run build
run: npm ci && npm run build
working-directory: frontend
- name: Install OS Requirements
run: sudo apt update && sudo apt install -y upx build-essential musl-tools
- uses: goto-bus-stop/setup-zig@v2
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
@@ -68,50 +72,8 @@ jobs:
id: modify-json
run: |
JSON='${{ steps.meta.outputs.json }}'
MODIFIED_JSON=$(echo "$JSON" | jq \
--argjson updateTrack '${{ steps.floating.outputs.update_track == 'true' }}' \
--argjson updateMajor '${{ steps.floating.outputs.update_major == 'true' }}' '
.tags |= map(
sub(":v"; ":") # Strip the 'v'
| select(test(":latest") | not)
) |
# Find all version tags with suffix (X.Y.Z-suffix) and add major.minor (always) and major tags (when allowed)
.tags += [
.tags[]
| select(test(":[0-9]+\\.[0-9]+\\.[0-9]+-"))
| split(":")
| .[-1] as $version
| ($version | split("-")) as $version_split
| ($version_split[0] | split(".")) as $version_parts
| ($version_split[1:] | join("-")) as $suffix
| ($version_parts | "\(.[0]).\(.[1])") as $major_minor
| ($version_parts[0]) as $major
| "gtstef/filebrowser:\($major_minor)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major_minor)-\($suffix)",
if $updateMajor then
"gtstef/filebrowser:\($major)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major)-\($suffix)"
else empty end
] |
# Add suffix-only tags for beta when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-beta")) then
.tags += [
"gtstef/filebrowser:beta",
"ghcr.io/gtsteffaniak/filebrowser:beta"
]
else . end |
# Add suffix-only tags for stable (and latest) when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-stable")) then
.tags += [
"gtstef/filebrowser:stable",
"ghcr.io/gtsteffaniak/filebrowser:stable",
"gtstef/filebrowser:latest",
"ghcr.io/gtsteffaniak/filebrowser:latest"
]
else . end
')
TAGS_CSV=$(echo "$MODIFIED_JSON" | jq -r '.tags | join(",")')
echo "$TAGS_CSV" > modified_tags.txt # Optional debug output
printf '%s' "$JSON" > metadata.json
TAGS_CSV=$(bash .github/scripts/docker-release-tags.sh metadata.json default '${{ steps.floating.outputs.update_track }}' '${{ steps.floating.outputs.update_major }}')
echo "cleaned_tag=$TAGS_CSV" >> $GITHUB_OUTPUT
- name: Build and push
uses: docker/build-push-action@v6
@@ -125,6 +87,62 @@ jobs:
push: true
tags: ${{ steps.modify-json.outputs.cleaned_tag }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=release-main
cache-to: type=gha,mode=max,scope=release-main
push_media_release_to_registry:
name: Push media release
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3.0.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3.0.0
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{github.actor}}
password: ${{ secrets.PAT }}
- name: Extract metadata (tags, labels) for Docker and GHCR
id: meta
uses: docker/metadata-action@v5
with:
images: |
gtstef/filebrowser-media
ghcr.io/gtsteffaniak/filebrowser-media
- name: Decide floating Docker tags
id: floating
env:
IMAGE_REPO: gtstef/filebrowser-media
run: bash .github/scripts/docker-floating-tag-policy.sh
- name: Modify tags (strip 'v' and add version tags)
id: modify-json
run: |
JSON='${{ steps.meta.outputs.json }}'
printf '%s' "$JSON" > metadata.json
TAGS_CSV=$(bash .github/scripts/docker-release-tags.sh metadata.json media '${{ steps.floating.outputs.update_track }}' '${{ steps.floating.outputs.update_major }}')
echo "cleaned_tag=$TAGS_CSV" >> $GITHUB_OUTPUT
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
build-args: |
VERSION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.version'] }}
REVISION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
platforms: linux/amd64,linux/arm64
file: ./_docker/Dockerfile.media
push: true
tags: ${{ steps.modify-json.outputs.cleaned_tag }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=release-media
cache-to: type=gha,mode=max,scope=release-media
push_slim_release_to_registry:
name: Push slim release
runs-on: ubuntu-latest
@@ -162,48 +180,8 @@ jobs:
id: modify-json-slim
run: |
JSON='${{ steps.meta.outputs.json }}'
MODIFIED_JSON=$(echo "$JSON" | jq \
--argjson updateTrack '${{ steps.floating.outputs.update_track == 'true' }}' \
--argjson updateMajor '${{ steps.floating.outputs.update_major == 'true' }}' '
.tags |= map(
sub(":v"; ":") # Strip the 'v'
| select(test(":latest") | not) # Remove the "latest" tag
| . + "-slim" # Append "-slim" to the versioned tag
) |
# Find all version tags with suffix (X.Y.Z-suffix-slim) and add major.minor (always) and major tags (when allowed)
.tags += [
.tags[]
| select(test(":[0-9]+\\.[0-9]+\\.[0-9]+-"))
| split(":")
| .[-1] as $version
| ($version | split("-")) as $version_split
| ($version_split[0] | split(".")) as $version_parts
| ($version_split[1:] | join("-")) as $suffix
| ($version_parts | "\(.[0]).\(.[1])") as $major_minor
| ($version_parts[0]) as $major
| "gtstef/filebrowser:\($major_minor)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major_minor)-\($suffix)",
if $updateMajor then
"gtstef/filebrowser:\($major)-\($suffix)",
"ghcr.io/gtsteffaniak/filebrowser:\($major)-\($suffix)"
else empty end
] |
# Add suffix-only tags for beta-slim when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-beta-slim")) then
.tags += [
"gtstef/filebrowser:beta-slim",
"ghcr.io/gtsteffaniak/filebrowser:beta-slim"
]
else . end |
# Add suffix-only tags for stable-slim when semver policy allows
if $updateTrack and any(.tags[]; split(":")[-1] | endswith("-stable-slim")) then
.tags += [
"gtstef/filebrowser:stable-slim",
"ghcr.io/gtsteffaniak/filebrowser:stable-slim"
]
else . end
')
TAGS_CSV=$(echo "$MODIFIED_JSON" | jq -r '.tags | join(",")')
printf '%s' "$JSON" > metadata.json
TAGS_CSV=$(bash .github/scripts/docker-release-tags.sh metadata.json slim '${{ steps.floating.outputs.update_track }}' '${{ steps.floating.outputs.update_major }}')
echo "cleaned_tag=$TAGS_CSV" >> $GITHUB_OUTPUT
- name: Build and push slim image
uses: docker/build-push-action@v6
@@ -217,3 +195,5 @@ jobs:
push: true
tags: ${{ steps.modify-json-slim.outputs.cleaned_tag }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=release-slim
cache-to: type=gha,mode=max,scope=release-slim
+2 -2
View File
@@ -43,8 +43,8 @@ frontend/.tmp-icons-*
/backend/*test_config.yaml
/backend/generated.yaml
/backend/srv
/backend/internal/web/dist
/backend/internal/web/embed
/backend/internal/web/dist/*
!/backend/internal/web/dist/.gitkeep
/backend/debug
/backend/http
+53
View File
@@ -2,6 +2,59 @@
All notable changes to this project will be documented in this file. For commit guidelines, please refer to [Standard Version](https://github.com/conventional-changelog/standard-version).
## v2.2.0
**New Features**:
- Core web assets are loaded into memory instead of served from disk.
- Media enhancements
- All server media settings can be managed in settings by admins.
- JXL image preview support (#3334) (#2551)
- Transcoding for media playback with quality presets -- requires ffmpeg.
- Hardware Acceleration device selector (Software / detected GPU devices)
- More operations use native api operations rather than ffmpeg, improving performance in many cases. Added temporary config option to force ffmpeg to be used
- Document enhancements
- Official Collabora support using WOPI config.
- Document previews use a pure-go in-process package (all versions get it now).
- Added **Always use office for document previews** option to force OnlyOffice/Collabora.
- Office previews can use embedded thumbnail if possible.
- Text file previews are more efficient direct creation.
- New text preview API (`/api/resources/preview/text`) returns a 16KB text preview for text, Markdown, HTML, CSS, code and RTF files. Markdown and HTML render in the listing as sanitized previews. Previews are not cached on disk. SVG files are served directly as images.
- Text previews only render inline on tiles at least 60px wide, so narrower tiles (list, compact, normal) show the file-type icon with a `saved_search` badge instead when popup previews are enabled, and hovering still opens the larger popup preview. The text preview endpoint is now bounded by its own concurrency limiter (`server.numImageProcessors`), and rendered preview HTML is cached in the browser so scrolling and hovering do not re-parse it.
- Embedded previews for Krita (`.kra`, [#2431](https://github.com/gtsteffaniak/filebrowser/issues/2431)), OpenRaster (`.ora`), Sketch (`.sketch`), MuseScore (`.mscz`), CBZ (first page), XPS (embedded thumbnail, when present), ODF (`.odt`, `.ods`, `.odp`, `.odg`), and Office Open XML (`.docx`, `.xlsx`, `.pptx`) thumbnails.
- Virtual scrolling for large lists: like the main listing, advanced search, file tree, etc. Only render the rows on screen, so very large folders stay responsive, resolving some performance issues (#2879) (#1773) (#3320) (#3326).
- Search and navigation improvements:
- Next/previous in a preview follows the results of quick search and advanced search, playback queue also uses if there's media (#3275).
- Closing a preview opened from quick or advanced search returns to the results and scrolls to the item like the normal listing (#3275).
- Advanced search now has almost all the features of the normal listing, including rectangle selection, keyboard navigation, etc (#3299).
- Quick search results use natural ordering.
- Listings now are cached (with a cap) allowing a more responsive navigation between folders (#3193).
- Upload and download prompts list is reversed (#2998) (#3328).
- Replaced `indented soft wrap` editor setting by a dropdown with more wrap settings (#3183) (#3282).
**Notes**:
- **Document preview formats:** New in-process support includes **`.azw3`**, **`.chm`**, **`.htm`**, and **`.html`**. However, `.doc` continues to require the office integration for thumbnails. **`.xps`** and **`.cbz`** previews come from embedded images. With **Always use office for document previews** enabled, office files still use the office converter
- UPX compression is removed from release builds.
- A `filebrowser-media` image is published alongside the main and slim images.
- Previews (thumbnails, popup, and text previews) are no longer requested for sources the user does not have view access to.
**Bug Fixes**:
- Closing a video preview from the header bar now stops playback immediately instead of leaving audio running in the background.
- Closing a preview no longer navigates back to the same file when a stale "previous item" was remembered from an earlier navigation.
- Transcoding recovers from unreliable hardware decoding without discarding the segments already produced: the retry resumes at the encode head with software decode instead of wiping the generation and re-encoding from the start.
- The player no longer aborts a segment request after 10 seconds while the transcode producer is repositioning or restarting, which previously surfaced as a 503 and a skipped segment.
- getDirectoryPreview no longer requests metadata, so folder previews skip the per-file probe and subtitle scan.
- Media probes are cached by file identity and concurrent reads are coalesced, so opening a file no longer triggers repeated ffprobe runs.
- Client-initiated stream cancels (seek/navigation) no longer log spurious write-error responses.
- Gzipped `.woff` font files are no longer generated without being served; only `.js` and `.woff2` are precompressed.
- Text preview tiles no longer show the bottom truncation fade; truncated content is clipped at the tile edge instead.
- Text preview tiles lay out at a minimum width and are clipped, instead of wrapping aggressively on narrow tiles.
- Advanced search no longer loses the previously selected sorting (#3275)
- Context menu in iOS don't opening in Advanced Search (#2776) (#3299)
- Improved mobile layout of inputs in search filters.
- Doing actions in the main list/adv search, like move, copy, delete, or create a new file no longer reloads the entire listing, now updates in place and doesn't lost the position you were browsing at (#3193).
- Uploading lot of files freezing the UI for a few seconds (#3321).
- Directory uploads are prioritized, and files wait until their parent is ready (#3221).
## v2.1.4
**Security**:
+1 -4
View File
@@ -16,10 +16,7 @@ Thank you for your interest in contributing to FileBrowser Quantum! This guide w
- macOS: `brew install ffmpeg`
- Windows: Download from [ffmpeg.org](https://ffmpeg.org/download.html)
- **mupdf-tools**: For PDF preview generation
- Ubuntu/Debian: `sudo apt-get install mupdf-tools`
- macOS: `brew install mupdf-tools`
- Windows: Download from [mupdf.com](https://mupdf.com/downloads/)
Document thumbnails (PDF, EPUB, Office Open XML, and similar) are generated in-process via [folio](https://github.com/gen2brain/folio). An optional office integration is used as a fallback for formats folio does not handle and for in-browser editing.
## Quick Start
+8 -7
View File
@@ -1,24 +1,25 @@
FROM gtstef/ffmpeg:9.0.2-decode AS ffmpeg
# syntax=docker/dockerfile:1.4
FROM gtstef/ffmpeg:9.0.2 AS ffmpeg
FROM golang:alpine AS base
ARG VERSION
ARG REVISION
WORKDIR /app
COPY ./backend/go.mod ./backend/go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY ./backend ./
RUN apk update && apk add --no-cache gcc musl-dev
RUN go mod download && go run ./scripts/setup-gofitz-cgo
ENV CGO_ENABLED=1
RUN go build -tags analytics,mupdf,musl -ldflags="-w -s \
ENV CGO_ENABLED=0
RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build go build -trimpath -tags analytics -ldflags="-w -s \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=${VERSION}' \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA=${REVISION}'" \
-o filebrowser .
FROM node:jod-slim AS nbuild
FROM --platform=$BUILDPLATFORM node:jod-slim AS nbuild
WORKDIR /build
COPY ./frontend/package.json ./frontend/package-lock.json ./frontend/
RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates \
&& git config --global url."https://github.com/".insteadOf "git@github.com:" \
&& git config --global url."https://github.com/".insteadOf "ssh://git@github.com/"
RUN cd frontend && npm ci --maxsockets 1
RUN --mount=type=cache,target=/root/.npm cd frontend && npm ci --maxsockets 1
COPY ./frontend/ ./frontend/
WORKDIR /build/frontend
RUN npm run build:docker
+44
View File
@@ -0,0 +1,44 @@
# syntax=docker/dockerfile:1.4
FROM golang:alpine AS base
ARG VERSION
ARG REVISION
WORKDIR /app
COPY ./backend/go.mod ./backend/go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY ./backend ./
ENV CGO_ENABLED=0
RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build go build -trimpath -tags analytics -ldflags="-w -s \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=${VERSION}' \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA=${REVISION}'" \
-o filebrowser .
FROM --platform=$BUILDPLATFORM node:jod-slim AS nbuild
WORKDIR /build
COPY ./frontend/package.json ./frontend/package-lock.json ./frontend/
RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates
RUN --mount=type=cache,target=/root/.npm cd frontend && npm ci --maxsockets 1
COPY ./frontend/ ./frontend/
WORKDIR /build/frontend
RUN npm run build:docker
FROM alpine:latest
ENV FILEBROWSER_DATABASE_PATH="/home/filebrowser/data/database.sqlite"
ENV FILEBROWSER_CONFIG="/home/filebrowser/data/config.yaml"
ENV PATH="$PATH:/home/filebrowser"
RUN apk --no-cache add ca-certificates mailcap tzdata curl
RUN adduser -D -s /bin/true -u 1000 filebrowser
USER filebrowser
WORKDIR /home/filebrowser
# copy config to two locations for compatibility
COPY --from=base --chown=filebrowser:1000 [ "/app/filebrowser", "./" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/config.yaml", "./data/config.yaml" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/config.yaml", "./" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/reduce-rounded-corners.css", "./" ]
COPY --from=nbuild --chown=filebrowser:1000 [ "/build/backend/internal/web/dist/", "./internal/web/dist/" ]
## sanity checks
RUN [ "filebrowser", "version" ]
# exposing default port for auto discovery.
EXPOSE 80
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
CMD curl -f http://localhost:80/health || exit 1
ENTRYPOINT [ "./filebrowser" ]
+78
View File
@@ -0,0 +1,78 @@
# syntax=docker/dockerfile:1.4
# GPU-capable media image: mostly-static glibc FFmpeg + dynamic VA/Vulkan/OpenCL drivers (amd64).
# Codecs/TLS are linked into gtstef/ffmpeg:<ver> (full track, not *-static).
#
# amd64: libva/Mesa/Vulkan/OpenCL userspace below; NVIDIA libs come from the host toolkit.
# arm64 (e.g. Raspberry Pi): h264_v4l2m2m uses kernel V4L2 only — no extra GPU .debs; pass
# /dev/video10 or /dev/video11 (and group video). Pi 5 VAAPI via /dev/dri is optional (mesa on arm64).
FROM gtstef/ffmpeg:9.0.2 AS ffmpeg
FROM golang:bookworm AS base
ARG VERSION
ARG REVISION
WORKDIR /app
COPY ./backend/go.mod ./backend/go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY ./backend ./
ENV CGO_ENABLED=0
RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build go build -trimpath -tags analytics -ldflags="-w -s \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=${VERSION}' \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA=${REVISION}'" \
-o filebrowser .
FROM --platform=$BUILDPLATFORM node:jod-slim AS nbuild
WORKDIR /build
COPY ./frontend/package.json ./frontend/package-lock.json ./frontend/
RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates
RUN --mount=type=cache,target=/root/.npm cd frontend && npm ci --maxsockets 1
COPY ./frontend/ ./frontend/
WORKDIR /build/frontend
RUN npm run build:docker
FROM debian:bookworm-slim AS runtime
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
curl \
mailcap \
tzdata \
&& rm -rf /var/lib/apt/lists/* \
&& (dpkg --print-architecture | grep -q amd64 \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
libva2 \
libva-drm2 \
libdrm2 \
libvulkan1 \
mesa-va-drivers \
mesa-vulkan-drivers \
ocl-icd-libopencl1 \
intel-media-va-driver \
libmfx1 \
&& rm -rf /var/lib/apt/lists/* \
|| true)
COPY --from=ffmpeg [ "/ffmpeg", "/ffprobe", "/usr/local/bin/" ]
ENV FILEBROWSER_FFMPEG_PATH="/usr/local/bin/"
ENV FILEBROWSER_DATABASE_PATH="/home/filebrowser/data/database.sqlite"
ENV FILEBROWSER_CONFIG="/home/filebrowser/data/config.yaml"
ENV PATH="$PATH:/home/filebrowser"
ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,video,utility,graphics
RUN useradd -m -u 1000 -s /usr/sbin/nologin filebrowser
USER filebrowser
WORKDIR /home/filebrowser
COPY --from=base --chown=filebrowser:1000 [ "/app/filebrowser", "./" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/config.yaml", "./data/config.yaml" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/config.yaml", "./" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/reduce-rounded-corners.css", "./" ]
COPY --from=nbuild --chown=filebrowser:1000 [ "/build/backend/internal/web/dist/", "./internal/web/dist/" ]
RUN filebrowser version && ffmpeg -version && ffprobe -version
EXPOSE 80
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
CMD curl -f http://localhost:80/health || exit 1
ENTRYPOINT [ "./filebrowser" ]
+1 -1
View File
@@ -12,7 +12,7 @@ WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
COPY [ "./backend/reduce-rounded-corners.css", "./no-rounded.css" ]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# generate duplicate 1.1MB files
RUN dd if=/dev/zero of=/app/frontend/tests/playwright-files/1.1MB.bin bs=1024 count=1126 && dd if=/dev/zero of=/app/frontend/tests/playwright-files/1.1mb.bin bs=1024 count=1126
+1 -1
View File
@@ -6,7 +6,7 @@ COPY [ "./frontend/tests", "./tests" ]
WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
RUN apt update && apt install nginx -y
RUN mv default.conf /etc/nginx/conf.d/default.conf
RUN sed -i 's/filebrowser/localhost/g' /etc/nginx/conf.d/default.conf
+1 -1
View File
@@ -11,7 +11,7 @@ COPY [ "./_docker/src/settings/backend/config.yaml", "./"]
COPY [ "./_docker/src/settings/backend/database.db.old", "./"]
COPY [ "./backend/reduce-rounded-corners.css", "./no-rounded.css"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
RUN ./filebrowser user set testuser1 --password testuser1
+1 -1
View File
@@ -11,7 +11,7 @@ COPY [ "./frontend/tests/", "./tests" ]
WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# Run tests
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
+1 -1
View File
@@ -11,7 +11,7 @@ COPY [ "./frontend/tests/playwright-files", "./tests/playwright-files2" ]
WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# Run tests
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
+1 -1
View File
@@ -7,7 +7,7 @@ WORKDIR /app/backend/
RUN mkdir -p ./internal/web/dist
COPY [ "./_docker/src/oidc/backend/", "./"]
COPY ["./backend/filebrowser", "./"]
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# Run tests
WORKDIR /app/frontend
+1 -1
View File
@@ -23,7 +23,7 @@ WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
COPY [ "./backend/reduce-rounded-corners.css", "./no-rounded.css" ]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
ENV CI=true
+4 -2
View File
@@ -1,5 +1,8 @@
FROM gtstef/ffmpeg:9.0.2 AS ffmpeg
FROM ghcr.io/gtsteffaniak/playwright-base:chromium
WORKDIR /app
COPY --from=ffmpeg [ "/ffmpeg", "/ffprobe", "/usr/local/bin/" ]
# Copy setup scripts first to leverage caching
COPY [ "./_docker/src/previews/", "./" ]
@@ -10,8 +13,7 @@ COPY [ "./frontend/tests", "./tests" ]
WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# Run tests
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
RUN ./filebrowser & sleep 2 && cd ../frontend && npx playwright test
+1 -1
View File
@@ -6,7 +6,7 @@ COPY [ "./frontend/tests", "./tests" ]
WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
RUN apt update && apt install nginx -y
RUN mv default.conf /etc/nginx/conf.d/default.conf
RUN mv .htpasswd /etc/nginx/conf.d/.htpasswd
+1 -1
View File
@@ -10,7 +10,7 @@ COPY [ "./backend/filebrowser", "./"]
COPY [ "./_docker/src/screenshots/backend/*", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
RUN ./filebrowser & sleep 2 && cd ../frontend && npx playwright test --project dark-screenshots
+1 -1
View File
@@ -13,7 +13,7 @@ WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
COPY [ "./backend/reduce-rounded-corners.css", "./no-rounded.css"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# Run tests
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
+1 -1
View File
@@ -11,7 +11,7 @@ COPY [ "./frontend/tests/", "./tests" ]
WORKDIR /app/backend/
COPY [ "./backend/filebrowser", "./"]
RUN mkdir -p ./internal/web/dist
COPY [ "./backend/internal/web/embed/", "./internal/web/dist/"]
COPY [ "./backend/internal/web/dist/", "./internal/web/dist/"]
# Run tests
ENV FILEBROWSER_PLAYWRIGHT_TEST=true
+43
View File
@@ -0,0 +1,43 @@
# syntax=docker/dockerfile:1.4
FROM golang:alpine AS base
ARG VERSION
ARG REVISION
WORKDIR /app
COPY ./backend/go.mod ./backend/go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY ./backend ./
RUN --mount=type=cache,target=/go/pkg/mod --mount=type=cache,target=/root/.cache/go-build go build -trimpath -tags analytics -ldflags="-w -s \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=${VERSION}' \
-X 'github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA=${REVISION}'" \
-o filebrowser .
FROM --platform=$BUILDPLATFORM node:jod-slim AS nbuild
WORKDIR /build
COPY ./frontend/package.json ./frontend/package-lock.json ./frontend/
RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates
RUN --mount=type=cache,target=/root/.npm cd frontend && npm ci --maxsockets 1
COPY ./frontend/ ./frontend/
WORKDIR /build/frontend
RUN npm run build:docker
FROM alpine:latest
ENV FILEBROWSER_DATABASE_PATH="/home/filebrowser/data/database.sqlite"
ENV FILEBROWSER_CONFIG="/home/filebrowser/data/config.yaml"
ENV PATH="$PATH:/home/filebrowser"
RUN apk --no-cache add ca-certificates mailcap tzdata curl
RUN adduser -D -s /bin/true -u 1000 filebrowser
USER filebrowser
WORKDIR /home/filebrowser
# copy config to two locations for compatibility
COPY --from=base --chown=filebrowser:1000 [ "/app/filebrowser", "./" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/config.yaml", "./data/config.yaml" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/config.yaml", "./" ]
COPY --from=base --chown=filebrowser:1000 [ "/app/reduce-rounded-corners.css", "./" ]
COPY --from=nbuild --chown=filebrowser:1000 [ "/build/backend/internal/web/dist/", "./internal/web/dist/" ]
## sanity checks
RUN [ "filebrowser", "version" ]
# exposing default port for auto discovery.
EXPOSE 80
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
CMD curl -f http://localhost:80/health || exit 1
ENTRYPOINT [ "./filebrowser" ]
+4
View File
@@ -20,3 +20,7 @@ userDefaults:
account:
permissions:
share: true
integrations:
media:
skipHwCheck: true
+4
View File
@@ -51,3 +51,7 @@ userDefaults:
video: true
office: true
highQuality: true
integrations:
media:
skipHwCheck: true
+4 -1
View File
@@ -64,4 +64,7 @@ userDefaults:
popup: true
video: true
office: true
highQuality: true
highQuality: true
integrations:
media:
skipHwCheck: true
+4 -1
View File
@@ -50,4 +50,7 @@ userDefaults:
permissions:
admin: false
share: false
api: false
api: false
integrations:
media:
skipHwCheck: true
@@ -18,3 +18,7 @@ userDefaults:
account:
permissions:
share: true
integrations:
media:
skipHwCheck: true
+7
View File
@@ -39,3 +39,10 @@ userDefaults:
office: true
models: true
highQuality: true
integrations:
media:
skipHwCheck: true
# Real HLS transcoding for the preview specs (requires ffmpeg in the image).
transcode:
enabled: true
@@ -13,23 +13,20 @@ export default defineConfig({
globalSetup: "./tests/playwright/previews-setup",
timeout: 10000,
testDir: "./tests/playwright/previews",
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
fullyParallel: false,
forbidOnly: false,
/* Retry on CI only */
retries: 2,
workers: 10,
reporter: "line",
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
launchOptions: {
args: [
'--disable-gpu',
'--use-gl=swiftshader', // CPU-based WebGL renderer built into Chromium
'--enable-webgl',
'--disable-dev-shm-usage', // Overcome limited resource problems in Docker
'--no-sandbox', // Required for Docker
"--autoplay-policy=no-user-gesture-required",
"--disable-gpu",
"--use-gl=swiftshader",
"--enable-webgl",
"--disable-dev-shm-usage",
"--no-sandbox",
],
},
storageState: "previews.json",
+4
View File
@@ -40,3 +40,7 @@ userDefaults:
account:
permissions:
share: true
integrations:
media:
skipHwCheck: true
+4 -1
View File
@@ -19,4 +19,7 @@ userDefaults:
# Size 4: switch-view "gallery" family resolves to icons (not gallery); list family → list.
gallerySize: 3
account:
showAdvancedProfile: true
showAdvancedProfile: true
integrations:
media:
skipHwCheck: true
+4 -1
View File
@@ -48,4 +48,7 @@ auth:
userDefaults:
listing:
showHidden: true
showHidden: true
integrations:
media:
skipHwCheck: true
+4 -1
View File
@@ -27,4 +27,7 @@ frontend:
url: "https://playwright.dev/"
title: "Playwright"
styling:
disableEventThemes: true
disableEventThemes: true
integrations:
media:
skipHwCheck: true
+1 -1
View File
@@ -7,7 +7,7 @@ env_files = [".air.env"]
entrypoint = "./tmp/filebrowser"
args_bin = ["-c", "test_config.yaml"]
include_ext = ["go", "tpl", "tmpl", "html"]
exclude_dir = ["tmp", "vendor", "internal/web/dist", "internal/web/embed", "internal/preview/testdata", "internal/preview/thumbnails", "internal/preview/heic", "swagger/docs", "internal/preview/gofitzinclude"]
exclude_dir = ["tmp", "vendor", "internal/web/dist", "internal/preview/testdata", "internal/preview/thumbnails", "internal/preview/heic", "swagger/docs"]
poll = true
poll_interval = 1000
log = "air.log"
+1 -1
View File
@@ -17,7 +17,7 @@ linters:
exhaustive:
check:
- switch
- map
- mapliteral
exhaustruct:
exclude:
- ^net/http.Client$
+39 -115
View File
@@ -5,137 +5,61 @@ version: 2
before:
hooks:
- go mod download
- go run ./scripts/setup-gofitz-cgo
builds:
# Build configuration for linux amd64
- id: linux-amd64
- id: filebrowser
main: main.go
# Flat dist layout without GOAMD64/GOARM64/GORISCV64 suffix folders (e.g. _v1, _v8.0).
no_unique_dist_dir: true
binary: >-
{{ .Os }}-{{ .Arch }}{{ if eq .Arch "arm" }}-{{ .Arm }}{{ end }}/filebrowser
env:
- CGO_ENABLED=1
- CC=zig cc -target x86_64-linux-musl
- CGO_ENABLED=0
flags:
- -trimpath
tags:
- analytics
- mupdf
- musl
ldflags: &ldflags
ldflags:
- -s -w -X github.com/gtsteffaniak/filebrowser/backend/internal/version.Version=v{{ .Version }} -X github.com/gtsteffaniak/filebrowser/backend/internal/version.CommitSHA={{ .ShortCommit }}
main: main.go
binary: filebrowser
goos:
- linux
goarch:
- amd64
hooks:
post:
- upx {{ .Path }}
# Build configuration for linux arm64
- id: linux-arm64
ldflags: *ldflags
main: main.go
binary: filebrowser
env:
- CGO_ENABLED=1
- CC=zig cc -target aarch64-linux-musl
tags:
- analytics
- mupdf
- musl
goos:
- linux
goarch:
- arm64
hooks:
post:
- upx {{ .Path }}
# Build configuration for linux riscv64
- id: linux-riscv64
ldflags: *ldflags
main: main.go
binary: filebrowser
env:
- CC=zig cc -target riscv64-linux-musl
# go-fitz does not provide riscv64 MuPDF libraries yet: https://github.com/gen2brain/go-fitz/tree/master/libs
tags:
- analytics
- musl
goos:
- linux
goarch:
- riscv64
# Build configuration for linux
- id: linux-arm32
ldflags: *ldflags
main: main.go
binary: filebrowser
tags:
- analytics
goos:
- linux
goarch:
- arm
goarm:
- "6"
- "7"
hooks:
post:
- upx {{ .Path }} # Compress the binary with UPX
# Build configuration for windows without arm
- id: windows
env:
- CGO_ENABLED=1
# windows-gnu avoids zig rejecting Go's -tsaware when cross-compiling from macOS
- CC=zig cc -target x86_64-windows-gnu -lc
tags:
- analytics
- mupdf
ldflags: *ldflags
main: main.go
binary: filebrowser
goos:
- windows
goarch:
- amd64
hooks:
post:
- upx {{ .Path }} # Compress the binary with UPX
# Build configuration for freebsd without arm & upx
- id: freeBSD and freebsd-arm
ldflags: *ldflags
main: main.go
binary: filebrowser
tags:
- analytics
goos:
- darwin
- freebsd
goarch:
- amd64
# Build configuration for macos without upx (no mupdf: CGO cross-compile from Linux CI is unsupported)
- id: macos
ldflags: *ldflags
main: main.go
binary: filebrowser
tags:
- analytics
goos:
- darwin
goarch:
- arm64
- amd64
- arm
- riscv64
goarm:
- "6"
- "7"
ignore:
- goos: windows
goarch: arm
- goos: windows
goarch: riscv64
- goos: darwin
goarch: arm
- goos: darwin
goarch: riscv64
- goos: freebsd
goarch: arm
- goos: freebsd
goarch: arm64
- goos: freebsd
goarch: riscv64
archives:
- name_template: >
{{- if eq .Os "windows" -}}
{{.ProjectName}}
{{.ProjectName}}{{ if ne .Arch "amd64" }}-{{ .Arch }}{{ end }}
{{- else if eq .Arch "arm" -}}
{{.Os}}-arm-{{ .Arm }}-{{.ProjectName}}
{{- else -}}
{{.Os}}-{{.Arch}}{{if .Arm}}v{{.Arm}}{{end}}-{{.ProjectName}}
{{.Os}}-{{.Arch}}-{{.ProjectName}}
{{- end -}}
formats: [ 'binary' ]
checksum:
disable: true
formats: ["tar.gz"]
format_overrides:
- goos: windows
formats: ["zip"]
+1 -53
View File
@@ -56,8 +56,6 @@ github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8v
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/chroma/v2 v2.27.0 h1:FodwmyOBgJULFYmDqibcp9pvfDLWdtPRh9v/r5BXYZs=
github.com/alecthomas/chroma/v2 v2.27.0/go.mod h1:NjJ3ciIgrqBNeIkWZ4e46nseoLDslxU1LmfCoL+wcY8=
github.com/alecthomas/go-check-sumtype v0.4.0 h1:gd4mE4QsA62sXlnFIs8GUv8/0xH3DOUqTkYQR1l2v28=
github.com/alecthomas/go-check-sumtype v0.4.0/go.mod h1:aYYllI8IXNjo7ZYUzI2+wyLRECMg3zo+8CZj9JTBAWU=
github.com/alecthomas/go-check-sumtype v0.5.1-0.20260828200218-ae6904d28606 h1:dXPJ+uOJLWb8rjlAuLX7JPiRJtZKw/53Ol+WxdDJ1Eo=
github.com/alecthomas/go-check-sumtype v0.5.1-0.20260828200218-ae6904d28606/go.mod h1:ZH31IJyNpxpStxP2ayCcPQz+3T2G/OnxIQqe4LWvSDw=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
@@ -122,8 +120,6 @@ github.com/bombsimon/wsl/v4 v4.7.0 h1:1Ilm9JBPRczjyUs6hvOPKvd7VL1Q++PL8M0SXBDf+j
github.com/bombsimon/wsl/v4 v4.7.0/go.mod h1:uV/+6BkffuzSAVYD+yGyld1AChO7/EuLrCF/8xTiapg=
github.com/bombsimon/wsl/v5 v5.9.0 h1:WCrgZ7RQnZO5oEwbVTlYgBdU3wL294kR1BSWV8vTfsU=
github.com/bombsimon/wsl/v5 v5.9.0/go.mod h1:kjo4HiAV5FDkHC8/uzJq9mBffEEd6WT/nvN7DoMovDM=
github.com/breml/bidichk v0.3.3 h1:WSM67ztRusf1sMoqH6/c4OBCUlRVTKq+CbSeo0R17sE=
github.com/breml/bidichk v0.3.3/go.mod h1:ISbsut8OnjB367j5NseXEGGgO/th206dVa427kR8YTE=
github.com/breml/bidichk v0.3.4 h1:70G6w80G6GDZTUURHTC5gBKVjMLMw2736CzXquSf5Ek=
github.com/breml/bidichk v0.3.4/go.mod h1:GOHprnh4i0V7HrvWjtbvbpicTUcqoYt9zYESKcJkk4M=
github.com/breml/errchkjson v0.4.1 h1:keFSS8D7A2T0haP9kzZTi7o26r7kE3vymjZNeNDRDwg=
@@ -142,8 +138,6 @@ github.com/charithe/durationcheck v0.0.11 h1:g1/EX1eIiKS57NTWsYtHDZ/APfeXKhye1Di
github.com/charithe/durationcheck v0.0.11/go.mod h1:x5iZaixRNl8ctbM+3B2RrPG5t856TxRyVQEnbIEM2X4=
github.com/charmbracelet/colorprofile v0.4.3 h1:QPa1IWkYI+AOB+fE+mg/5/4HRMZcaXex9t5KX76i20Q=
github.com/charmbracelet/colorprofile v0.4.3/go.mod h1:/zT4BhpD5aGFpqQQqw7a+VtHCzu+zrQtt1zhMt9mR4Q=
github.com/charmbracelet/ultraviolet v0.0.0-20260910203606-6c9e17dc7a16 h1:/hSCZO1puetbGd2JikHKN0nAcb2mFevzxJElFoduhaw=
github.com/charmbracelet/ultraviolet v0.0.0-20260910203606-6c9e17dc7a16/go.mod h1:JFvmQPZh3uAu7PpSZU91bcR0BrIy24HhW+0EsW1uGxE=
github.com/charmbracelet/ultraviolet v0.0.0-20260922123528-4e49372c11f9 h1:ZGHc3eaN2TIhkpwRibtr8N9jLQJOY1rh6GEByRoRmp4=
github.com/charmbracelet/ultraviolet v0.0.0-20260922123528-4e49372c11f9/go.mod h1:D9SVXVpAsD1pP6xR6BiERbe4+dUU6EW9US1TFrNutGA=
github.com/charmbracelet/x/ansi v0.11.8 h1:JMFwp0CgDC2+jcOB162HH5k7I3FVbgFSMMYg7dSPBQQ=
@@ -288,8 +282,6 @@ github.com/gohugoio/hugo-goldmark-extensions/passthrough v0.5.0 h1:p13Q0DBCrBRpJ
github.com/gohugoio/hugo-goldmark-extensions/passthrough v0.5.0/go.mod h1:ob9PCHy/ocsQhTz68uxhyInaYCbbVNpOOrJkIoSeD+8=
github.com/golangci/asciicheck v0.5.0 h1:jczN/BorERZwK8oiFBOGvlGPknhvq0bjnysTj4nUfo0=
github.com/golangci/asciicheck v0.5.0/go.mod h1:5RMNAInbNFw2krqN6ibBxN/zfRFa9S6tA1nPdM0l8qQ=
github.com/golangci/canonicalheader v0.0.0-20260827115959-a25c71c521f6 h1:fVLolA3dG6s0brGetsiDAtcnpMSwVa2LqXJEw/9RJG4=
github.com/golangci/canonicalheader v0.0.0-20260827115959-a25c71c521f6/go.mod h1:1xo+NFW5S+bEf2DKXhaxuzvcDN5AR6o/KMwvqC5Mkpo=
github.com/golangci/canonicalheader v0.0.0-20260928145534-e18e5c5a078f h1:pOnwRNeWLPkADfkgqKcCP7RPQAS9OIzffaBaktRsKnk=
github.com/golangci/canonicalheader v0.0.0-20260928145534-e18e5c5a078f/go.mod h1:aXo+AVSX0pC3TRo2wAfR92YwBKfG6C2eal+omjKXwYw=
github.com/golangci/dupl v0.0.0-20260401084720-c99c5cf5c202 h1:CbTB8KpqnViI6lIXxp03Oclc4VFHi3K4BWC1TacsZ+A=
@@ -298,14 +290,8 @@ github.com/golangci/go-printf-func-name v0.1.1 h1:hIYTFJqAGp1iwoIfsNTpoq1xZAarog
github.com/golangci/go-printf-func-name v0.1.1/go.mod h1:Es64MpWEZbh0UBtTAICOZiB+miW53w/K9Or/4QogJss=
github.com/golangci/gofmt v0.0.0-20260820135601-e84e05053792 h1:WL8YKrt3UbOBqSRU7GpP5BTtQTMWtVtj+mfPijgZeIg=
github.com/golangci/gofmt v0.0.0-20260820135601-e84e05053792/go.mod h1:te5hX0dW4C5r6YbXs+6ysNr8Q5UTmdIqGbb+mlFiYmA=
github.com/golangci/golangci-lint/v2 v2.13.2 h1:bCyq3E4vo9qwzifjpzJqYndEt7Ncva80qkk8G2B4TXU=
github.com/golangci/golangci-lint/v2 v2.13.2/go.mod h1:5xaMd1kAxV7GSBPEyngw8gnnjoRocqW9UVKdBR6627w=
github.com/golangci/golangci-lint/v2 v2.14.0 h1:ot8QffRa4LzAAEgtvNYrVs9esxQ7xoTwoAv6uhp4ngA=
github.com/golangci/golangci-lint/v2 v2.14.0/go.mod h1:amHimCzNDP1GfVno3hIiNssjvqXmuW+CPZAHOdR4cIc=
github.com/golangci/golangci-lint/v2 v2.15.0 h1:w94zf8HAYI7hm3164kwKsvZRuqXvY+YxeFoTbu1OLg8=
github.com/golangci/golangci-lint/v2 v2.15.0/go.mod h1:/FRI304IxqDVhX2o3ZZ0qp8Vvz8egJdzCY5A4GjvmTU=
github.com/golangci/golines v0.15.0 h1:Qnph25g8Y1c5fdo1X7GaRDGgnMHgnxh4Gk4VfPTtRx0=
github.com/golangci/golines v0.15.0/go.mod h1:AZjXd23tbHMpowhtnGlj9KCNsysj72aeZVVHnVcZx10=
github.com/golangci/golines v0.16.0 h1:xKdwzRvwzLd7V131B/dv90vUiEoTA0vQybdeqperPOg=
github.com/golangci/golines v0.16.0/go.mod h1:mCMgRQHiCvuALJOeaNtylu09zbzvS9Ka01dfvLLJios=
github.com/golangci/misspell v0.8.0 h1:qvxQhiE2/5z+BVRo1kwYA8yGz+lOlu5Jfvtx2b04Jbg=
@@ -314,8 +300,6 @@ github.com/golangci/plugin-module-register v0.1.2 h1:e5WM6PO6NIAEcij3B053CohVp3H
github.com/golangci/plugin-module-register v0.1.2/go.mod h1:1+QGTsKBvAIvPvoY/os+G5eoqxWn70HYDm2uvUyGuVw=
github.com/golangci/revgrep v0.8.0 h1:EZBctwbVd0aMeRnNUsFogoyayvKHyxlV3CdUA46FX2s=
github.com/golangci/revgrep v0.8.0/go.mod h1:U4R/s9dlXZsg8uJmaR1GrloUr14D7qDl8gi2iPXJH8k=
github.com/golangci/rowserrcheck v0.0.0-20260901154044-241134db7241 h1:0kPqfqytQizeQ9WBk487NcsGcRdS8YhHeUhXxkfEAhA=
github.com/golangci/rowserrcheck v0.0.0-20260901154044-241134db7241/go.mod h1:lczq5oJ8PxpMESQG6dO3iFYTQPhpqoMOYw//n2FEnHA=
github.com/golangci/rowserrcheck v0.0.0-20261005233414-e0740d52dee8 h1:oN042y1yEcpldrqY81puyNA4ZjQZJzt0U/t7HyYrMic=
github.com/golangci/rowserrcheck v0.0.0-20261005233414-e0740d52dee8/go.mod h1:T1Bg58Sk3N352aBbPLJPpHykFWxEEwV3o4/Enn+PPU8=
github.com/golangci/swaggoswag v0.0.0-20250504205917-77f2aca3143e h1:ai0EfmVYE2bRA5htgAG9r7s3tHsfjIhN98WshBTJ9jM=
@@ -373,8 +357,6 @@ github.com/julz/importas v0.2.0 h1:y+MJN/UdL63QbFJHws9BVC5RpA2iq0kpjrFajTGivjQ=
github.com/julz/importas v0.2.0/go.mod h1:pThlt589EnCYtMnmhmRYY/qn9lCf/frPOK+WMx3xiJY=
github.com/karamaru-alpha/copyloopvar v1.2.2 h1:yfNQvP9YaGQR7VaWLYcfZUlRP2eo2vhExWKxD/fP6q0=
github.com/karamaru-alpha/copyloopvar v1.2.2/go.mod h1:oY4rGZqZ879JkJMtX3RRkcXRkmUvH0x35ykgaKgsgJY=
github.com/kisielk/errcheck v1.20.0 h1:9rwHBNKzd4wkDWcROy3DvFGNqEPlkxBg305rvk7HabI=
github.com/kisielk/errcheck v1.20.0/go.mod h1:O+f80MKNwX8Oor2jwgpeQ9An7uJm+hRSgT+h22knRJU=
github.com/kisielk/errcheck v1.30.0 h1:ZIxNDAfAd4larXHTceoZP/dYPAB90dHg1OYhA2sVh5k=
github.com/kisielk/errcheck v1.30.0/go.mod h1:uiCAInmFuyOje4VDdbQSP9mgqf5gORYSXrDf19XilsI=
github.com/kkHAIKE/contextcheck v1.1.6 h1:7HIyRcnyzxL9Lz06NGhiKvenXq7Zw6Q0UQu/ttjfJCE=
@@ -388,8 +370,6 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kulti/thelper v0.7.1 h1:fI8QITAoFVLx+y+vSyuLBP+rcVIB8jKooNSCT2EiI98=
github.com/kulti/thelper v0.7.1/go.mod h1:NsMjfQEy6sd+9Kfw8kCP61W1I0nerGSYSFnGaxQkcbs=
github.com/kunwardeep/paralleltest v1.0.15 h1:ZMk4Qt306tHIgKISHWFJAO1IDQJLc6uDyJMLyncOb6w=
github.com/kunwardeep/paralleltest v1.0.15/go.mod h1:di4moFqtfz3ToSKxhNjhOZL+696QtJGCFe132CbBLGk=
github.com/kunwardeep/paralleltest v1.0.16 h1:I0gQkLNgfigjpxWAGhx7PYj3VpswzfGrSV/715t6g+o=
github.com/kunwardeep/paralleltest v1.0.16/go.mod h1:di4moFqtfz3ToSKxhNjhOZL+696QtJGCFe132CbBLGk=
github.com/kyokomi/emoji/v2 v2.2.14 h1:YOF6VL52613M0Qr9v4puJDD9QQPmyyjXedDDlrGzH80=
@@ -428,18 +408,12 @@ github.com/matoous/godox v1.1.0 h1:W5mqwbyWrwZv6OQ5Z1a/DHGMOvXYCBP3+Ht7KMoJhq4=
github.com/matoous/godox v1.1.0/go.mod h1:jgE/3fUXiTurkdHOLT5WEkThTSuE7yxHv5iWPa80afs=
github.com/matryer/is v1.4.0 h1:sosSmIWwkYITGrxZ25ULNDeKiMNzFSr4V/eqBQP0PeE=
github.com/matryer/is v1.4.0/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwMRU=
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-colorable v0.1.16 h1:+2YjeAe7xP1OQbZW1jERPWWCDLlNqFNCvjiqGiAt84A=
github.com/mattn/go-colorable v0.1.16/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-runewidth v0.0.30 h1:+KUuiDA4fF0R1p5FeueHefjDm+GIM+kWfFnDjybOPgk=
github.com/mattn/go-runewidth v0.0.30/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/mgechev/revive v1.16.0 h1:9OZVO8xOPUwIYynplHXv7liMllPQ/BTNZLp07sM/Im0=
github.com/mgechev/revive v1.16.0/go.mod h1:nT854jMmvQsS/qqxEaMPM4pJSUpxhcPoHM788ZxDvdQ=
github.com/mgechev/revive v1.17.0 h1:hwTTVC3svSv9X2FA2hiYT17Uws+lfxDWhPxzBZZCREg=
github.com/mgechev/revive v1.17.0/go.mod h1:uiVNUF8ruVh4WHm/aPs7kKEeuo+m6oO3tuOfniyzP/g=
github.com/mgechev/revive v1.17.1 h1:z8hV0L3uJ6yrwOMnJbkc5qBJpxXa2AxYKFKERGJyA4s=
github.com/mgechev/revive v1.17.1/go.mod h1:FRd64pJwWSEE6j8ESoaX0PixPzjh2rpNSfckbEu3rIE=
github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk=
@@ -460,13 +434,8 @@ github.com/nakabonne/nestif v0.3.1 h1:wm28nZjhQY5HyYPx+weN3Q65k6ilSBxDb8v5S81B81
github.com/nakabonne/nestif v0.3.1/go.mod h1:9EtoZochLn5iUprVDmDjqGKPofoUEBL8U4Ngq6aY7OE=
github.com/niklasfasching/go-org v1.9.1 h1:/3s4uTPOF06pImGa2Yvlp24yKXZoTYM+nsIlMzfpg/0=
github.com/niklasfasching/go-org v1.9.1/go.mod h1:ZAGFFkWvUQcpazmi/8nHqwvARpr1xpb+Es67oUGX/48=
github.com/nishanths/exhaustive v0.13.0 h1:Rof0axOSNQaaPaPmLK5RxcRrJV2Vcv+vfUmuRy+2EAM=
github.com/nishanths/exhaustive v0.13.0/go.mod h1:3/adQjJgvD/UFHHSYBjpK9Wbz8DOvf0FBtzx+zI0Rlo=
github.com/nishanths/exhaustive v0.15.0 h1:uFjiO/WvrwQG+ojSZ+vtl/jAFbqA2oAtfpUWK/iKQ8M=
github.com/nishanths/exhaustive v0.15.0/go.mod h1:2by9rYmg9FnsnLQC4Ixig4fQaWxRsY7RluxJuAu9sa0=
github.com/nishanths/exhaustive v0.16.0/go.mod h1:/nBwAMRn2QmCjWQ3PiJ939yRSwvWLKSa21L2C6Tp84E=
github.com/nishanths/predeclared v0.2.2 h1:V2EPdZPliZymNAn79T8RkNApBjMmVKh5XRpLm/w98Vk=
github.com/nishanths/predeclared v0.2.2/go.mod h1:RROzoN6TnGQupbC+lqggsOlcgysk3LMK/HI84Mp280c=
github.com/nishanths/predeclared v0.3.0 h1:W3i3JK/DZS5h7QwsEIyU8ler4Z2vwEwQSxNrZzhnlfk=
github.com/nishanths/predeclared v0.3.0/go.mod h1:gN4uryPExDME/zmZdqkKIPSAqcfprj316ZDI6UTqz70=
github.com/nunnatsa/ginkgolinter v0.24.0 h1:Mp0EagluLFP98JatP6nqp/gGEoljNG97uf9AcxcBVy8=
@@ -598,16 +567,10 @@ github.com/tetafro/godot v1.5.6 h1:IEkrFCwXaYHlOn4mGzGS3F3dkP6m9t0jpwqBFPIkKiA=
github.com/tetafro/godot v1.5.6/go.mod h1:eOkMrVQurDui411nBY2FA05EYH01r14LuWY/NrVDVcU=
github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZopYU=
github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0=
github.com/timakin/bodyclose v0.0.0-20260723120731-857993a2939c h1:d/FUP8GD4QiY/rdFzZmBu3KqL9eBNVzMu7W/5pGkaMM=
github.com/timakin/bodyclose v0.0.0-20260723120731-857993a2939c/go.mod h1:sDHLK7rb/59v/ZxZ7KtymgcoxuUMxjXq8gtu9VMOK8M=
github.com/timakin/bodyclose v0.0.0-20260923065923-98ded9ab4ef2 h1:HECVafB2AKntWWRfsiQ4gdunLx5WOYQzFdCsxqOrzp8=
github.com/timakin/bodyclose v0.0.0-20260923065923-98ded9ab4ef2/go.mod h1:JLoPVtROtNgJ99yXTIxuGNttvmJa85tkJAO6lr05qBA=
github.com/timonwong/loggercheck v0.12.0 h1:7Cb9XGLJZSWYNjgL459DIhpSiEHupD+MLQ6vaN2Pl3g=
github.com/timonwong/loggercheck v0.12.0/go.mod h1:Wa6Y+SJHQR01mViZlLLY5GAZVOWOLNMbkXddMjFWQCs=
github.com/timonwong/loggercheck v0.13.0 h1:6amIOmLLPFYt24rXUbNWzcJeNGuJc+3c31tZtO85VHs=
github.com/timonwong/loggercheck v0.13.0/go.mod h1:thn3allpox0fI5JeUWTCwiwp45peQxCZZHkHuA1Oc6I=
github.com/tomarrell/wrapcheck/v2 v2.12.0 h1:H/qQ1aNWz/eeIhxKAFvkfIA+N7YDvq6TWVFL27Of9is=
github.com/tomarrell/wrapcheck/v2 v2.12.0/go.mod h1:AQhQuZd0p7b6rfW+vUwHm5OMCGgp63moQ9Qr/0BpIWo=
github.com/tomarrell/wrapcheck/v2 v2.13.0 h1:jYbLNs8PDkzGCY6d5LUzJdY7Bq//CTlb2HxZGV5z0JY=
github.com/tomarrell/wrapcheck/v2 v2.13.0/go.mod h1:AQhQuZd0p7b6rfW+vUwHm5OMCGgp63moQ9Qr/0BpIWo=
github.com/tommy-muehle/go-mnd/v2 v2.5.1 h1:NowYhSdyE/1zwK9QCLeRb6USWdoif80Ie+v+yU8u1Zw=
@@ -618,12 +581,8 @@ github.com/ultraware/whitespace v0.2.0 h1:TYowo2m9Nfj1baEQBjuHzvMRbp19i+RCcRYrSW
github.com/ultraware/whitespace v0.2.0/go.mod h1:XcP1RLD81eV4BW8UhQlpaR+SDc2givTvyI8a586WjW8=
github.com/urfave/cli/v2 v2.27.7 h1:bH59vdhbjLv3LAvIu6gd0usJHgoTTPhCFib8qqOwXYU=
github.com/urfave/cli/v2 v2.27.7/go.mod h1:CyNAG/xg+iAOg0N4MPGZqVmv2rCoP267496AOXUZjA4=
github.com/uudashr/gocognit v1.2.1 h1:CSJynt5txTnORn/DkhiB4mZjwPuifyASC8/6Q0I/QS4=
github.com/uudashr/gocognit v1.2.1/go.mod h1:acaubQc6xYlXFEMb9nWX2dYBzJ/bIjEkc1zzvyIZg5Q=
github.com/uudashr/gocognit v1.3.0 h1:Xdw5JwAoYmX3WAB+wsXY3nxEwdw7+MWAnM0wCAAv7A4=
github.com/uudashr/gocognit v1.3.0/go.mod h1:acaubQc6xYlXFEMb9nWX2dYBzJ/bIjEkc1zzvyIZg5Q=
github.com/uudashr/iface v1.5.1 h1:BS3yrgaT55s3dAtJoxuTnsZNbsejq/mVuzp49YDyfBA=
github.com/uudashr/iface v1.5.1/go.mod h1:5UWoT6SvTdTww/KToRj6clO+n4Kg3/NrAQvgOB+5Ggw=
github.com/uudashr/iface v1.5.3 h1:N61bzr11L7BHP6Rct+C6zQePHjSTMeyVBq+s03uB/Ds=
github.com/uudashr/iface v1.5.3/go.mod h1:5UWoT6SvTdTww/KToRj6clO+n4Kg3/NrAQvgOB+5Ggw=
github.com/xen0n/gosmopolitan v1.3.0 h1:zAZI1zefvo7gcpbCOrPSHJZJYA9ZgLfJqtKzZ5pHqQM=
@@ -690,8 +649,6 @@ golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/mod v0.42.0 h1:ICeL3t2L30ljSPHHxDnR4fqVcBt/zR2zJAb9juavmQg=
golang.org/x/mod v0.42.0/go.mod h1:u5fzIrRInkL8MaO1cbRkUHT64v303YIT5KEqpNr7jRs=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
@@ -703,9 +660,8 @@ golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -713,8 +669,6 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -730,8 +684,6 @@ golang.org/x/sys v0.0.0-20211105183446-c75c47738b0c/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
@@ -754,8 +706,6 @@ golang.org/x/tools v0.1.1-0.20210302220138-2ac05c832e1a/go.mod h1:9bzcO0MWcOuT0t
golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.50.0 h1:c2ifzfcuY7L90lZ2aKd8S4K2NpASF08SZx9ZuJkHmSU=
golang.org/x/tools v0.50.0/go.mod h1:7ulVMw3831Mwi5EZD6RomGyffr4VFjuNYXf2BbCEAV0=
golang.org/x/tools v0.52.0 h1:ryxg4U63oaUYk7uOLTF37n43f4qgtGvqV4Ioit1BaEs=
golang.org/x/tools v0.52.0/go.mod h1:ItcP5UNaVGG+Yc4eIYmaFLNJ4soEopkx5Y2d4lcEurs=
golang.org/x/tools/go/expect v0.1.1-deprecated h1:jpBZDwmgPhXsKZC6WhL20P4b/wmnpsEAGHaNy0n/rJM=
@@ -774,8 +724,6 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EV
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.8.1 h1:+JKf3xJ1ni4CwrhVg4/pqsfPGP6vNAXcKbMXJodYx3w=
honnef.co/go/tools v0.8.1/go.mod h1:XA+OnlRA9EDh/ukGvXMNSZNKGwFQJ+5dER0ioUkOxks=
honnef.co/go/tools v0.8.2 h1:+jPbCzuYxYrcoOvCecYUFDj+XJQRmzIhJmlbpps21Zk=
honnef.co/go/tools v0.8.2/go.mod h1:xG1Q4n+mg3bKZrO6RWFA9jFlkqplflmDRHGeuexiLDs=
mvdan.cc/gofumpt v0.12.0 h1:1Lbudkz2kpM9Cjz2pL4M19u7q+GaEhCTNf7N9mfpcho=
+16 -11
View File
@@ -293,18 +293,23 @@ func createConfig(configpath string, noInput bool) error {
return nil
}
// setupConfigYAMLCandidates lists on-disk setup templates for config.generated.yaml.
var setupConfigYAMLCandidates = []string{
"frontend/public/config.generated.yaml",
"../frontend/public/config.generated.yaml",
"internal/web/dist/config.generated.yaml",
"../internal/web/dist/config.generated.yaml",
"backend/internal/web/dist/config.generated.yaml",
}
func readSetupConfigTemplateYAML() (string, error) {
return settings.ReadFirstExistingFile(setupConfigYAMLCandidates)
}
func generateSetupConfigYAML(config *settings.Settings) (string, error) {
embeddedPaths := []string{
"frontend/public/config.generated.yaml",
"../frontend/public/config.generated.yaml",
"internal/web/dist/config.generated.yaml",
"../internal/web/dist/config.generated.yaml",
}
for _, path := range embeddedPaths {
embeddedYaml, err := os.ReadFile(path)
if err == nil && len(embeddedYaml) > 0 {
return settings.GenerateConfigYamlWithEmbedded(config, true, false, false, string(embeddedYaml))
}
embeddedYaml, err := readSetupConfigTemplateYAML()
if err == nil {
return settings.GenerateConfigYamlWithEmbedded(config, true, false, false, embeddedYaml)
}
return settings.GenerateConfigYaml(config, true, false, false)
}
+47
View File
@@ -0,0 +1,47 @@
package cmd
import (
"os"
"path/filepath"
"testing"
)
func writeSetupConfigYAML(t *testing.T, root, relPath, content string) {
t.Helper()
path := filepath.Join(root, filepath.FromSlash(relPath))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
func TestReadSetupConfigTemplateYAML_fromDist(t *testing.T) {
root := t.TempDir()
writeSetupConfigYAML(t, root, "internal/web/dist/config.generated.yaml", "dist-content")
t.Chdir(root)
got, err := readSetupConfigTemplateYAML()
if err != nil {
t.Fatal(err)
}
if got != "dist-content" {
t.Fatalf("readSetupConfigTemplateYAML() = %q, want dist-content", got)
}
}
func TestReadSetupConfigTemplateYAML_prefersFrontendPublic(t *testing.T) {
root := t.TempDir()
writeSetupConfigYAML(t, root, "frontend/public/config.generated.yaml", "from-public")
writeSetupConfigYAML(t, root, "internal/web/dist/config.generated.yaml", "from-dist")
t.Chdir(root)
got, err := readSetupConfigTemplateYAML()
if err != nil {
t.Fatal(err)
}
if got != "from-public" {
t.Fatalf("readSetupConfigTemplateYAML() = %q, want from-public", got)
}
}
+23 -14
View File
@@ -9,14 +9,14 @@ import (
"syscall"
"time"
_ "net/http/pprof"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
"github.com/gtsteffaniak/filebrowser/backend/internal/analytics"
"github.com/gtsteffaniak/filebrowser/backend/internal/app"
"github.com/gtsteffaniak/filebrowser/backend/internal/icons"
"github.com/gtsteffaniak/filebrowser/backend/internal/preview"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/internal/textpreview"
"github.com/gtsteffaniak/filebrowser/backend/internal/transcoding"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/internal/version"
"github.com/gtsteffaniak/filebrowser/backend/internal/web"
@@ -52,6 +52,12 @@ func initializeDatabase(configFile string, cliMode bool) bool {
if err != nil {
logger.Fatalf("could not initialize state: %v", err)
}
// state.Open applies persisted media settings, so initialize ffmpeg once here
// (before wiring, which reads Env.FFmpegAvailable) instead of during config
// load, which would run the multi-second hardware codec probe twice.
if !cliMode {
settings.InitFFmpeg()
}
runtimeApp, err = app.WireServices(store)
if err != nil {
logger.Fatalf("could not wire services: %v", err)
@@ -87,14 +93,7 @@ func StartFilebrowser() {
done := make(chan struct{}) // Signals server has stopped
shutdownComplete := make(chan struct{}) // Signals shutdown process is complete
// Dev mode enables development features like template hot-reloading
_, err := os.Stat("internal/web/dist")
// In dev mode, always use filesystem assets. Otherwise, check if internal/web/dist exists
if !settings.Env.IsDevMode {
settings.Env.EmbeddedFs = os.IsNotExist(err)
}
err = utils.SetInvalidPasswordHash()
err := utils.SetInvalidPasswordHash()
if err != nil {
logger.Fatalf("Failed to set security hash: %v", err)
}
@@ -180,6 +179,13 @@ func StartFilebrowser() {
fileutils.ClearCacheDir(settings.Config.Server.CacheDir)
}
<-shutdownComplete
if transcodeMgr := transcoding.Default(); transcodeMgr != nil {
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 30*time.Second)
if err := transcodeMgr.Close(shutdownCtx); err != nil {
logger.Warningf("transcode shutdown: %v", err)
}
shutdownCancel()
}
if err := fileutils.ClearDirectoryContents(settings.DownloadCacheDir()); err != nil {
logger.Warningf("failed to clear download spool on shutdown: %v", err)
}
@@ -194,14 +200,16 @@ func rootCMD(ctx context.Context, serverConfig *settings.Server, a *app.App, shu
numWorkers := settings.Config.Server.NumImageProcessors
// Initialize asset filesystem before starting services
if settings.Env.EmbeddedFs {
if !settings.Env.IsDevMode && web.EmbeddedFrontendAvailable() {
embeddedAssets := web.GetEmbeddedAssets()
subAssets, err := fs.Sub(embeddedAssets, "embed")
subAssets, err := fs.Sub(embeddedAssets, "dist")
if err != nil {
logger.Fatalf("Failed to create sub filesystem: %v", err)
}
settings.Env.EmbeddedFs = true
fileutils.InitAssetFS(subAssets, true, false)
} else {
settings.Env.EmbeddedFs = false
fileutils.InitAssetFS(nil, false, settings.Env.IsDevMode)
}
@@ -210,8 +218,9 @@ func rootCMD(ctx context.Context, serverConfig *settings.Server, a *app.App, shu
if err != nil {
logger.Fatalf("Error starting preview service: %v", err)
}
logger.Debugf("MuPDF Enabled : %v", settings.Env.MuPdfAvailable)
// Bound concurrent text preview reads/conversions with the same worker count
// the image pipeline uses, but on a separate limiter.
textpreview.SetConcurrency(numWorkers)
// Generate PWA icons after preview service is initialized
if err := icons.GeneratePWAIcons(); err != nil {
logger.Warningf("Failed to generate PWA icons: %v", err)
+4
View File
@@ -62,3 +62,7 @@ userDefaults:
#office:
# url: "http://localhost:9052"
# secret: "6H1H7GzUk2alRkNxAPav0W02g8JS5oUd"
integrations:
media:
skipHwCheck: true
Binary file not shown.
+23
View File
@@ -0,0 +1,23 @@
{
"generatedAt": "2026-10-27T14:46:52Z",
"gitHead": "880900c18dc7655899398cfb7327ab4e191fc3f0",
"profileSha256": "371d012d4ed5bc3907667f6d35876c998af98a6cc3c71f6ea4ddf57ab4ed4bbd",
"workloads": [
"internal/thumb BenchmarkMJPEGSmall/Large",
"internal/thumb BenchmarkMP4H264Large",
"internal/thumb BenchmarkMP4H264Small (full pipeline)",
"internal/thumb BenchmarkMP4H264Baseline1080Small (CAVLC + thumb)",
"internal/thumb BenchmarkMP4H264Main1080Large",
"internal/thumb BenchmarkMP4H264High1080Large",
"internal/thumb BenchmarkThumbnailLargeWebM",
"internal/h264 BenchmarkBaselineThumbnailDecode (CAVLC IDR)"
],
"hotPathGlobs": [
"internal/thumb/**/*.go",
"internal/h264/**/*.go",
"internal/kernels/**/*.go",
"internal/isobmff/**/*.go",
"internal/matroska/**/*.go",
"internal/jpegfast/**/*.go"
]
}
+14 -13
View File
@@ -1,6 +1,6 @@
module github.com/gtsteffaniak/filebrowser/backend
go 1.27.0
go 1.27.1
require (
github.com/alecthomas/kong v1.16.1
@@ -10,7 +10,7 @@ require (
github.com/dhowden/tag v0.0.0-20240417053706-3d75831295e8
github.com/dsoprea/go-exif/v3 v3.0.1
github.com/evanoberholster/imagemeta v1.1.0
github.com/gen2brain/go-fitz v1.28.2
github.com/gen2brain/folio v0.2.8
github.com/go-ldap/ldap/v3 v3.4.15
github.com/go-playground/validator/v10 v10.30.5
github.com/go-webauthn/webauthn v0.18.2
@@ -19,20 +19,20 @@ require (
github.com/google/go-cmp v0.7.0
github.com/google/uuid v1.6.0
github.com/gtsteffaniak/go-cache v1.1.0
github.com/gtsteffaniak/go-ffmpeg v1.4.0
github.com/gtsteffaniak/go-ffmpeg v1.6.2
github.com/gtsteffaniak/go-logger v1.1.0
github.com/gtsteffaniak/go-media v1.2.0
github.com/kovidgoyal/imaging v1.9.0
github.com/mattn/go-sqlite3 v1.14.52
github.com/pquerna/otp v1.5.0
github.com/spf13/afero v1.15.0
github.com/stretchr/testify v1.12.1
github.com/swaggo/http-swagger v1.3.4
github.com/swaggo/swag v1.16.6
golang.org/x/crypto v0.58.0
golang.org/x/image v0.47.0
golang.org/x/mod v0.42.0
golang.org/x/net v0.61.0
golang.org/x/oauth2 v0.37.0
golang.org/x/sync v0.24.0
golang.org/x/sys v0.49.0
golang.org/x/term v0.47.0
golang.org/x/text v0.43.0
@@ -44,14 +44,20 @@ require (
require (
github.com/Azure/go-ntlmssp v0.1.1 // indirect
github.com/Eyevinn/mp4ff v0.61.0 // indirect
github.com/KyleBanks/depth v1.2.1 // indirect
github.com/abema/go-mp4 v1.7.4 // indirect
github.com/at-wat/ebml-go v0.19.4 // indirect
github.com/boombuler/barcode v1.1.0 // indirect
github.com/dsoprea/go-logging v0.0.0-20200710184922-b02d349568dd // indirect
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 // indirect
github.com/dustin/go-humanize v1.1.0 // indirect
github.com/ebitengine/purego v0.11.2 // indirect
github.com/fxamacker/cbor/v2 v2.9.6 // indirect
github.com/gabriel-vasile/mimetype v1.4.16 // indirect
github.com/gen2brain/gav1d v0.2.6 // indirect
github.com/gen2brain/h265 v0.2.4 // indirect
github.com/gen2brain/jxl v0.2.0 // indirect
github.com/gen2brain/vpx v0.2.1 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8 // indirect
github.com/go-errors/errors v1.5.1 // indirect
github.com/go-jose/go-jose/v4 v4.1.6 // indirect
@@ -74,6 +80,8 @@ require (
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/gtsteffaniak/go-av1 v1.0.0 // indirect
github.com/gtsteffaniak/go-vpx v1.0.1 // indirect
github.com/kovidgoyal/go-parallel v1.1.1 // indirect
github.com/kovidgoyal/go-shm v1.0.0 // indirect
github.com/kr/pretty v0.3.1 // indirect
@@ -85,12 +93,10 @@ require (
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rogpeppe/go-internal v1.16.0 // indirect
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd // indirect
github.com/swaggo/files v1.0.1 // indirect
github.com/tinylib/msgp v1.6.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
go.etcd.io/bbolt v1.5.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/sync v0.24.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/protobuf v1.36.12 // indirect
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
@@ -99,8 +105,3 @@ require (
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
exclude (
github.com/alecthomas/go-check-sumtype v0.5.0
github.com/gtsteffaniak/go-ffmpeg v1.1.0
)
+39 -16
View File
@@ -2,10 +2,14 @@ github.com/Azure/go-ntlmssp v0.1.1 h1:l+FM/EEMb0U9QZE7mKNEDw5Mu3mFiaa2GKOoTSsNDP
github.com/Azure/go-ntlmssp v0.1.1/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
github.com/DataDog/zstd v1.4.1 h1:3oxKN3wbHibqx897utPC2LTQU4J+IHWWJO+glkAkpFM=
github.com/DataDog/zstd v1.4.1/go.mod h1:1jcaCB/ufaK+sKp1NBhlGmpz41jOoPQ35bpF36t7BBo=
github.com/Eyevinn/mp4ff v0.61.0 h1:uX3GBauEouxuGepRbv15JB7X2RyVyE2EZEfLpgimL6o=
github.com/Eyevinn/mp4ff v0.61.0/go.mod h1:97zsDIJUiDTYqKpqq/b1K9foamxXmvsgE4GdPrPK3ks=
github.com/KyleBanks/depth v1.2.1 h1:5h8fQADFrWtarTdtDudMmGsC7GPbOAu6RVB3ffsVFHc=
github.com/KyleBanks/depth v1.2.1/go.mod h1:jzSb9d0L43HxTQfT+oSA1EEp2q+ne2uh6XgeJcm8brE=
github.com/Sereal/Sereal v0.0.0-20190618215532-0b8ac451a863 h1:BRrxwOZBolJN4gIwvZMJY1tzqBvQgpaZiQRuIDD40jM=
github.com/Sereal/Sereal v0.0.0-20190618215532-0b8ac451a863/go.mod h1:D0JMgToj/WdxCgd30Kc1UcA9E+WdZoJqeVOuYW7iTBM=
github.com/abema/go-mp4 v1.7.4 h1:MztDjMx3nac+Lktig9wrUa962Mnyi4+ZtDtpbHuckww=
github.com/abema/go-mp4 v1.7.4/go.mod h1:vPl9t5ZK7K0x68jh12/+ECWBCXoWuIDtNgPtU2f04ws=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/kong v1.16.1 h1:ixhCt93XkJ98kGposQ54+bl0IK6XwqB40AsMynU7Z8E=
@@ -16,6 +20,8 @@ github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e h1:4dAU9FXIyQktp
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
github.com/asdine/storm/v3 v3.2.1 h1:I5AqhkPK6nBZ/qJXySdI7ot5BlXSZ7qvDY1zAn5ZJac=
github.com/asdine/storm/v3 v3.2.1/go.mod h1:LEpXwGt4pIqrE/XcTvCnZHT5MgZCV6Ub9q7yQzOFWr0=
github.com/at-wat/ebml-go v0.19.4 h1:6F767ThzH/JqNUsF79PwRFP0pwUF5ALXWJ6pXuiolI8=
github.com/at-wat/ebml-go v0.19.4/go.mod h1:wu3fQj+DWo6SOOl9c4yDi+gFs9Oa96Z2B5qxo1OxFYY=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/boombuler/barcode v1.1.0 h1:ChaYjBR63fr4LFyGn8E8nt7dBSt3MiU3zMOZqFvVkHo=
github.com/boombuler/barcode v1.1.0/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
@@ -23,6 +29,7 @@ github.com/coreos/go-oidc/v3 v3.21.0 h1:wZo4Q9Pum8dYEj0eMUPrqR+kvuGkeUplbLpNCkBq
github.com/coreos/go-oidc/v3 v3.21.0/go.mod h1:DYCf24+ncYi+XkIH97GY1+dqoRlbaSI26KVTCI9SrY4=
github.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA=
github.com/coreos/go-systemd/v22 v22.7.0/go.mod h1:xNUYtjHu2EDXbsxz1i41wouACIwT7Ybq9o0BQhMwD0w=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -47,16 +54,22 @@ github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349 h1:DilThiXje
github.com/dsoprea/go-utility/v2 v2.0.0-20221003172846-a3e1774ef349/go.mod h1:4GC5sXji84i/p+irqghpPFZBF8tRN/Q7+700G0/DLe8=
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
github.com/ebitengine/purego v0.11.2 h1:WWiTx8rD8zYdOwSj8O/NiNbsOvSAgJMTlyV1XlbUSeU=
github.com/ebitengine/purego v0.11.2/go.mod h1:DCHPP08djqhNSoTfImcnHYQRZmd0qhakvrozqaEYhGQ=
github.com/evanoberholster/imagemeta v1.1.0 h1:0iBtgVxLORZTZAlKxRcNodBmjMfHCSFwUh8Asmi83oI=
github.com/evanoberholster/imagemeta v1.1.0/go.mod h1:nmf6le+qSyoRAXgp2psPbtfkxcEgf/AZGgai9pnN8d0=
github.com/fxamacker/cbor/v2 v2.9.6 h1:NHZPxuUjJXQjX9itA785y/5NKSlI4HQ+S7noZsmI7hs=
github.com/fxamacker/cbor/v2 v2.9.6/go.mod h1:4EaZp6ClVunETjQ4GHmGNzysDK91Lr+r6Yz6tJqzVCw=
github.com/gabriel-vasile/mimetype v1.4.16 h1:5izNFuvA37st1dDRAcA7TtmngBmRgiIPDmTtiEvjx1E=
github.com/gabriel-vasile/mimetype v1.4.16/go.mod h1:3oFFwxX8kTklZwzIl2aQEDZT0lKY8xH92ST88bAwba4=
github.com/gen2brain/go-fitz v1.28.2 h1:845G85N5TUgnq5oDqyYrW0JvehAkeo35UkkK2dJtW1M=
github.com/gen2brain/go-fitz v1.28.2/go.mod h1:pY2hqAjp9Zy7qfPI2gwbJMHBFAdZpVXOLrRxD82l3Bs=
github.com/gen2brain/folio v0.2.8 h1:9iDI7h+zt4mjwEnzm1x8Pl9BrWqLwUwB74X/fnc1PZs=
github.com/gen2brain/folio v0.2.8/go.mod h1:4/PcXfaXIYmW1nwQ4bJeE+aG5ybG58Aw9xrwODXaXII=
github.com/gen2brain/gav1d v0.2.6 h1:7TSkrWTdE22k5Ul7vYZML7Nu80gNW6OovV7FSCITSE0=
github.com/gen2brain/gav1d v0.2.6/go.mod h1:ReFKjHq7pvRhEk+pgRPAo9gOlcrf51sdzVcAbbX7e0E=
github.com/gen2brain/h265 v0.2.4 h1:JUSbGkRAYZpd7ucCL192yx4ViEo75ZO6OEdm1TQt9qM=
github.com/gen2brain/h265 v0.2.4/go.mod h1:kefpLxLhGpNCUG/q/aabJcD6GqB9Nhet3fNi9xX9Wzw=
github.com/gen2brain/jxl v0.2.0 h1:cXSQnfbcC0JGFzaI344VzPMbe7mh6rYc0tjPcBUsE4U=
github.com/gen2brain/jxl v0.2.0/go.mod h1:VjJHRai/8I8Es01mULqyiOihRBOjJr1Zo+NoN/MyIwo=
github.com/gen2brain/vpx v0.2.1 h1:UD6tibcs/zlaIOKNFXeVyZuHBEaW9GYfxKSxvAN9cc8=
github.com/gen2brain/vpx v0.2.1/go.mod h1:EVIdqguMSiapbtjDm4AWgFgYYJ4d+k8trXqjxYZ/4XE=
github.com/go-asn1-ber/asn1-ber v1.5.8 h1:H9AZkK22UOmfX8J84ubyaZxKJZ3FMHVwn8swoMML7iQ=
github.com/go-asn1-ber/asn1-ber v1.5.8/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
github.com/go-errors/errors v1.0.1/go.mod h1:f4zRHt4oKfwPJE5k8C9vpYG+aDHdBFUsgrm6/TyX73Q=
@@ -104,6 +117,8 @@ github.com/go-playground/universal-translator v0.18.2 h1:LCsMLC9RzmbUMNUPVYD15dm
github.com/go-playground/universal-translator v0.18.2/go.mod h1:67VZIMp5lQpDWlnStOct22q1bkdJGJqHghbOtmkawxk=
github.com/go-playground/validator/v10 v10.30.5 h1:YyCXvVShZbs2Sm3Mb53eNOlhRXctSOzW5QJAouCTZL4=
github.com/go-playground/validator/v10 v10.30.5/go.mod h1:wEqiaov48pXX1kjhc3Da8y0M0Dtg/BK7gurFBLgwFrQ=
github.com/go-test/deep v1.1.0 h1:WOcxcdHcvdgThNXjw0t76K42FXTU7HpNQWHpA2HHNlg=
github.com/go-test/deep v1.1.0/go.mod h1:5C2ZWiW0ErCdrYzpqxLbTX7MG14M9iiw8DgHncVwcsE=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/go-webauthn/webauthn v0.18.2 h1:0BeftmEHU7i3Dv0VFwBtidy/ba37Vcdjvqst9EYu8Sk=
@@ -139,14 +154,21 @@ github.com/google/go-tpm-tools v0.3.13-0.20230620182252-4639ecce2aba h1:qJEJcuLz
github.com/google/go-tpm-tools v0.3.13-0.20230620182252-4639ecce2aba/go.mod h1:EFYHy8/1y2KfgTAsx7Luu7NGhoxtuVHnNo8jE7FikKc=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gtsteffaniak/go-av1 v1.0.0 h1:CILp67bcOtiDXvnrQubnH9ctub0OSKnUNA2JJuoIP+c=
github.com/gtsteffaniak/go-av1 v1.0.0/go.mod h1:C/dIrjfdUcHgdyChr2YXwLY3pIVjENI9npSd98PU4lE=
github.com/gtsteffaniak/go-cache v1.1.0 h1:ni/2cJpCxpGmaMZd92iTDIGOy+yUcMcxQuW7bSKhkCE=
github.com/gtsteffaniak/go-cache v1.1.0/go.mod h1:rHUZvf4huhMX/bnNK4/oQ3sNAE2fYC2jFpAWZiBPWac=
github.com/gtsteffaniak/go-ffmpeg v1.4.0 h1:XkNFlx6koCCh8cFXn9HxYEEpvLlbnxmIvWX+5HLuN+8=
github.com/gtsteffaniak/go-ffmpeg v1.4.0/go.mod h1:FefhoD+KZzHWqBaUpntcKp6lXtUzdFW00F74axnENUQ=
github.com/gtsteffaniak/go-ffmpeg v1.6.2 h1:gtvBg3G4PHO1Z/ZIPEK/XP4XT/g+l9S0Rpmmq6v5tnA=
github.com/gtsteffaniak/go-ffmpeg v1.6.2/go.mod h1:FefhoD+KZzHWqBaUpntcKp6lXtUzdFW00F74axnENUQ=
github.com/gtsteffaniak/go-logger v1.1.0 h1:baFds+iibuLLNpNLwygz32bDqdsunI6dGWcdxwMIVKI=
github.com/gtsteffaniak/go-logger v1.1.0/go.mod h1:rapapTju4D7bwt6h4JOdZNUXDjnp0WkTBd6TN5yWglI=
github.com/gtsteffaniak/go-media v1.2.0 h1:lbNL/G86uzzH8NaT45DBHKm1r0P7yndIo9pNC7gdqQs=
github.com/gtsteffaniak/go-media v1.2.0/go.mod h1:EH+5KAVLWfFl0g4W0Xni0qJJH8MUDIhAw4wPcC0mhxY=
github.com/gtsteffaniak/go-vpx v1.0.1 h1:9lSI8yoz6omDChaeV0AJrhdfe2YTrWvJrDENI/bnkAM=
github.com/gtsteffaniak/go-vpx v1.0.1/go.mod h1:tRCdDZ15oZZTsiwmQAwyH56PTt1jn1C1O0P03oOkoJ8=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
@@ -178,6 +200,7 @@ github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfn
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
@@ -185,10 +208,10 @@ github.com/leodido/go-urn v1.5.0 h1:pLqT2kq1zpHW/1D18QMjMpdtX7cekxqtJJjg5ANyWw0=
github.com/leodido/go-urn v1.5.0/go.mod h1:9BORnCDhdPBJNDEX+w1bJisa8yOKYi116VeO96s4ifE=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-sqlite3 v1.14.52 h1:wVbm2Qnf4OXkqhBTSPuCRZDRnxfbVrrmiCEroVdog8U=
github.com/mattn/go-sqlite3 v1.14.52/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/ncruces/go-strftime v1.1.0 h1:vtPmPWMkb539RGV9WpWT9EwJGvrOCKvMNI4obWL47Hk=
github.com/ncruces/go-strftime v1.1.0/go.mod h1:yVaq7iwCN09vHRrKBrrflXT0M9tTMG2KmZ9Fj7xeG/g=
github.com/orcaman/writerseeker v0.0.0-20200621085525-1d3f536ff85e h1:s2RNOM/IGdY0Y6qfTeUKhDawdHDpK9RGBdx80qN4Ttw=
github.com/orcaman/writerseeker v0.0.0-20200621085525-1d3f536ff85e/go.mod h1:nBdnFKj15wFbf94Rwfq4m30eAcyY9V/IyKAGQFtqkW0=
github.com/philhofer/fwd v1.2.1 h1:UtpS73q8bFgy/yH3gCq8LSNvj5gv/FowNOiWZa7uKwQ=
github.com/philhofer/fwd v1.2.1/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
@@ -209,12 +232,10 @@ github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/swaggo/files v1.0.1 h1:J1bVJ4XHZNq0I46UU90611i9/YzdrF7x92oX1ig5IdE=
github.com/swaggo/files v1.0.1/go.mod h1:0qXmMNH6sXNf+73t65aKeB+ApmgxdnkQzVTAj2uaMUg=
github.com/swaggo/http-swagger v1.3.4 h1:q7t/XLx0n15H1Q9/tk3Y9L4n210XzJF5WtnDX64a5ww=
github.com/swaggo/http-swagger v1.3.4/go.mod h1:9dAh0unqMBAlbp1uE2Uc2mQTxNMU/ha4UbucIg1MFkQ=
github.com/sunfish-shogi/bufseekio v0.0.0-20210207115823-a4185644b365/go.mod h1:dEzdXgvImkQ3WLI+0KQpmEx8T/C/ma9KeS3AfmU899I=
github.com/swaggo/swag v1.16.6 h1:qBNcx53ZaX+M5dxVyTrgQ0PJ/ACK+NzhwcbieTt+9yI=
github.com/swaggo/swag v1.16.6/go.mod h1:ngP2etMK5a0P3QBizic5MEwpRmluJZPHjXcMoj4Xesg=
github.com/tinylib/msgp v1.6.5 h1:iAH6XTP7BpzErjBZlujJQXxA+GmRi2YWs6M4KdLtNbE=
@@ -250,7 +271,6 @@ golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20221002022538-bcab6841153b/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
golang.org/x/oauth2 v0.37.0 h1:JUlcxA8oAtauLfiH8FX2/FkAWHAdi0QtGCGc+hofE98=
@@ -260,6 +280,7 @@ golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -269,12 +290,10 @@ golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA=
golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.47.0 h1:cpyOwQkr2Nzxe2p6KnrTG7Ei2wtzjsuSjhpXXepZ/BM=
golang.org/x/term v0.47.0/go.mod h1:efAfAuG6RKABvg1ArGebRkvX6LXul14SM67o2e4mUCY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -282,7 +301,6 @@ golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc=
golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
@@ -302,10 +320,15 @@ google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQ
google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/src-d/go-billy.v4 v4.3.2 h1:0SQA1pRztfTFx2miS8sA97XvooFeNOmvUenF4o0EcVg=
gopkg.in/src-d/go-billy.v4 v4.3.2/go.mod h1:nDjArDMp+XMs1aFAESLRjfGSgfvoYN0hDfzEk0GjC98=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.7/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
+20 -12
View File
@@ -141,6 +141,14 @@ func TestUserUpdateChangesExpandsNestedStructFields(t *testing.T) {
}
}
// Mock source paths used only as lookup keys in tests. They never touch the
// filesystem. They are relative (and not equal to the source names) so the
// tests are portable across machines.
const (
testDownloadsPath = "./testdata/Downloads"
testAccessPath = "./testdata/playwright-files"
)
type activityChangePair struct {
from string
to string
@@ -150,9 +158,9 @@ func testDownloadsSourceConfig() *users.SourceConfigProvider {
return &users.SourceConfigProvider{
GetSourceByPath: func(path string) (users.SourceInfo, bool) {
switch path {
case "/Users/steffag/Downloads":
case testDownloadsPath:
return users.SourceInfo{Path: path, Name: "Downloads"}, true
case "/Users/steffag/git/personal/filebrowser/frontend/tests/playwright-files":
case testAccessPath:
return users.SourceInfo{Path: path, Name: "access"}, true
default:
return users.SourceInfo{}, false
@@ -229,8 +237,8 @@ func TestUserUpdateChangesExpandsPreviewBulkToggle(t *testing.T) {
func TestUserUpdateChangesExpandsBackendSourcePermissions(t *testing.T) {
withTestSourceConfig(t, testDownloadsSourceConfig())
downloadsPath := "/Users/steffag/Downloads"
accessPath := "/Users/steffag/git/personal/filebrowser/frontend/tests/playwright-files"
downloadsPath := testDownloadsPath
accessPath := testAccessPath
beforePerms := users.MarkSourceFilePermissionsConfigured(users.SourceFilePermissions{
View: true, Download: true,
})
@@ -274,8 +282,8 @@ func TestUserUpdateChangesExpandsBackendSourcePermissions(t *testing.T) {
func TestUserUpdateChangesExpandsScopePermissions(t *testing.T) {
withTestSourceConfig(t, testDownloadsSourceConfig())
downloadsPath := "/Users/steffag/Downloads"
accessPath := "/Users/steffag/git/personal/filebrowser/frontend/tests/playwright-files"
downloadsPath := testDownloadsPath
accessPath := testAccessPath
basePerms := users.MarkSourceFilePermissionsConfigured(users.SourceFilePermissions{
View: true, Download: true,
})
@@ -353,9 +361,9 @@ func TestSidebarLinksFieldChangeIgnoresPathVsName(t *testing.T) {
withTestSourceConfig(t, &users.SourceConfigProvider{
GetSourceByPath: func(path string) (users.SourceInfo, bool) {
switch path {
case "/Users/steffag/Downloads":
case testDownloadsPath:
return users.SourceInfo{Path: path, Name: "Downloads"}, true
case "/Users/steffag/git/personal/filebrowser/frontend/tests/playwright-files":
case testAccessPath:
return users.SourceInfo{Path: path, Name: "access"}, true
default:
return users.SourceInfo{}, false
@@ -364,9 +372,9 @@ func TestSidebarLinksFieldChangeIgnoresPathVsName(t *testing.T) {
GetSourceByName: func(name string) (users.SourceInfo, bool) {
switch name {
case "Downloads":
return users.SourceInfo{Path: "/Users/steffag/Downloads", Name: "Downloads"}, true
return users.SourceInfo{Path: testDownloadsPath, Name: "Downloads"}, true
case "access":
return users.SourceInfo{Path: "/Users/steffag/git/personal/filebrowser/frontend/tests/playwright-files", Name: "access"}, true
return users.SourceInfo{Path: testAccessPath, Name: "access"}, true
default:
return users.SourceInfo{}, false
}
@@ -380,8 +388,8 @@ func TestSidebarLinksFieldChangeIgnoresPathVsName(t *testing.T) {
ShowToolsInSidebar: true,
SidebarLinks: []users.SidebarLink{
toolsLink,
{Name: "Downloads", Category: "source", Target: "/", SourceName: "/Users/steffag/Downloads"},
{Name: "access", Category: "source", Target: "/", SourceName: "/Users/steffag/git/personal/filebrowser/frontend/tests/playwright-files"},
{Name: "Downloads", Category: "source", Target: "/", SourceName: testDownloadsPath},
{Name: "access", Category: "source", Target: "/", SourceName: testAccessPath},
},
},
},
+150 -28
View File
@@ -16,11 +16,13 @@ import (
"github.com/gtsteffaniak/filebrowser/backend/internal/database/users"
"github.com/gtsteffaniak/filebrowser/backend/internal/errors"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/gomedia"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-logger/logger"
"github.com/gtsteffaniak/go-media/pkg/media"
)
// CheckPermissionsFunc allows tests to override CheckPermissions behavior
@@ -93,7 +95,7 @@ func processDirectoryMetadata(response *iteminfo.ExtendedFileInfo, idx *indexing
// finalizeResponse handles final response adjustments (OnlyOffice ID, scope stripping).
func finalizeResponse(response *iteminfo.ExtendedFileInfo, info *iteminfo.FileInfo, realPath string, user *users.User, userScope string) {
// Add OnlyOffice ID if applicable
if settings.Config.Integrations.OnlyOffice.Secret != "" && info.Type != "directory" && iteminfo.IsOnlyOffice(info.Name) {
if settings.Config.Integrations.OnlyOffice.Secret != "" && !settings.Config.Integrations.OnlyOffice.IsCollabora() && info.Type != "directory" && iteminfo.IsOnlyOffice(info.Name) {
response.OnlyOfficeId = generateOfficeId(realPath)
}
@@ -380,7 +382,7 @@ func processContent(info *iteminfo.ExtendedFileInfo, idx *indexing.Index, opts u
}
err := extractVideoMetadata(context.Background(), extItem, info.RealPath, nil)
if err != nil {
logger.Debugf("failed to extract video metadata for file: "+info.RealPath, info.Name, err)
logger.Debugf("failed to extract video metadata for file: %s (%s): %v", info.RealPath, info.Name, err)
} else {
info.Metadata = extItem.Metadata
}
@@ -390,10 +392,13 @@ func processContent(info *iteminfo.ExtendedFileInfo, idx *indexing.Index, opts u
if exists {
info.GetSubtitles(parentInfo)
}
if opts.ExtractEmbeddedSubtitles {
subtitles := ffmpeg.DetectEmbeddedSubtitles(info.RealPath, info.ModTime)
info.Subtitles = append(info.Subtitles, subtitles...)
subtitles, ok := gomedia.EmbeddedSubtitles(info.RealPath, info.ModTime, func() ([]utils.SubtitleTrack, bool) {
return ffmpeg.DetectEmbeddedSubtitles(info.RealPath, info.ModTime), true
})
if !ok {
subtitles = ffmpeg.DetectEmbeddedSubtitles(info.RealPath, info.ModTime)
}
info.Subtitles = append(info.Subtitles, subtitles...)
return
}
@@ -463,6 +468,29 @@ func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
return fmt.Errorf("file with size %d MB exceeds metadata check limit: %d MB", fileInfo.Size()/1024/1024, maxSize)
}
// go-media fast path: tags, lyrics availability, and artwork. Unsupported or
// failed reads fall through to the existing dhowden/tag code below.
if info, ok := gomedia.Audio(realPath, getArt, func() (gomedia.AudioInfo, bool) { return legacyAudioInfo(realPath, getArt) }); ok {
item.Metadata = &iteminfo.MediaMetadata{
Title: info.Tags.Title,
Artist: info.Tags.Artist,
Album: info.Tags.Album,
Year: info.Tags.Year,
Genre: info.Tags.Genre,
Track: info.Tags.Track,
HasLyrics: info.HasLyrics,
}
if getDuration {
audioDuration(ctx, item, realPath, ffmpegService)
}
if n := len(info.Artwork); n > 0 && n <= 5*1024*1024 {
item.Metadata.AlbumArt = info.Artwork
} else if n > 5*1024*1024 {
logger.Debugf("Skipping album art for %s: too large (%d bytes)", realPath, n)
}
return nil
}
m, err := tag.ReadFrom(file)
if err != nil {
return err
@@ -486,15 +514,7 @@ func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
// Extract duration ONLY if explicitly requested using the ffmpeg VideoService
// This respects concurrency limits and gracefully handles missing ffmpeg
if getDuration {
service := ffmpegService
if service == nil {
service = ffmpeg.Get()
}
if service != nil {
if duration, err := service.GetMediaDuration(ctx, realPath); err == nil {
item.Metadata.Duration = int(duration)
}
}
audioDuration(ctx, item, realPath, ffmpegService)
}
if !getArt {
@@ -514,6 +534,30 @@ func extractAudioMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo,
return nil
}
// audioDuration sets the duration on item.Metadata. go-media reads it from the
// container header first, then ffmpeg is used.
func audioDuration(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPath string, ffmpegService *ffmpeg.FFmpegService) {
service := ffmpegService
if service == nil {
service = ffmpeg.Get()
}
// The shadow run can outlive the request, so it uses a context that is not cancelled with it.
shadowCtx := context.WithoutCancel(ctx)
legacy := func() (float64, bool) {
d, err := service.GetMediaDuration(shadowCtx, realPath)
return d, err == nil
}
if duration, ok := gomedia.Duration(realPath, legacy); ok {
item.Metadata.Duration = int(duration)
return
}
if service != nil {
if duration, err := service.GetMediaDuration(ctx, realPath); err == nil {
item.Metadata.Duration = int(duration)
}
}
}
var lyricsSidecarExts = []string{".elrc", ".lrc", ".srt", ".vtt"}
func lyricsSidecar(dir, nameWithoutExt, ext string) (path string, ok bool) {
@@ -583,6 +627,11 @@ func hasLyricsSidecar(realPath string) bool {
// Returns raw lyrics from an audio file (from embedded tags or from sidecar files with the same name).
func ExtractLyrics(realPath string) (string, string, error) {
// go-media fast path: sidecar first, then embedded tags. Falls through to the
// existing code below when unsupported or failed.
if text, format, ok := gomedia.Lyrics(realPath, func() (gomedia.LyricsResult, bool) { return legacyLyrics(realPath) }); ok {
return text, format, nil
}
if data, ext, ok := findLyricsSidecar(realPath); ok {
return string(data), ext, nil
}
@@ -601,27 +650,100 @@ func ExtractLyrics(realPath string) (string, string, error) {
return "", "", nil
}
// legacyAudioInfo is the dhowden tag read used as the shadow-mode comparison
// for gomedia.Audio. It returns the same fields go-media returns, so the two
// can be compared. Cover bytes are read only when withArt is true.
func legacyAudioInfo(realPath string, withArt bool) (gomedia.AudioInfo, bool) {
f, err := os.Open(realPath)
if err != nil {
return gomedia.AudioInfo{}, false
}
defer f.Close()
m, err := tag.ReadFrom(f)
if err != nil {
return gomedia.AudioInfo{}, false
}
track, _ := m.Track()
info := gomedia.AudioInfo{
Tags: media.Tags{
Title: m.Title(),
Artist: m.Artist(),
Album: m.Album(),
Year: m.Year(),
Genre: m.Genre(),
Track: track,
},
HasLyrics: strings.TrimSpace(m.Lyrics()) != "" || hasLyricsSidecar(realPath),
}
if withArt {
if pic := m.Picture(); pic != nil {
info.Artwork = pic.Data
}
}
return info, true
}
// legacyLyrics is the sidecar and dhowden lyrics read used as the shadow-mode
// comparison for gomedia.Lyrics. Sidecar first, then embedded, as in go-media.
func legacyLyrics(realPath string) (gomedia.LyricsResult, bool) {
if data, ext, ok := findLyricsSidecar(realPath); ok {
return gomedia.LyricsResult{Text: strings.TrimSpace(string(data)), Format: ext}, true
}
f, err := os.Open(realPath)
if err != nil {
return gomedia.LyricsResult{}, false
}
defer f.Close()
m, err := tag.ReadFrom(f)
if err != nil {
return gomedia.LyricsResult{}, false
}
text := strings.TrimSpace(m.Lyrics())
if text == "" {
return gomedia.LyricsResult{}, false
}
return gomedia.LyricsResult{Text: text, Format: "embedded"}, true
}
// extractVideoMetadata extracts duration and codec info from video files using ffprobe.
// If ffmpegService is nil, a new service will be created (for backward compatibility)
func extractVideoMetadata(ctx context.Context, item *iteminfo.ExtendedItemInfo, realPath string, ffmpegService *ffmpeg.FFmpegService) error {
service := ffmpegService
if service == nil {
service = ffmpeg.Get()
// go-media fast path. Works without ffmpeg installed.
probeService := ffmpegService
if probeService == nil {
probeService = ffmpeg.Get()
}
if service == nil {
return nil
// The shadow run can outlive the request, so it uses a context that is not cancelled with it.
shadowCtx := context.WithoutCancel(ctx)
legacyProbe := func() (gomedia.ProbeResult, bool) {
info, err := probeService.ProbeFile(shadowCtx, realPath)
if err != nil || info == nil {
return gomedia.ProbeResult{}, false
}
return *info, true
}
info, ok := gomedia.Probe(realPath, legacyProbe)
if !ok {
service := ffmpegService
if service == nil {
service = ffmpeg.Get()
}
if service == nil {
return nil
}
info, err := service.ProbeFile(ctx, realPath)
if err != nil {
duration, durErr := service.GetMediaDuration(ctx, realPath)
if durErr != nil {
return err
probed, err := service.ProbeFile(ctx, realPath)
if err != nil {
duration, durErr := service.GetMediaDuration(ctx, realPath)
if durErr != nil {
return err
}
if duration > 0 {
item.Metadata = &iteminfo.MediaMetadata{Duration: int(duration)}
}
return nil
}
if duration > 0 {
item.Metadata = &iteminfo.MediaMetadata{Duration: int(duration)}
}
return nil
info = *probed
}
meta := &iteminfo.MediaMetadata{}
+52 -49
View File
@@ -55,6 +55,8 @@ const (
var types = map[string]string{
".3dm": "x-world/x-3dmf",
".3dmf": "x-world/x-3dmf",
".3g2": "video/3gpp2",
".3gp": "video/3gpp",
".7z": "application/x-7z-compressed",
".a": "application/octet-stream",
".aab": "application/x-authorware-bin",
@@ -525,6 +527,7 @@ var types = map[string]string{
".wmlc": "application/vnd.wap.wmlc",
".wmls": "text/vnd.wap.wmlscript",
".wmlsc": "application/vnd.wap.wmlscriptc",
".wmv": "video/x-ms-wmv",
".word": "application/msword",
".wp5": "application/wordperfect",
".wp6": "application/wordperfect",
@@ -650,55 +653,55 @@ var types = map[string]string{
".tmp": "text/tmp",
".temp": "text/tmp",
// the ones below are some from: https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types
".adp": "audio/adpcm",
".mp4a": "audio/mp4",
".mp2a": "audio/mpeg",
".m3a": "audio/mpeg",
".sil": "audio/silk",
".uva": "audio/vnd.dece.audio",
".uvva": "audio/vnd.dece.audio",
".eol": "audio/vnd.digital-winds",
".dra": "audio/vnd.dra",
".dts": "audio/vnd.dts",
".dtshd": "audio/vnd.dts.hd",
".lvp": "audio/vnd.lucent.voice",
".pya": "audio/vnd.ms-playready.media.pya",
".rip": "audio/vnd.rip",
".mka": "audio/x-matroska",
".h261": "video/h261",
".h263": "video/h263",
".h264": "video/h264",
".jpgv": "video/jpeg",
".jpm": "video/jpm",
".jpgm": "video/jpm",
".mj2": "video/mj2",
".mjp2": "video/mj2",
".m2t": "video/mp2t",
".m2ts": "video/mp2t",
".mts": "video/mp2t",
".mp4v": "video/mp4",
".mpg4": "video/mp4",
".uvh": "video/vnd.dece.hd",
".uvvh": "video/vnd.dece.hd",
".uvm": "video/vnd.dece.mobile",
".uvvm": "video/vnd.dece.mobile",
".uvp": "video/vnd.dece.pd",
".uvvp": "video/vnd.dece.pd",
".uvs": "video/vnd.dece.sd",
".uvvs": "video/vnd.dece.sd",
".uvv": "video/vnd.dece.video",
".uvvv": "video/vnd.dece.video",
".dvb": "video/vnd.dvb.file",
".fvt": "video/vnd.fvt",
".mxu": "video/vnd.mpegurl",
".pyv": "video/vnd.ms-playready.media.pyv",
".uvu": "video/vnd.uvvu.mp4",
".uvvu": "video/vnd.uvvu.mp4",
".f4v": "video/x-f4v",
".mk3d": "video/x-matroska",
".mks": "video/x-matroska",
".vob": "video/x-ms-vob",
".smv": "video/x-smv",
".adp": "audio/adpcm",
".mp4a": "audio/mp4",
".mp2a": "audio/mpeg",
".m3a": "audio/mpeg",
".sil": "audio/silk",
".uva": "audio/vnd.dece.audio",
".uvva": "audio/vnd.dece.audio",
".eol": "audio/vnd.digital-winds",
".dra": "audio/vnd.dra",
".dts": "audio/vnd.dts",
".dtshd": "audio/vnd.dts.hd",
".lvp": "audio/vnd.lucent.voice",
".pya": "audio/vnd.ms-playready.media.pya",
".rip": "audio/vnd.rip",
".mka": "audio/x-matroska",
".h261": "video/h261",
".h263": "video/h263",
".h264": "video/h264",
".jpgv": "video/jpeg",
".jpm": "video/jpm",
".jpgm": "video/jpm",
".mj2": "video/mj2",
".mjp2": "video/mj2",
".m2t": "video/mp2t",
".m2ts": "video/mp2t",
".mts": "video/mp2t",
".mp4v": "video/mp4",
".mpg4": "video/mp4",
".uvh": "video/vnd.dece.hd",
".uvvh": "video/vnd.dece.hd",
".uvm": "video/vnd.dece.mobile",
".uvvm": "video/vnd.dece.mobile",
".uvp": "video/vnd.dece.pd",
".uvvp": "video/vnd.dece.pd",
".uvs": "video/vnd.dece.sd",
".uvvs": "video/vnd.dece.sd",
".uvv": "video/vnd.dece.video",
".uvvv": "video/vnd.dece.video",
".dvb": "video/vnd.dvb.file",
".fvt": "video/vnd.fvt",
".mxu": "video/vnd.mpegurl",
".pyv": "video/vnd.ms-playready.media.pyv",
".uvu": "video/vnd.uvvu.mp4",
".uvvu": "video/vnd.uvvu.mp4",
".f4v": "video/x-f4v",
".mk3d": "video/x-matroska",
".mks": "video/x-matroska",
".vob": "video/x-ms-vob",
".smv": "video/x-smv",
}
//nolint:gochecknoinits
@@ -0,0 +1,25 @@
package files
import (
"mime"
"testing"
)
// Video containers that browsers cannot play must still be detected as video so
// the preview routes to the transcode UI instead of showing "no preview".
func TestVideoContainerExtensionsAreRegisteredAsVideo(t *testing.T) {
tests := map[string]string{
".wmv": "video/x-ms-wmv",
".3gp": "video/3gpp",
".3g2": "video/3gpp2",
".mp4": "video/mp4",
".avi": "video/x-msvideo",
".mkv": "video/x-matroska",
}
for ext, want := range tests {
got := mime.TypeByExtension(ext)
if got != want {
t.Errorf("mime.TypeByExtension(%q) = %q, want %q", ext, got, want)
}
}
}
@@ -3,6 +3,8 @@ package fileutils
import (
"io/fs"
"net/http"
"github.com/gtsteffaniak/go-logger/logger"
)
var (
@@ -28,7 +30,17 @@ func InitAssetFS(embeddedAssets fs.FS, useEmbedded bool, devMode bool) {
// Dev mode: serve frontend source assets (templates, fonts, icons)
assetFs = dirFS{Dir: http.Dir("../frontend/public")}
} else {
assetFs = dirFS{Dir: http.Dir("internal/web/dist")}
// Non-dev disk build (e.g. Docker): dist is shipped beside the binary. Preload it
// into memory when possible; otherwise serve from disk.
diskAssets := dirFS{Dir: http.Dir("internal/web/dist")}
cached, err := loadMemFS(diskAssets)
if err != nil {
logger.Warningf("could not preload frontend assets into memory, serving from disk: %v", err)
assetFs = diskAssets
} else {
logger.Debugf("preloaded %d frontend assets (%d bytes) into memory", cached.Len(), cached.Size())
assetFs = cached
}
}
}
@@ -0,0 +1,110 @@
package fileutils
import (
"bytes"
"io/fs"
"time"
)
// memFS is a read-only fs.FS that holds every file's contents in memory.
// It is built once at startup from a disk-backed filesystem so that asset
// requests never touch the disk.
type memFS struct {
files map[string]memFile
}
type memFile struct {
data []byte
modTime time.Time
}
// loadMemFS reads every regular file under src into memory.
func loadMemFS(src fs.FS) (*memFS, error) {
m := &memFS{files: make(map[string]memFile)}
err := fs.WalkDir(src, ".", func(name string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
data, err := fs.ReadFile(src, name)
if err != nil {
return err
}
info, err := d.Info()
if err != nil {
return err
}
m.files[name] = memFile{data: data, modTime: info.ModTime()}
return nil
})
if err != nil {
return nil, err
}
return m, nil
}
// Size returns the total number of bytes held in memory.
func (m *memFS) Size() int {
total := 0
for _, f := range m.files {
total += len(f.data)
}
return total
}
// Len returns the number of files held in memory.
func (m *memFS) Len() int {
return len(m.files)
}
// Open implements fs.FS. Only regular files are supported, which is all the
// asset callers need (ReadFile, Stat and template parsing).
func (m *memFS) Open(name string) (fs.File, error) {
if !fs.ValidPath(name) {
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrInvalid}
}
f, ok := m.files[name]
if !ok {
return nil, &fs.PathError{Op: "open", Path: name, Err: fs.ErrNotExist}
}
return &memOpenFile{
Reader: bytes.NewReader(f.data),
info: memFileInfo{
name: baseName(name),
size: int64(len(f.data)),
modTime: f.modTime,
},
}, nil
}
type memOpenFile struct {
*bytes.Reader
info memFileInfo
}
func (f *memOpenFile) Stat() (fs.FileInfo, error) { return f.info, nil }
func (f *memOpenFile) Close() error { return nil }
type memFileInfo struct {
name string
size int64
modTime time.Time
}
func (i memFileInfo) Name() string { return i.name }
func (i memFileInfo) Size() int64 { return i.size }
func (i memFileInfo) Mode() fs.FileMode { return 0o444 }
func (i memFileInfo) ModTime() time.Time { return i.modTime }
func (i memFileInfo) IsDir() bool { return false }
func (i memFileInfo) Sys() any { return nil }
func baseName(name string) string {
for i := len(name) - 1; i >= 0; i-- {
if name[i] == '/' {
return name[i+1:]
}
}
return name
}
@@ -0,0 +1,69 @@
package fileutils
import (
"io/fs"
"net/http"
"os"
"path/filepath"
"testing"
)
func TestLoadMemFSServesFromMemory(t *testing.T) {
dir := t.TempDir()
files := map[string]string{
"index.html": "<html>index</html>",
"assets/app.js.gz": "gzip-bytes",
"fonts/icons.woff2": "font-bytes",
}
for name, content := range files {
path := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
m, err := loadMemFS(dirFS{Dir: http.Dir(dir)})
if err != nil {
t.Fatalf("loadMemFS: %v", err)
}
if m.Len() != len(files) {
t.Fatalf("loaded %d files, want %d", m.Len(), len(files))
}
// Remove everything from disk: the cache must still serve every file.
if removeErr := os.RemoveAll(dir); removeErr != nil {
t.Fatal(removeErr)
}
for name, want := range files {
got, readErr := fs.ReadFile(m, name)
if readErr != nil {
t.Fatalf("ReadFile(%s) after disk removal: %v", name, readErr)
}
if string(got) != want {
t.Errorf("ReadFile(%s) = %q, want %q", name, got, want)
}
info, statErr := fs.Stat(m, name)
if statErr != nil {
t.Fatalf("Stat(%s): %v", name, statErr)
}
if info.IsDir() || info.Size() != int64(len(want)) || info.Name() != filepath.Base(name) {
t.Errorf("Stat(%s) = %+v, unexpected", name, info)
}
}
// Template-style glob on a plain name must resolve via Stat.
matches, globErr := fs.Glob(m, "index.html")
if globErr != nil || len(matches) != 1 {
t.Fatalf("Glob(index.html) = %v, %v", matches, globErr)
}
if _, readErr := fs.ReadFile(m, "missing.css"); readErr == nil {
t.Error("ReadFile(missing.css) should fail")
}
if _, readErr := fs.ReadFile(m, "../escape"); readErr == nil {
t.Error("invalid path should be rejected")
}
}
+2 -2
View File
@@ -205,8 +205,8 @@ func collectPayload(versionLabel string) (snapshotPayload, error) {
Integrations: integrationsPayload{
OnlyOfficeConfigured: strings.TrimSpace(settings.Config.Integrations.OnlyOffice.Url) != "",
OnlyOfficeViewOnly: settings.Config.Integrations.OnlyOffice.ViewOnly,
MediaEnabled: settings.MediaEnabled(),
HardwareAcceleration: settings.Config.Integrations.Media.HardwareAcceleration,
MediaEnabled: settings.FFmpegEnabled(),
HardwareAcceleration: settings.HardwareAccelerationEnabled(),
},
Features: featuresPayload{
DisablePreviews: settings.Config.Server.DisablePreviews,
+28
View File
@@ -1,11 +1,16 @@
package app
import (
"context"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/internal/activity"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/files"
"github.com/gtsteffaniak/filebrowser/backend/internal/auth"
"github.com/gtsteffaniak/filebrowser/backend/internal/state"
"github.com/gtsteffaniak/filebrowser/backend/internal/transcoding"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
// App holds wired dependencies for the running process.
@@ -17,6 +22,11 @@ type App struct {
// WireServices connects domain packages to the state store after Open.
func WireServices(store *state.Store) (*App, error) {
return WireServicesWithContext(context.Background(), store)
}
// WireServicesWithContext wires services using ctx for background workers.
func WireServicesWithContext(ctx context.Context, store *state.Store) (*App, error) {
filesSvc := files.New(store, store, store)
authSvc := auth.New(store)
files.SetDefault(filesSvc)
@@ -26,6 +36,24 @@ func WireServices(store *state.Store) (*App, error) {
if err := auth.InitWebAuthn(store); err != nil {
return nil, err
}
// Always create the manager: the engine re-checks ffmpeg availability per
// request, so a runtime admin settings change takes effect without restart.
storeDir, err := transcoding.NewStore(settings.TranscodeCacheDir())
if err != nil {
return nil, err
}
// Delivery URLs stay root-relative so a subpath deployment or alternate
// hostname still resolves and the session cookie is sent.
transcodeMgr, err := transcoding.NewManager(ctx, transcoding.NewMediaEngine(), storeDir,
strings.TrimRight(settings.Config.Http.BaseURL, "/"))
if err != nil {
return nil, err
}
// Exported so the web layer (and shutdown) can reach it without threading
// it through StartHttp's Deps.
transcoding.SetDefault(transcodeMgr)
return &App{
Store: store,
Files: filesSvc,
+2 -3
View File
@@ -45,9 +45,8 @@ type FrontendShareInfo struct {
type ShareLimits struct {
MaxBandwidth int `json:"maxBandwidth,omitempty"`
AllowedUsernames []string `json:"allowedUsernames,omitempty"`
PerUserDownloadLimit bool `json:"perUserDownloadLimit,omitempty"`
ExtractEmbeddedSubtitles bool `json:"extractEmbeddedSubtitles,omitempty"`
DownloadsLimit int `json:"downloadsLimit,omitempty"`
PerUserDownloadLimit bool `json:"perUserDownloadLimit,omitempty"`
DownloadsLimit int `json:"downloadsLimit,omitempty"`
QuotaLimitBytes int64 `json:"quotaLimitBytes,omitempty"`
HideFileExt string `json:"hideFileExt,omitempty"` // show hidden files based on extensions in shares
Banner string `json:"banner,omitempty"`
@@ -1,6 +1,3 @@
//go:build !cgosql
// +build !cgosql
package sql
import (
@@ -1,10 +0,0 @@
//go:build cgosql
// +build cgosql
package sql
import (
_ "github.com/mattn/go-sqlite3"
)
const SqliteDriver = "sqlite3"
+131
View File
@@ -0,0 +1,131 @@
package sql
import (
"fmt"
"github.com/gtsteffaniak/go-logger/logger"
)
// supportsFTS5 reports whether the driver can create an FTS5 table with the trigram
// tokenizer (SQLite >= 3.34 with FTS5 compiled in).
func (db *IndexDB) supportsFTS5() bool {
if _, err := db.Exec(`CREATE VIRTUAL TABLE IF NOT EXISTS temp.__fts_probe USING fts5(x, tokenize='trigram')`); err != nil {
return false
}
_, _ = db.Exec(`DROP TABLE IF EXISTS temp.__fts_probe`)
return true
}
// ftsTableExists reports whether the FTS search table is present.
func (db *IndexDB) ftsTableExists() bool {
var n int
if err := db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = 'index_items_fts'`).Scan(&n); err != nil {
return false
}
return n > 0
}
// FTSReady reports whether the trigram search index is available for queries.
func (db *IndexDB) FTSReady() bool {
return db.ftsReady.Load()
}
const ftsDDL = `
CREATE VIRTUAL TABLE IF NOT EXISTS index_items_fts USING fts5(
name, spath,
content='index_items', content_rowid='rowid',
tokenize='trigram', detail=none
);
CREATE TRIGGER IF NOT EXISTS index_items_fts_ai AFTER INSERT ON index_items BEGIN
INSERT INTO index_items_fts(rowid, name, spath) VALUES (new.rowid, new.name, new.spath);
END;
CREATE TRIGGER IF NOT EXISTS index_items_fts_ad AFTER DELETE ON index_items BEGIN
INSERT INTO index_items_fts(index_items_fts, rowid, name, spath) VALUES ('delete', old.rowid, old.name, old.spath);
END;
CREATE TRIGGER IF NOT EXISTS index_items_fts_au AFTER UPDATE OF name ON index_items
WHEN old.name IS NOT new.name BEGIN
INSERT INTO index_items_fts(index_items_fts, rowid, name, spath) VALUES ('delete', old.rowid, old.name, old.spath);
INSERT INTO index_items_fts(rowid, name, spath) VALUES (new.rowid, new.name, new.spath);
END;
`
// syncSearchIndex creates the FTS5 trigram index when enabled and drops it when
// disabled. The index is derived data: enabling it builds from existing rows (no
// rescan), and disabling it removes the table and triggers.
func (db *IndexDB) syncSearchIndex(enabled bool) {
if !enabled {
db.dropSearchIndex()
return
}
if !db.supportsFTS5() {
logger.Errorf("indexSqlConfig.fts5 is enabled but FTS5 trigram support is unavailable in this build; search will use the unindexed fallback.")
db.ftsReady.Store(false)
return
}
if err := db.ensureSpathColumn(); err != nil {
logger.Errorf("failed to add the spath column (%v); search will use the unindexed fallback", err)
db.ftsReady.Store(false)
return
}
existed := db.ftsTableExists()
if _, err := db.Exec(ftsDDL); err != nil {
logger.Errorf("failed to create the FTS search index (%v); search will use the unindexed fallback", err)
db.ftsReady.Store(false)
return
}
if !existed {
if _, err := db.Exec(`INSERT INTO index_items_fts(index_items_fts) VALUES('rebuild')`); err != nil {
logger.Errorf("failed to populate the FTS search index (%v); search will use the unindexed fallback", err)
db.ftsReady.Store(false)
return
}
}
db.ftsReady.Store(true)
}
// dropSearchIndex removes the FTS table and its triggers (used when the feature is
// disabled or the index needs a rebuild).
func (db *IndexDB) dropSearchIndex() {
db.ftsReady.Store(false)
if !db.ftsTableExists() {
return
}
if _, err := db.Exec(`
DROP TRIGGER IF EXISTS index_items_fts_ai;
DROP TRIGGER IF EXISTS index_items_fts_ad;
DROP TRIGGER IF EXISTS index_items_fts_au;
DROP TABLE IF EXISTS index_items_fts;
`); err != nil {
logger.Warningf("failed to drop the FTS search index: %v", err)
}
}
// checkFTSHealth compares the FTS index against the base table and runs a cheap
// internal consistency check. The full content check (rank=1) is far slower and is
// skipped.
func (db *IndexDB) checkFTSHealth() error {
var items, docs int
if err := db.QueryRow(`SELECT count(*) FROM index_items`).Scan(&items); err != nil {
return fmt.Errorf("count index_items: %w", err)
}
if err := db.QueryRow(`SELECT count(*) FROM index_items_fts_docsize`).Scan(&docs); err != nil {
return fmt.Errorf("count fts docsize: %w", err)
}
if items != docs {
return fmt.Errorf("search index row mismatch: index_items=%d fts=%d", items, docs)
}
if _, err := db.Exec(`INSERT INTO index_items_fts(index_items_fts, rank) VALUES('integrity-check', 0)`); err != nil {
return fmt.Errorf("fts integrity check: %w", err)
}
return nil
}
// optimizeFTS merges the FTS index segments; call after bulk deletes or heavy churn.
func (db *IndexDB) optimizeFTS() {
if !db.FTSReady() {
return
}
if _, err := db.Exec(`INSERT INTO index_items_fts(index_items_fts) VALUES('optimize')`); err != nil {
logger.Warningf("FTS optimize failed: %v", err)
}
}
+49 -3
View File
@@ -7,11 +7,12 @@ import (
"os"
"path/filepath"
"strings"
"sync/atomic"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/database/sqlitebusy"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-logger/logger"
)
@@ -19,6 +20,7 @@ import (
// It wraps the underlying sql.DB connection and provides type-safe methods.
type IndexDB struct {
*TempDB
ftsReady atomic.Bool
}
func createIndexDB(name string, journalMode string, lockingMode string, batchSize int, cacheSizeMB int, disableReuse bool) (*IndexDB, error) {
@@ -111,6 +113,17 @@ func NewIndexDB(name string, journalMode string, batchSize int, cacheSizeMB int,
idxDB.Close()
return nil, false, err
}
// The FTS5 search index is optional (indexSqlConfig.fts5, default off). Enabling it
// builds from existing rows; disabling it drops the table and triggers.
idxDB.syncSearchIndex(settings.Config.Server.IndexSqlConfig.FTS5)
if !disableReuse && !isNewDb && idxDB.FTSReady() {
if err := idxDB.checkFTSHealth(); err != nil {
logger.Warningf("[DB_INIT] FTS search index is unhealthy (%v); rebuilding", err)
idxDB.dropSearchIndex()
idxDB.syncSearchIndex(true)
}
}
go idxDB.startPeriodicCleanup()
return idxDB, isNewDb, nil
}
@@ -234,6 +247,7 @@ func (db *IndexDB) startPeriodicCleanup() {
} else if deletedCount > 0 {
logger.Infof("[DB_MAINTENANCE] Cleaned up %d stale index entries", deletedCount)
}
db.optimizeFTS()
}
}
@@ -259,13 +273,45 @@ func (db *IndexDB) CreateIndexTable() error {
CREATE INDEX IF NOT EXISTS idx_source_size ON index_items(source, size);
CREATE INDEX IF NOT EXISTS idx_last_updated ON index_items(source, last_updated);
`
_, err := db.Exec(query)
if err != nil {
if _, err := db.Exec(query); err != nil {
return err
}
return db.ensureCreatedTimeColumn()
}
// ensureSpathColumn adds the generated spath column (source:path) used by the FTS
// search index when it is missing from an existing table. PRAGMA table_xinfo is used
// because table_info omits generated columns.
func (db *IndexDB) ensureSpathColumn() error {
rows, err := db.Query(`PRAGMA table_xinfo(index_items)`)
if err != nil {
return err
}
found := false
for rows.Next() {
var cid, notNull, pk, hidden int
var name, columnType string
var defaultValue interface{}
if scanErr := rows.Scan(&cid, &name, &columnType, &notNull, &defaultValue, &pk, &hidden); scanErr != nil {
rows.Close()
return scanErr
}
if name == "spath" {
found = true
}
}
if rowsErr := rows.Err(); rowsErr != nil {
rows.Close()
return rowsErr
}
rows.Close()
if found {
return nil
}
_, err = db.Exec(`ALTER TABLE index_items ADD COLUMN spath TEXT GENERATED ALWAYS AS (source || ':' || path) VIRTUAL`)
return err
}
// ensureCreatedTimeColumn adds the created_time column to index_items when it is
// missing, so databases created before the column existed are upgraded in place.
func (db *IndexDB) ensureCreatedTimeColumn() error {
@@ -56,10 +56,17 @@ func testIndexSQLConfig(mode string) settings.IndexSqlConfig {
BatchSize: 1000,
CacheSizeMB: 32,
DisableReuse: false,
FTS5: true,
StartupIntegrityCheck: mode,
}
}
func testIndexSQLConfigNoFTS(mode string) settings.IndexSqlConfig {
cfg := testIndexSQLConfig(mode)
cfg.FTS5 = false
return cfg
}
func TestNewIndexDB_StartupProbe_NewFile(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(settings.IndexStartupIntegrityProbe))
@@ -129,3 +136,123 @@ func TestNewIndexDB_StartupOff(t *testing.T) {
}
defer db.Close()
}
// TestNewIndexDB_ReusesHealthyDB checks that a matching schema is reused, not recreated.
func TestNewIndexDB_ReusesHealthyDB(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(settings.IndexStartupIntegrityOff))
defer pop()
db, isNew, err := NewIndexDB("reuse_ok", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
if !isNew {
t.Fatalf("fresh database should report isNewDb=true")
}
if err = db.InsertItem("src1", "/a.txt", &iteminfo.FileInfo{
Path: "/a.txt",
ItemInfo: iteminfo.ItemInfo{Name: "a.txt", Type: "text/plain", ModTime: time.Unix(1, 0)},
}); err != nil {
t.Fatal(err)
}
if err = db.Close(); err != nil {
t.Fatal(err)
}
db2, isNew2, err := NewIndexDB("reuse_ok", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db2.Close()
if isNew2 {
t.Fatalf("a healthy database should be reused, not recreated")
}
if !db2.FTSReady() {
t.Fatalf("expected the FTS index to be ready on reuse")
}
}
// TestNewIndexDB_FTSDisabledWhenConfigOff checks that FTS is not created when disabled.
func TestNewIndexDB_FTSDisabledWhenConfigOff(t *testing.T) {
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfigNoFTS(settings.IndexStartupIntegrityOff))
defer pop()
db, _, err := NewIndexDB("fts_off", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db.Close()
if db.FTSReady() {
t.Fatalf("FTS should be disabled when indexSqlConfig.fts5 is false")
}
if db.ftsTableExists() {
t.Fatalf("the FTS table should not exist when disabled")
}
}
// TestNewIndexDB_FTSToggleBuildsAndDrops checks that enabling FTS builds from existing
// rows (no rescan) and disabling it removes the table.
func TestNewIndexDB_FTSToggleBuildsAndDrops(t *testing.T) {
dir := t.TempDir()
// Start with FTS off and insert a row.
pop := pushTestIndexConfig(t, dir, testIndexSQLConfigNoFTS(settings.IndexStartupIntegrityOff))
db, _, err := NewIndexDB("fts_toggle", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
if err = db.InsertItem("s1", "/a/report.txt", &iteminfo.FileInfo{
Path: "/a/report.txt",
ItemInfo: iteminfo.ItemInfo{Name: "report.txt", Type: "text/plain", ModTime: time.Unix(1, 0)},
}); err != nil {
t.Fatal(err)
}
if db.FTSReady() {
t.Fatalf("FTS should be off")
}
if err = db.Close(); err != nil {
t.Fatal(err)
}
pop()
// Reopen with FTS on: it should build from the existing row without a rescan.
pop2 := pushTestIndexConfig(t, dir, testIndexSQLConfig(settings.IndexStartupIntegrityOff))
db2, isNew, err := NewIndexDB("fts_toggle", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
if isNew {
t.Fatalf("reused database should not be recreated when enabling FTS")
}
if !db2.FTSReady() {
t.Fatalf("FTS should be ready after enabling")
}
if err = db2.checkFTSHealth(); err != nil {
t.Fatalf("health after enabling: %v", err)
}
filter := iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"report"}}, false, false)
if got := runSearchItems(t, db2, "s1", "/", filter); !sameSet(got, []string{"/a/report.txt"}) {
t.Fatalf("existing row should be searchable via FTS, got %v", got)
}
if err = db2.Close(); err != nil {
t.Fatal(err)
}
pop2()
// Disable again: the table and triggers should be dropped.
pop3 := pushTestIndexConfig(t, dir, testIndexSQLConfigNoFTS(settings.IndexStartupIntegrityOff))
defer pop3()
db3, _, err := NewIndexDB("fts_toggle", "OFF", 1000, 32, false)
if err != nil {
t.Fatal(err)
}
defer db3.Close()
if db3.FTSReady() {
t.Fatalf("FTS should be off after disabling")
}
if db3.ftsTableExists() {
t.Fatalf("the FTS table should be dropped when disabled")
}
}
+157 -81
View File
@@ -1,9 +1,12 @@
package sql
import (
"context"
"database/sql"
"fmt"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
)
// SearchItem represents a single search result row from the database
@@ -19,19 +22,15 @@ type SearchItem struct {
HasPreview bool
}
// appendNameGlobSQL adds AND (name GLOB ? OR ...) or AND (name GLOB ? AND ...) when joinAnd is true.
func appendNameGlobSQL(query string, args []interface{}, patterns []string, joinAnd, caseExact bool) (string, []interface{}) {
// appendGlob adds "AND (col GLOB ? [OR/AND] ...)" when patterns is non-empty.
func appendGlob(query string, args []interface{}, column string, patterns []string, joinAnd bool) (string, []interface{}) {
if len(patterns) == 0 {
return query, args
}
nameExpr := "name"
if !caseExact {
nameExpr = "lower(name)"
}
parts := make([]string, len(patterns))
for i := range patterns {
parts[i] = nameExpr + " GLOB ?"
args = append(args, patterns[i])
for i, p := range patterns {
parts[i] = column + " GLOB ?"
args = append(args, p)
}
join := " OR "
if joinAnd {
@@ -40,96 +39,173 @@ func appendNameGlobSQL(query string, args []interface{}, patterns []string, join
return query + " AND (" + strings.Join(parts, join) + ")", args
}
// appendLike adds "AND (col LIKE ? [OR/AND] ...)" when patterns is non-empty.
func appendLike(query string, args []interface{}, column string, patterns []string, joinAnd bool) (string, []interface{}) {
if len(patterns) == 0 {
return query, args
}
parts := make([]string, len(patterns))
for i, p := range patterns {
parts[i] = column + " LIKE ?"
args = append(args, p)
}
join := " OR "
if joinAnd {
join = " AND "
}
return query + " AND (" + strings.Join(parts, join) + ")", args
}
// useFTSPath reports whether the trigram index can serve this name filter.
func useFTSPath(db *IndexDB, filter *iteminfo.NameFilter) bool {
return filter != nil && filter.UseFTS && db.FTSReady()
}
// appendNamePrefilter narrows rows using the FTS trigram index (a superset of the
// exact predicate). It is only valid in the FTS query path.
func appendNamePrefilter(query string, args []interface{}, filter *iteminfo.NameFilter) (string, []interface{}) {
if filter == nil || len(filter.FTS) == 0 {
return query, args
}
if filter.PrefilterLike {
return appendLike(query, args, "f.name", filter.FTS, filter.JoinAnd)
}
return appendGlob(query, args, "f.name", filter.FTS, filter.JoinAnd)
}
// appendNameExact adds the exact name predicate applied to the base table. alias is
// "" for the single-table query or "i." in the FTS join.
func appendNameExact(query string, args []interface{}, filter *iteminfo.NameFilter, alias string) (string, []interface{}) {
if filter == nil || len(filter.Exact) == 0 {
return query, args
}
column := alias + "name"
if filter.ExactLower {
column = "lower(" + alias + "name)"
}
return appendGlob(query, args, column, filter.Exact, filter.JoinAnd)
}
// SearchItems queries the database for items matching the search criteria for a single source.
// When nameGlobPatterns is non-empty (and largest is false), rows are restricted with SQLite name GLOB ... OR ....
// Returns rows that can be iterated to scan search results.
func (db *IndexDB) SearchItems(source string, scope string, largest bool, nameGlobPatterns []string, nameGlobPatternsAnd, caseExact bool) (*sql.Rows, error) {
// When the filter is usable and the FTS index is ready, rows are narrowed with the trigram
// index and then filtered by the exact name predicate; otherwise the base table is scanned.
func (db *IndexDB) SearchItems(ctx context.Context, source, scope string, largest bool, filter *iteminfo.NameFilter) (*sql.Rows, error) {
if largest {
query := `
SELECT path, name, size, created_time, mod_time, type, is_dir, has_preview
FROM index_items
WHERE source = ?
`
args := []interface{}{source}
if scope != "" {
query += " AND path GLOB ?"
args = append(args, iteminfo.EscapeGlobLiteral(scope)+"*")
}
query += " ORDER BY size DESC"
return db.QueryContext(ctx, query, args...)
}
if useFTSPath(db, filter) {
query := `
SELECT i.path, i.name, i.size, i.created_time, i.mod_time, i.type, i.is_dir, i.has_preview
FROM index_items_fts f CROSS JOIN index_items i ON i.rowid = f.rowid
WHERE i.source = ?
`
args := []interface{}{source}
if scope != "" {
query += " AND i.path GLOB ?"
args = append(args, iteminfo.EscapeGlobLiteral(scope)+"*")
}
// Narrow by source+scope inside the global FTS index (superset; exact filters below).
query += " AND f.spath LIKE ?"
args = append(args, source+":"+scope+"%")
query, args = appendNamePrefilter(query, args, filter)
query, args = appendNameExact(query, args, filter, "i.")
return db.QueryContext(ctx, query, args...)
}
query := `
SELECT path, name, size, created_time, mod_time, type, is_dir, has_preview
FROM index_items
WHERE source = ?
`
args := []interface{}{source}
// Apply scope filter
if scope != "" {
// Use GLOB for prefix matching which is supported by SQLite and efficient
query += " AND path GLOB ?"
args = append(args, scope+"*")
args = append(args, iteminfo.EscapeGlobLiteral(scope)+"*")
}
if !largest && len(nameGlobPatterns) > 0 {
query, args = appendNameGlobSQL(query, args, nameGlobPatterns, nameGlobPatternsAnd, caseExact)
}
if largest {
query += " ORDER BY size DESC"
}
return db.Query(query, args...)
query, args = appendNameExact(query, args, filter, "")
return db.QueryContext(ctx, query, args...)
}
// SearchItemsMultiSource queries the database for items matching the search criteria across multiple sources.
// When nameGlobPatterns is non-empty (and largest is false), restricts rows with SQLite name GLOB ... OR ....
// Returns rows that can be iterated to scan search results.
func (db *IndexDB) SearchItemsMultiSource(sources []string, sourceScopes map[string]string, largest bool, nameGlobPatterns []string, nameGlobPatternsAnd, caseExact bool) (*sql.Rows, error) {
func (db *IndexDB) SearchItemsMultiSource(ctx context.Context, sources []string, sourceScopes map[string]string, largest bool, filter *iteminfo.NameFilter) (*sql.Rows, error) {
if len(sources) == 0 {
return nil, fmt.Errorf("at least one source is required")
}
args := []interface{}{}
placeholders := make([]string, len(sources))
for i, source := range sources {
placeholders[i] = "?"
args = append(args, source)
}
sourceClause := "source = ?"
if len(sources) > 1 {
sourceClause = "source IN (" + strings.Join(placeholders, ",") + ")"
}
// Per-source scope conditions (on the base table).
scopeConditions := []string{}
scopeArgs := []interface{}{}
spathConditions := []string{}
spathArgs := []interface{}{}
for _, source := range sources {
scope, hasScope := sourceScopes[source]
if hasScope && scope != "" {
scopeConditions = append(scopeConditions, "(source = ? AND path GLOB ?)")
scopeArgs = append(scopeArgs, source, iteminfo.EscapeGlobLiteral(scope)+"*")
spathConditions = append(spathConditions, "f.spath LIKE ?")
spathArgs = append(spathArgs, source+":"+scope+"%")
} else {
scopeConditions = append(scopeConditions, "source = ?")
scopeArgs = append(scopeArgs, source)
}
}
if largest {
query := `
SELECT source, path, name, size, created_time, mod_time, type, is_dir, has_preview
FROM index_items
WHERE ` + sourceClause
query += " AND (" + strings.Join(scopeConditions, " OR ") + ")"
query += " ORDER BY size DESC"
allArgs := append(args, scopeArgs...)
return db.QueryContext(ctx, query, allArgs...)
}
if useFTSPath(db, filter) {
query := `
SELECT i.source, i.path, i.name, i.size, i.created_time, i.mod_time, i.type, i.is_dir, i.has_preview
FROM index_items_fts f CROSS JOIN index_items i ON i.rowid = f.rowid
WHERE i.` + sourceClause
query += " AND (" + strings.Join(scopeConditions, " OR ") + ")"
allArgs := append(args, scopeArgs...)
if len(spathConditions) > 0 {
query += " AND (" + strings.Join(spathConditions, " OR ") + ")"
allArgs = append(allArgs, spathArgs...)
}
query, allArgs = appendNamePrefilter(query, allArgs, filter)
query, allArgs = appendNameExact(query, allArgs, filter, "i.")
return db.QueryContext(ctx, query, allArgs...)
}
query := `
SELECT source, path, name, size, created_time, mod_time, type, is_dir, has_preview
FROM index_items
`
args := []interface{}{}
whereClauses := []string{}
// Filter by sources using IN clause
if len(sources) == 1 {
whereClauses = append(whereClauses, "source = ?")
args = append(args, sources[0])
} else {
placeholders := make([]string, len(sources))
for i, source := range sources {
placeholders[i] = "?"
args = append(args, source)
}
whereClauses = append(whereClauses, "source IN ("+strings.Join(placeholders, ",")+")")
}
// Apply scope filters - need OR conditions for each source+scope combination
// Build conditions for each source: if it has a scope, filter by it; otherwise include all paths for that source
scopeConditions := []string{}
for _, source := range sources {
scope, hasScope := sourceScopes[source]
if hasScope && scope != "" {
// Source has a scope - filter by it
scopeConditions = append(scopeConditions, "(source = ? AND path GLOB ?)")
args = append(args, source, scope+"*")
} else {
// Source has no scope or empty scope - include all paths for this source
// The source IN clause already filters by source, so we just need to ensure
// this source is included in the OR condition (which it already is via source IN)
// We add a simple source check to make the OR condition work correctly
scopeConditions = append(scopeConditions, "source = ?")
args = append(args, source)
}
}
if len(scopeConditions) > 0 {
whereClauses = append(whereClauses, "("+strings.Join(scopeConditions, " OR ")+")")
}
if len(whereClauses) > 0 {
query += " WHERE " + strings.Join(whereClauses, " AND ")
}
if !largest && len(nameGlobPatterns) > 0 {
query, args = appendNameGlobSQL(query, args, nameGlobPatterns, nameGlobPatternsAnd, caseExact)
}
if largest {
query += " ORDER BY size DESC"
}
return db.Query(query, args...)
WHERE ` + sourceClause
query += " AND (" + strings.Join(scopeConditions, " OR ") + ")"
allArgs := append(args, scopeArgs...)
query, allArgs = appendNameExact(query, allArgs, filter, "")
return db.QueryContext(ctx, query, allArgs...)
}
@@ -0,0 +1,231 @@
package sql
import (
"context"
"strings"
"testing"
"time"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
)
func newSearchTestDB(t *testing.T, name string) *IndexDB {
t.Helper()
dir := t.TempDir()
pop := pushTestIndexConfig(t, dir, testIndexSQLConfig(settings.IndexStartupIntegrityOff))
t.Cleanup(pop)
db, _, err := NewIndexDB(name, "OFF", 1000, 32, false)
if err != nil {
t.Fatalf("NewIndexDB: %v", err)
}
t.Cleanup(func() { _ = db.Close() })
return db
}
func insertSearchItem(t *testing.T, db *IndexDB, source, path, name string) {
t.Helper()
info := &iteminfo.FileInfo{
Path: path,
ItemInfo: iteminfo.ItemInfo{
Name: name,
Type: "text/plain",
ModTime: time.Unix(1, 0),
},
}
if err := db.InsertItem(source, path, info); err != nil {
t.Fatalf("InsertItem(%s): %v", path, err)
}
}
func runSearchItems(t *testing.T, db *IndexDB, source, scope string, filter *iteminfo.NameFilter) []string {
t.Helper()
rows, err := db.SearchItems(context.Background(), source, scope, false, filter)
if err != nil {
t.Fatalf("SearchItems: %v", err)
}
defer rows.Close()
var got []string
for rows.Next() {
var path, name string
var size, createdTime, modTime int64
var mimeType string
var isDir, hasPreview bool
if err := rows.Scan(&path, &name, &size, &createdTime, &modTime, &mimeType, &isDir, &hasPreview); err != nil {
t.Fatalf("scan: %v", err)
}
got = append(got, path)
}
if err := rows.Err(); err != nil {
t.Fatalf("rows: %v", err)
}
return got
}
func seedSearchItems(t *testing.T, db *IndexDB) {
t.Helper()
insertSearchItem(t, db, "s1", "/a/Report 2024.pdf", "Report 2024.pdf")
insertSearchItem(t, db, "s1", "/a/my_file.txt", "my_file.txt")
insertSearchItem(t, db, "s1", "/a/myXfile.txt", "myXfile.txt")
insertSearchItem(t, db, "s1", "/a/abc.txt", "abc.txt")
insertSearchItem(t, db, "s1", "/b/report.txt", "report.txt")
insertSearchItem(t, db, "s2", "/a/report-other.txt", "report-other.txt")
}
func TestSearchItemsFTSDefault(t *testing.T) {
db := newSearchTestDB(t, "search_fts_default")
if !db.FTSReady() {
t.Fatal("expected the FTS index to be ready")
}
seedSearchItems(t, db)
filter := iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"report"}}, false, false)
if filter == nil || !filter.UseFTS {
t.Fatalf("expected an FTS filter, got %#v", filter)
}
got := runSearchItems(t, db, "s1", "/", filter)
want := []string{"/a/Report 2024.pdf", "/b/report.txt"}
if !sameSet(got, want) {
t.Fatalf("default search got %v, want %v", got, want)
}
// The LIKE prefilter treats '_' as a wildcard; the exact lower(name) GLOB must remove
// the false positive myXfile.txt.
filter = iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"my_file"}}, false, false)
got = runSearchItems(t, db, "s1", "/", filter)
want = []string{"/a/my_file.txt"}
if !sameSet(got, want) {
t.Fatalf("underscore search got %v, want %v", got, want)
}
// Scope is respected.
filter = iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"report"}}, false, false)
got = runSearchItems(t, db, "s1", "/b/", filter)
want = []string{"/b/report.txt"}
if !sameSet(got, want) {
t.Fatalf("scoped search got %v, want %v", got, want)
}
}
func TestSearchItemsCaseExactAndWildcard(t *testing.T) {
db := newSearchTestDB(t, "search_fts_modes")
seedSearchItems(t, db)
// case:exact is case-sensitive.
exact := iteminfo.NameFilterForSearch(iteminfo.SearchOptions{
Terms: []string{"Report"},
Conditions: map[string]bool{"exact": true},
}, false, false)
got := runSearchItems(t, db, "s1", "/", exact)
want := []string{"/a/Report 2024.pdf"}
if !sameSet(got, want) {
t.Fatalf("case:exact got %v, want %v", got, want)
}
// Wildcard stays on the legacy (case-insensitive) path.
wild := iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"report*"}}, true, false)
if wild == nil || wild.UseFTS {
t.Fatalf("wildcard should use the legacy path, got %#v", wild)
}
got = runSearchItems(t, db, "s1", "/", wild)
want = []string{"/a/Report 2024.pdf", "/b/report.txt"}
if !sameSet(got, want) {
t.Fatalf("wildcard got %v, want %v", got, want)
}
// Too short for trigrams => legacy path, still correct.
short := iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"ab"}}, false, false)
if short == nil || short.UseFTS {
t.Fatalf("short term should use the legacy path, got %#v", short)
}
got = runSearchItems(t, db, "s1", "/", short)
want = []string{"/a/abc.txt"}
if !sameSet(got, want) {
t.Fatalf("short search got %v, want %v", got, want)
}
}
func TestSearchItemsMultiSource(t *testing.T) {
db := newSearchTestDB(t, "search_fts_multi")
seedSearchItems(t, db)
filter := iteminfo.NameFilterForSearch(iteminfo.SearchOptions{Terms: []string{"report"}}, false, false)
rows, err := db.SearchItemsMultiSource(context.Background(),
[]string{"s1", "s2"}, map[string]string{"s1": "/", "s2": "/"}, false, filter)
if err != nil {
t.Fatalf("SearchItemsMultiSource: %v", err)
}
defer rows.Close()
var got []string
for rows.Next() {
var source, path, name string
var size, ct, mt int64
var mime string
var isDir, hasPreview bool
if err := rows.Scan(&source, &path, &name, &size, &ct, &mt, &mime, &isDir, &hasPreview); err != nil {
t.Fatalf("scan: %v", err)
}
got = append(got, source+":"+path)
}
want := []string{"s1:/a/Report 2024.pdf", "s1:/b/report.txt", "s2:/a/report-other.txt"}
if !sameSet(got, want) {
t.Fatalf("multi-source got %v, want %v", got, want)
}
}
// TestSearchItemsQueryPlan guards the planner fix: the FTS table must drive the query.
func TestSearchItemsQueryPlan(t *testing.T) {
db := newSearchTestDB(t, "search_fts_plan")
seedSearchItems(t, db)
plan, err := db.ExplainQueryPlan(
`SELECT i.path FROM index_items_fts f CROSS JOIN index_items i ON i.rowid = f.rowid
WHERE i.source = ? AND i.path GLOB ? AND f.spath LIKE ? AND f.name LIKE ? AND lower(i.name) GLOB ?`,
"s1", "/%", "s1:/%", "%report%", "*report*",
)
if err != nil {
t.Fatalf("ExplainQueryPlan: %v", err)
}
if !strings.Contains(plan, "VIRTUAL TABLE") {
t.Fatalf("expected the FTS table to drive the query, plan:\n%s", plan)
}
if !strings.Contains(plan, "SEARCH i USING INTEGER PRIMARY KEY") {
t.Fatalf("expected a rowid lookup into index_items, plan:\n%s", plan)
}
}
func TestSearchItemsFTSHealthAfterDelete(t *testing.T) {
db := newSearchTestDB(t, "search_fts_health")
seedSearchItems(t, db)
if err := db.checkFTSHealth(); err != nil {
t.Fatalf("health before delete: %v", err)
}
if err := db.DeleteItem("s1", "/a/abc.txt", false); err != nil {
t.Fatalf("DeleteItem: %v", err)
}
if err := db.checkFTSHealth(); err != nil {
t.Fatalf("health after delete: %v", err)
}
// A name-changing upsert must also keep the index in sync.
insertSearchItem(t, db, "s1", "/a/abc.txt", "renamed.txt")
if err := db.checkFTSHealth(); err != nil {
t.Fatalf("health after re-insert: %v", err)
}
}
func sameSet(got, want []string) bool {
if len(got) != len(want) {
return false
}
set := make(map[string]int, len(got))
for _, g := range got {
set[g]++
}
for _, w := range want {
set[w]--
if set[w] < 0 {
return false
}
}
return true
}
+9 -9
View File
@@ -1,6 +1,7 @@
package sql
import (
"context"
"database/sql"
"fmt"
"os"
@@ -22,14 +23,6 @@ const (
defaultCacheSizePages = -4096 // ~16MB
)
func init() {
if SqliteDriver == "sqlite3" {
logger.Debugf("CGO SQLite driver initialized")
} else {
logger.Debugf("Default SQLite driver initialized")
}
}
// TempDB manages a temporary SQLite database for operations that need
// to stream large datasets without loading everything into memory.
// This can be used by any part of the codebase that needs temporary SQLite storage.
@@ -227,7 +220,6 @@ func NewTempDB(id string, config ...*TempDBConfig) (*TempDB, error) {
}
// Open SQLite database with basic connection string.
// Driver is selected at compile time: "sqlite3" (CGO) or "sqlite" (pure Go).
// busy_timeout is set in the DSN so every pooled connection honors it.
db, err := sql.Open(SqliteDriver, sqlitebusy.WithBusyTimeout(tmpPath))
if err != nil {
@@ -405,6 +397,14 @@ func (t *TempDB) Query(query string, args ...interface{}) (*sql.Rows, error) {
return t.db.Query(query, args...)
}
// QueryContext executes a query that returns rows, honoring context cancellation so
// an abandoned request interrupts the running statement instead of holding the connection.
func (t *TempDB) QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error) {
t.mu.Lock()
defer t.mu.Unlock()
return t.db.QueryContext(ctx, query, args...)
}
// QueryRow executes a query that is expected to return at most one row.
// This is a convenience method that handles locking.
func (t *TempDB) QueryRow(query string, args ...interface{}) *sql.Row {
@@ -1,6 +1,3 @@
//go:build !cgosql
// +build !cgosql
package sqldb
import (
@@ -1,10 +0,0 @@
//go:build cgosql
// +build cgosql
package sqldb
import (
_ "github.com/mattn/go-sqlite3"
)
const SqliteDriver = "sqlite3"
+1 -1
View File
@@ -14,7 +14,7 @@ var ErrBusy = errors.New("sqlite database is busy or locked")
// WithBusyTimeout appends the busy_timeout connection parameter to dsn so that
// every connection in the pool honors the timeout. Both supported drivers
// (mattn/go-sqlite3 and modernc.org/sqlite) apply the parameter at
// (modernc.org/sqlite) applies the parameter at
// connection-open time, unlike a one-off PRAGMA which only configures the
// single connection that database/sql happens to pick.
func WithBusyTimeout(dsn string) string {
@@ -1,23 +0,0 @@
//go:build cgosql
// +build cgosql
package sqlitebusy
import (
"errors"
sqlite3 "github.com/mattn/go-sqlite3"
)
const testDriver = "sqlite3"
// driverErrorCode returns the driver's extended result code for err, so tests
// can assert which busy/locked variant was produced. mattn reports the
// extended code in Error.ExtendedCode.
func driverErrorCode(err error) int {
var sqliteErr sqlite3.Error
if errors.As(err, &sqliteErr) {
return int(sqliteErr.ExtendedCode)
}
return -1
}
@@ -1,6 +1,3 @@
//go:build !cgosql
// +build !cgosql
package sqlitebusy
import (
@@ -1,6 +1,3 @@
//go:build !cgosql
// +build !cgosql
package sqlitebusy
import (
@@ -1,21 +0,0 @@
//go:build cgosql
// +build cgosql
package sqlitebusy
import (
"errors"
sqlite3 "github.com/mattn/go-sqlite3"
)
// isBusyOrLocked reports whether err is a SQLITE_BUSY or SQLITE_LOCKED error
// from the CGO (mattn/go-sqlite3) driver. Error.Code is already the primary
// result code; extended codes are reported separately in Error.ExtendedCode.
func isBusyOrLocked(err error) bool {
var sqliteErr sqlite3.Error
if !errors.As(err, &sqliteErr) {
return false
}
return sqliteErr.Code == sqlite3.ErrBusy || sqliteErr.Code == sqlite3.ErrLocked
}
+15 -12
View File
@@ -233,18 +233,21 @@ type Preview struct {
// FrontendUser holds fields safe to return from user APIs (embedded on User).
type FrontendUser struct {
NonAdminEditable
DisableSettings bool `json:"disableSettings"`
Username string `json:"username"`
FrontendScopes []FrontendScope `json:"scopes"`
LockPassword bool `json:"lockPassword"`
RequirePasswordChange bool `json:"requirePasswordChange"`
Permissions Permissions `json:"permissions"` // global: admin, api, share, realtime
SourcePermissions map[string]SourceFilePermissions `json:"sourcePermissions,omitempty"` // deprecated: use scopes[].permissions
LoginMethod LoginMethod `json:"loginMethod"`
OtpEnabled bool `json:"otpEnabled"`
ShowFirstLogin bool `json:"showFirstLogin"`
EffectiveToolAccess map[string]bool `json:"effectiveToolAccess,omitempty"`
Perm Permissions `json:"perm,omitzero"`
DisableSettings bool `json:"disableSettings"`
// MaxConcurrentTranscodes is admin-set: it stays out of NonAdminEditable so a
// user cannot raise their own limit through a self-service PATCH.
MaxConcurrentTranscodes int `json:"maxConcurrentTranscodes"`
Username string `json:"username"`
FrontendScopes []FrontendScope `json:"scopes"`
LockPassword bool `json:"lockPassword"`
RequirePasswordChange bool `json:"requirePasswordChange"`
Permissions Permissions `json:"permissions"` // global: admin, api, share, realtime
SourcePermissions map[string]SourceFilePermissions `json:"sourcePermissions,omitempty"` // deprecated: use scopes[].permissions
LoginMethod LoginMethod `json:"loginMethod"`
OtpEnabled bool `json:"otpEnabled"`
ShowFirstLogin bool `json:"showFirstLogin"`
EffectiveToolAccess map[string]bool `json:"effectiveToolAccess,omitempty"`
Perm Permissions `json:"perm,omitzero"`
}
// PinnedItems maps source filesystem path -> index directory path -> pinned item names.
@@ -0,0 +1,38 @@
package embeddedpreview
import (
"bytes"
"image"
// Decoders for embedded images, so Usable can decode every format the preview path accepts.
_ "image/gif"
_ "image/jpeg"
_ "image/png"
_ "golang.org/x/image/webp"
)
// maxPixels caps width x height of an embedded image. The read cap bounds encoded bytes, not
// decoded size: a small file can declare huge dimensions, and decoding allocates every pixel
// (about 4 bytes each as RGBA).
const maxPixels = 40_000_000
// WithinPixelBudget reports whether an image of the given size can be decoded safely.
func WithinPixelBudget(width, height int) bool {
if width <= 0 || height <= 0 {
return false
}
return int64(width)*int64(height) <= maxPixels
}
// Usable reports whether data is an image that is safe to use as a preview. The header must
// declare a size within the pixel budget, and the whole image must decode. The budget check runs
// first, so an oversized header is rejected without allocating pixels. The full decode catches
// truncated or corrupt files, which a header check alone accepts.
func Usable(data []byte) bool {
cfg, _, err := image.DecodeConfig(bytes.NewReader(data))
if err != nil || !WithinPixelBudget(cfg.Width, cfg.Height) {
return false
}
_, _, err = image.Decode(bytes.NewReader(data))
return err == nil
}
@@ -0,0 +1,68 @@
package embeddedpreview
import (
"bytes"
"encoding/base64"
"encoding/binary"
"hash/crc32"
"image"
"image/png"
"testing"
"github.com/stretchr/testify/require"
)
// pngBytes encodes a w x h image. Keep w x h small in tests.
func pngBytes(t *testing.T, w, h int) []byte {
t.Helper()
var buf bytes.Buffer
require.NoError(t, png.Encode(&buf, image.NewGray(image.Rect(0, 0, w, h))))
return buf.Bytes()
}
// pngWithDeclaredSize rewrites the IHDR of a small PNG to declare w x h, with a valid CRC.
// The pixel data is still the small original, so the file is truncated relative to its header.
func pngWithDeclaredSize(t *testing.T, w, h uint32) []byte {
t.Helper()
data := pngBytes(t, 1, 1)
binary.BigEndian.PutUint32(data[16:20], w)
binary.BigEndian.PutUint32(data[20:24], h)
binary.BigEndian.PutUint32(data[29:33], crc32.ChecksumIEEE(data[12:29]))
return data
}
func TestWithinPixelBudget(t *testing.T) {
require.True(t, WithinPixelBudget(1, 1))
require.True(t, WithinPixelBudget(8000, 5000))
require.False(t, WithinPixelBudget(0, 100))
require.False(t, WithinPixelBudget(100, -1))
require.False(t, WithinPixelBudget(10000, 10000))
}
func TestUsable_acceptsValidImage(t *testing.T) {
require.True(t, Usable(pngBytes(t, 64, 64)))
}
func TestUsable_rejectsGarbage(t *testing.T) {
require.False(t, Usable([]byte("not an image")))
}
func TestUsable_rejectsOversizedHeader(t *testing.T) {
require.False(t, Usable(pngWithDeclaredSize(t, 10000, 10000)))
}
// The header parses and fits the budget, but the pixel data is cut short.
func TestUsable_rejectsTruncatedImage(t *testing.T) {
full := pngBytes(t, 256, 256)
truncated := full[:len(full)/2]
_, _, err := image.DecodeConfig(bytes.NewReader(truncated))
require.NoError(t, err, "header must still parse for this test to mean anything")
require.False(t, Usable(truncated))
}
// The same PNG bytes the Playwright remote-image stub serves. They must be a real image.
func TestUsable_playwrightStubPNG(t *testing.T) {
stub, err := base64.StdEncoding.DecodeString("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==")
require.NoError(t, err)
require.True(t, Usable(stub))
}
@@ -0,0 +1,118 @@
// Package embeddedpreview returns a preview image that is already stored inside a file:
// camera RAW and HEIC previews, and thumbnails or merged images in zip-based documents and
// project files. Every extractor returns ErrNotFound when a file has no usable image, so callers
// fall back to go-media, ffmpeg, and the document pipeline. Nothing is extracted to disk.
package embeddedpreview
import (
"archive/zip"
"context"
"errors"
"fmt"
"path/filepath"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/internal/imagemeta"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
)
// ErrNotFound means the file has no embedded image this package can read.
var ErrNotFound = errors.New("no embedded preview image found")
// Result is the embedded image and how the caller should treat it.
type Result struct {
Data []byte
// Ext is the lowercase extension of the embedded image, for example ".png".
Ext string
// Oriented is true when EXIF orientation applies to Data (camera RAW and HEIC).
Oriented bool
}
// Extractor reads one family of formats.
type Extractor interface {
Supports(ext, mimetype string) bool
Extract(ctx context.Context, path string) (Result, error)
}
var extractors = []Extractor{
rawExtractor{},
zipExtractor{exts: []string{".kra"}, pick: pickKrita},
zipExtractor{exts: []string{".ora"}, pick: pickORA},
zipExtractor{exts: []string{".sketch"}, pick: pickSketch},
zipExtractor{exts: []string{".mscz"}, pick: pickMuseScore},
zipExtractor{exts: []string{".odt", ".ods", ".odp", ".odg"}, pick: pickODF},
zipExtractor{exts: []string{".docx", ".xlsx", ".pptx"}, pick: pickOOXML},
zipExtractor{exts: []string{".cbz"}, pick: pickCBZ},
zipExtractor{exts: []string{".xps"}, pick: pickXPS},
}
func lookup(name, mimetype string) (Extractor, bool) {
ext := strings.ToLower(filepath.Ext(name))
for _, e := range extractors {
if e.Supports(ext, mimetype) {
return e, true
}
}
return nil, false
}
// Supports reports whether an extractor in this package handles the file.
func Supports(name, mimetype string) bool {
_, ok := lookup(name, mimetype)
return ok
}
// Extract returns the embedded image for the file at path. It returns ErrNotFound when the
// format has no extractor or the file has no usable image.
func Extract(ctx context.Context, name, mimetype, path string) (Result, error) {
e, ok := lookup(name, mimetype)
if !ok {
return Result{}, ErrNotFound
}
if err := ctx.Err(); err != nil {
return Result{}, err
}
return e.Extract(ctx, path)
}
// rawExtractor wraps imagemeta for camera RAW and HEIC files.
type rawExtractor struct{}
func (rawExtractor) Supports(ext, mimetype string) bool {
return iteminfo.IsRawImage(ext) || strings.HasPrefix(mimetype, "image/hei")
}
func (rawExtractor) Extract(ctx context.Context, path string) (Result, error) {
data, err := imagemeta.ExtractEmbeddedPreview(ctx, path)
if err != nil {
return Result{}, err
}
if len(data) == 0 {
return Result{}, ErrNotFound
}
return Result{Data: data, Ext: ".jpg", Oriented: true}, nil
}
// zipExtractor handles a zip container, using pick to choose the entry.
type zipExtractor struct {
exts []string
pick func(zr *zip.Reader) (Result, error)
}
func (z zipExtractor) Supports(ext, _ string) bool {
for _, e := range z.exts {
if e == ext {
return true
}
}
return false
}
func (z zipExtractor) Extract(_ context.Context, path string) (Result, error) {
zr, err := zip.OpenReader(path)
if err != nil {
return Result{}, fmt.Errorf("open archive: %w", err)
}
defer zr.Close()
return z.pick(&zr.Reader)
}
@@ -0,0 +1,214 @@
package embeddedpreview
import (
"archive/zip"
"bytes"
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
// writeZip builds a zip container with the given entries and returns its path.
func writeZip(t *testing.T, name string, entries map[string][]byte) string {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for entryName, data := range entries {
w, err := zw.Create(entryName)
require.NoError(t, err)
_, err = w.Write(data)
require.NoError(t, err)
}
require.NoError(t, zw.Close())
p := filepath.Join(t.TempDir(), name)
require.NoError(t, os.WriteFile(p, buf.Bytes(), 0o644))
return p
}
func extract(t *testing.T, name, path string) (Result, error) {
t.Helper()
return Extract(context.Background(), name, "application/octet-stream", path)
}
func TestKrita_mergedImageThenPreview(t *testing.T) {
merged, preview := pngBytes(t, 8, 8), pngBytes(t, 4, 4)
p := writeZip(t, "art.kra", map[string][]byte{
"mergedimage.png": merged,
"preview.png": preview,
})
res, err := extract(t, "art.kra", p)
require.NoError(t, err)
require.Equal(t, merged, res.Data)
require.Equal(t, ".png", res.Ext)
require.False(t, res.Oriented)
p = writeZip(t, "fallback.kra", map[string][]byte{"preview.png": preview})
res, err = extract(t, "fallback.kra", p)
require.NoError(t, err)
require.Equal(t, preview, res.Data)
}
// An oversized mergedimage.png must not hide a usable preview.png.
func TestKrita_oversizedMergedImageFallsBackToPreview(t *testing.T) {
preview := pngBytes(t, 4, 4)
p := writeZip(t, "big.kra", map[string][]byte{
"mergedimage.png": pngWithDeclaredSize(t, 10000, 10000),
"preview.png": preview,
})
res, err := extract(t, "big.kra", p)
require.NoError(t, err)
require.Equal(t, preview, res.Data)
}
// A truncated mergedimage.png is skipped the same way.
func TestKrita_truncatedMergedImageFallsBackToPreview(t *testing.T) {
full := pngBytes(t, 256, 256)
preview := pngBytes(t, 4, 4)
p := writeZip(t, "cut.kra", map[string][]byte{
"mergedimage.png": full[:len(full)/2],
"preview.png": preview,
})
res, err := extract(t, "cut.kra", p)
require.NoError(t, err)
require.Equal(t, preview, res.Data)
}
func TestKrita_missingImage(t *testing.T) {
p := writeZip(t, "art.kra", map[string][]byte{"maindoc.xml": []byte("<doc/>")})
_, err := extract(t, "art.kra", p)
require.ErrorIs(t, err, ErrNotFound)
}
func TestCBZ_firstPageInNaturalOrder(t *testing.T) {
p := writeZip(t, "comic.cbz", map[string][]byte{
"page10.jpg": []byte("ten"),
"page2.jpg": []byte("two"),
"__MACOSX/._page1.jpg": []byte("junk"),
".hidden.png": []byte("hidden"),
"notes.txt": []byte("text"),
})
res, err := extract(t, "comic.cbz", p)
require.NoError(t, err)
require.Equal(t, []byte("two"), res.Data)
require.Equal(t, ".jpg", res.Ext)
}
func TestCBZ_noImages(t *testing.T) {
p := writeZip(t, "comic.cbz", map[string][]byte{"readme.txt": []byte("x")})
_, err := extract(t, "comic.cbz", p)
require.ErrorIs(t, err, ErrNotFound)
}
const xpsRels = `<?xml version="1.0" encoding="utf-8"?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
<Relationship Id="rId1" Type="http://schemas.microsoft.com/xps/2005/06/thumbnail" Target="/Metadata/Thumbnail.jpeg"/>
<Relationship Id="rId2" Type="http://schemas.microsoft.com/xps/2005/06/fixedrepresentation" Target="/FixedDocSeq.fdseq"/>
</Relationships>`
func TestXPS_thumbnailRelationship(t *testing.T) {
p := writeZip(t, "doc.xps", map[string][]byte{
"_rels/.rels": []byte(xpsRels),
"Metadata/Thumbnail.jpeg": []byte("thumb"),
})
res, err := extract(t, "doc.xps", p)
require.NoError(t, err)
require.Equal(t, []byte("thumb"), res.Data)
require.Equal(t, ".jpeg", res.Ext)
}
func TestXPS_noThumbnail(t *testing.T) {
rels := `<Relationships><Relationship Type="http://schemas.microsoft.com/xps/2005/06/fixedrepresentation" Target="/FixedDocSeq.fdseq"/></Relationships>`
p := writeZip(t, "doc.xps", map[string][]byte{"_rels/.rels": []byte(rels)})
_, err := extract(t, "doc.xps", p)
require.ErrorIs(t, err, ErrNotFound)
}
func TestOpenDocument_thumbnail(t *testing.T) {
p := writeZip(t, "letter.odt", map[string][]byte{"Thumbnails/thumbnail.png": []byte("odf")})
res, err := extract(t, "letter.odt", p)
require.NoError(t, err)
require.Equal(t, []byte("odf"), res.Data)
require.Equal(t, ".png", res.Ext)
}
func TestOOXML_thumbnailVariants(t *testing.T) {
for _, entry := range []string{"docProps/thumbnail.jpeg", "docProps/thumbnail.png"} {
p := writeZip(t, "deck.pptx", map[string][]byte{entry: []byte("thumb")})
res, err := extract(t, "deck.pptx", p)
require.NoError(t, err, entry)
require.Equal(t, []byte("thumb"), res.Data)
}
p := writeZip(t, "sheet.xlsx", map[string][]byte{"xl/workbook.xml": []byte("<x/>")})
_, err := extract(t, "sheet.xlsx", p)
require.ErrorIs(t, err, ErrNotFound)
}
func TestORA_mergedImage(t *testing.T) {
p := writeZip(t, "paint.ora", map[string][]byte{"mergedimage.png": []byte("ora")})
res, err := extract(t, "paint.ora", p)
require.NoError(t, err)
require.Equal(t, []byte("ora"), res.Data)
}
func TestSketch_preview(t *testing.T) {
p := writeZip(t, "ui.sketch", map[string][]byte{"previews/preview.png": []byte("sketch")})
res, err := extract(t, "ui.sketch", p)
require.NoError(t, err)
require.Equal(t, []byte("sketch"), res.Data)
}
func TestMuseScore_thumbnail(t *testing.T) {
p := writeZip(t, "score.mscz", map[string][]byte{"Thumbnails/thumbnail.png": []byte("score")})
res, err := extract(t, "score.mscz", p)
require.NoError(t, err)
require.Equal(t, []byte("score"), res.Data)
}
func TestExtract_notAZip(t *testing.T) {
p := filepath.Join(t.TempDir(), "broken.cbz")
require.NoError(t, os.WriteFile(p, []byte("not a zip"), 0o644))
_, err := extract(t, "broken.cbz", p)
require.Error(t, err)
}
func TestExtract_unsupportedFormat(t *testing.T) {
_, err := extract(t, "notes.txt", "/nonexistent")
require.ErrorIs(t, err, ErrNotFound)
}
func TestSupports(t *testing.T) {
require.True(t, Supports("art.kra", "application/octet-stream"))
require.True(t, Supports("letter.ODT", ""))
require.True(t, Supports("photo.cr2", "image/x-canon-cr2"))
require.True(t, Supports("photo.heic", "image/heic"))
require.False(t, Supports("photo.jpg", "image/jpeg"))
require.False(t, Supports("movie.mp4", "video/mp4"))
}
func TestNaturalKey_ordersNumbers(t *testing.T) {
require.Less(t, naturalKey("page2.jpg"), naturalKey("page10.jpg"))
require.Less(t, naturalKey("page002.jpg"), naturalKey("page010.jpg"))
}
// Digit runs longer than any fixed padding width must not panic and must still sort numerically.
func TestNaturalKey_longDigitRuns(t *testing.T) {
long := strings.Repeat("9", 40)
require.NotPanics(t, func() { naturalKey("p" + long + ".jpg") })
require.Less(t, naturalKey("page10.jpg"), naturalKey("page"+long+".jpg"))
require.Less(t, naturalKey("page"+long+".jpg"), naturalKey("page1"+long+".jpg"))
}
func TestCBZ_longPageNumbersDoNotPanic(t *testing.T) {
long := strings.Repeat("1", 25)
p := writeZip(t, "comic.cbz", map[string][]byte{
"page" + long + ".jpg": []byte("long"),
"page2.jpg": []byte("two"),
})
res, err := extract(t, "comic.cbz", p)
require.NoError(t, err)
require.Equal(t, []byte("two"), res.Data)
}
+187
View File
@@ -0,0 +1,187 @@
package embeddedpreview
import (
"archive/zip"
"encoding/xml"
"fmt"
"io"
"path"
"regexp"
"sort"
"strings"
)
// maxEntryBytes caps how much of one embedded entry is read. It matches the 20 MB limit
// the text pipeline uses (adapters/fs/files), and guards against zip bombs.
const maxEntryBytes = 20 << 20
var imageExts = map[string]bool{".jpg": true, ".jpeg": true, ".png": true, ".webp": true, ".gif": true}
// pickFirstEntry returns the first named entry that exists and has data.
func pickFirstEntry(zr *zip.Reader, names ...string) (Result, error) {
return pickFirstValidEntry(zr, func([]byte) bool { return true }, names...)
}
// pickFirstValidEntry returns the first named entry that exists, has data, and passes valid.
// A candidate that fails valid is skipped, so the next name is tried.
func pickFirstValidEntry(zr *zip.Reader, valid func([]byte) bool, names ...string) (Result, error) {
for _, name := range names {
f := findEntry(zr, name)
if f == nil {
continue
}
data, err := readEntry(f)
if err != nil || len(data) == 0 || !valid(data) {
continue
}
return Result{Data: data, Ext: strings.ToLower(path.Ext(f.Name))}, nil
}
return Result{}, ErrNotFound
}
// Krita stores a flattened composite at the root. preview.png is the fallback.
// An oversized or corrupt mergedimage.png is skipped in favour of preview.png.
func pickKrita(zr *zip.Reader) (Result, error) {
return pickFirstValidEntry(zr, Usable, "mergedimage.png", "preview.png")
}
// OpenRaster stores a merged image at the root and an optional thumbnail.
func pickORA(zr *zip.Reader) (Result, error) {
return pickFirstEntry(zr, "mergedimage.png", "Thumbnails/thumbnail.png")
}
// Sketch stores a preview next to the document data.
func pickSketch(zr *zip.Reader) (Result, error) {
return pickFirstEntry(zr, "previews/preview.png")
}
// MuseScore writes a thumbnail into the score container.
func pickMuseScore(zr *zip.Reader) (Result, error) {
return pickFirstEntry(zr, "Thumbnails/thumbnail.png")
}
// OpenDocument (ODT, ODS, ODP, ODG) writes Thumbnails/thumbnail.png when the app saves one.
func pickODF(zr *zip.Reader) (Result, error) {
return pickFirstEntry(zr, "Thumbnails/thumbnail.png")
}
// Office Open XML (DOCX, XLSX, PPTX) keeps an optional thumbnail under docProps.
func pickOOXML(zr *zip.Reader) (Result, error) {
return pickFirstEntry(zr, "docProps/thumbnail.jpeg", "docProps/thumbnail.jpg", "docProps/thumbnail.png")
}
// CBZ cover: the first image in natural page order, ignoring macOS metadata and hidden files.
func pickCBZ(zr *zip.Reader) (Result, error) {
var pages []*zip.File
for _, f := range zr.File {
if f.FileInfo().IsDir() || strings.HasPrefix(f.Name, "__MACOSX/") {
continue
}
base := path.Base(f.Name)
if strings.HasPrefix(base, ".") {
continue
}
if imageExts[strings.ToLower(path.Ext(base))] {
pages = append(pages, f)
}
}
sort.SliceStable(pages, func(i, j int) bool {
return naturalKey(pages[i].Name) < naturalKey(pages[j].Name)
})
for _, f := range pages {
data, err := readEntry(f)
if err != nil || len(data) == 0 {
continue
}
return Result{Data: data, Ext: strings.ToLower(path.Ext(f.Name))}, nil
}
return Result{}, ErrNotFound
}
const xpsThumbnailRel = "http://schemas.microsoft.com/xps/2005/06/thumbnail"
type xpsRelationships struct {
Relationships []struct {
Type string `xml:"Type,attr"`
Target string `xml:"Target,attr"`
} `xml:"Relationship"`
}
// XPS thumbnails are optional. The package-level relationship in _rels/.rels points at one when present.
func pickXPS(zr *zip.Reader) (Result, error) {
rels := findEntry(zr, "_rels/.rels")
if rels == nil {
return Result{}, ErrNotFound
}
relData, err := readEntry(rels)
if err != nil {
return Result{}, ErrNotFound
}
var parsed xpsRelationships
if err := xml.Unmarshal(relData, &parsed); err != nil {
return Result{}, ErrNotFound
}
for _, rel := range parsed.Relationships {
if rel.Type != xpsThumbnailRel {
continue
}
target := strings.TrimPrefix(rel.Target, "/")
ext := strings.ToLower(path.Ext(target))
if ext != ".jpg" && ext != ".jpeg" && ext != ".png" {
continue
}
return pickFirstEntry(zr, target)
}
return Result{}, ErrNotFound
}
// findEntry matches the exact name first, then case-insensitively (some writers vary the case).
func findEntry(zr *zip.Reader, name string) *zip.File {
for _, f := range zr.File {
if f.Name == name {
return f
}
}
for _, f := range zr.File {
if strings.EqualFold(f.Name, name) {
return f
}
}
return nil
}
// readEntry reads one zip entry, refusing anything larger than maxEntryBytes.
func readEntry(f *zip.File) ([]byte, error) {
if f.UncompressedSize64 > maxEntryBytes {
return nil, fmt.Errorf("embedded entry %q is too large", f.Name)
}
rc, err := f.Open()
if err != nil {
return nil, err
}
defer rc.Close()
data, err := io.ReadAll(io.LimitReader(rc, maxEntryBytes+1))
if err != nil {
return nil, err
}
if len(data) > maxEntryBytes {
return nil, fmt.Errorf("embedded entry %q is too large", f.Name)
}
return data, nil
}
var digitRun = regexp.MustCompile(`\d+`)
// naturalKey makes "page2" sort before "page10" by rewriting each digit run as a
// fixed-width length prefix followed by the run without leading zeros. Comparing those
// keys orders runs numerically, and it works for runs of any length (no padding that
// could go negative on long digit runs).
func naturalKey(name string) string {
return digitRun.ReplaceAllStringFunc(strings.ToLower(name), func(d string) string {
digits := strings.TrimLeft(d, "0")
if digits == "" {
digits = "0"
}
return fmt.Sprintf("%06d%s", len(digits), digits)
})
}
@@ -0,0 +1,94 @@
package ffmpeg
import (
"strings"
"testing"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
"github.com/gtsteffaniak/go-ffmpeg/platform"
)
func TestAccelDeviceOptionsOmitsAuto(t *testing.T) {
s := &Service{}
opts := s.AccelDeviceOptions()
for _, opt := range opts {
if opt.ID == "auto" {
t.Fatalf("options must not include auto: %+v", opts)
}
}
if len(opts) != 1 || opts[0].ID != "software" {
t.Fatalf("without capabilities want software only, got %+v", opts)
}
}
func TestResolveAccelDeviceWithoutCapabilities(t *testing.T) {
s := &Service{}
if got := s.ResolveAccelDevice("auto"); got != "software" {
t.Fatalf("ResolveAccelDevice(auto) = %q, want software", got)
}
if got := s.ResolveAccelDevice("/dev/dri/renderD128"); got != "/dev/dri/renderD128" {
t.Fatalf("ResolveAccelDevice(explicit) = %q", got)
}
}
func TestResolveAccelDeviceWSLNVIDIA(t *testing.T) {
caps := capabilities.NewCapabilities()
caps.Platform = capabilities.PlatformInfo{OS: "linux", WSL: true, NVIDIA: true}
caps.Platform.Details = map[string]string{"gpu": "NVIDIA GeForce RTX 3050"}
// Inject caps through a minimal service wrapper is not available; test helper directly.
id, label := wslNVIDIAAccelDefault(caps)
if id != "dgpu" || label != "Dedicated GPU (NVIDIA GeForce RTX 3050)" {
t.Fatalf("wsl default = %q / %q", id, label)
}
}
func TestAccelDeviceFallbackReason(t *testing.T) {
// A blank or auto/software value is not a device selection.
for _, dev := range []string{"", "auto", "software", "cpu", "none", "AUTO"} {
if _, ok := accelDeviceFallbackReason(dev); ok {
t.Fatalf("%q must not report a fallback", dev)
}
}
// A GPU name that is not on this host must be reported as falling back.
// The synthetic device will not match host enumeration in CI.
if reason, ok := accelDeviceFallbackReason("Definitely Not A Real GPU 9999"); !ok {
t.Fatal("an unmatched device name must report a fallback reason")
} else if reason == "" {
t.Fatal("the fallback reason must be non-empty")
}
}
func wslCaps() *capabilities.Capabilities {
caps := capabilities.NewCapabilities()
caps.Platform = capabilities.PlatformInfo{OS: "linux", WSL: true, NVIDIA: true}
caps.Platform.Details = map[string]string{"gpu": "NVIDIA GeForce RTX 3050"}
return caps
}
// WSL enumerates the NVIDIA GPU without a render node; the device menu must
// list it once, whether it is resolved through enumeration or the WSL default.
func TestAccelDeviceOptionsWSLListsNVIDIAOnce(t *testing.T) {
gpus := []platform.GPUDevice{{Name: "NVIDIA GeForce RTX 3050", Vendor: "nvidia", DeviceClass: "dgpu"}}
label, id, opts := resolvedAccelDeviceOptions(wslCaps(), gpus)
if id == "" || label == "" {
t.Fatalf("expected a resolved default, got %q/%q", id, label)
}
for _, o := range opts {
if o.ID == id || strings.Contains(o.Label, "RTX 3050") {
t.Fatalf("device listed twice: default=%q opts=%+v", id, opts)
}
}
}
func TestAccelDeviceOptionsKeepsDistinctGPUs(t *testing.T) {
gpus := []platform.GPUDevice{
{Name: "AMD Radeon Graphics", Vendor: "amd", DeviceClass: "igpu", Integrated: true, RenderNode: "/dev/dri/renderD128"},
{Name: "NVIDIA GeForce RTX 3050", Vendor: "nvidia", DeviceClass: "dgpu", RenderNode: "/dev/dri/renderD129"},
}
caps := capabilities.NewCapabilities()
caps.Platform.Details = map[string]string{"render_device": "/dev/dri/renderD128"}
_, id, opts := resolvedAccelDeviceOptions(caps, gpus)
if id != "/dev/dri/renderD128" || len(opts) != 1 || opts[0].ID != "/dev/dri/renderD129" {
t.Fatalf("id=%q opts=%+v", id, opts)
}
}
@@ -0,0 +1,39 @@
package ffmpeg
import (
"strings"
"testing"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
"github.com/gtsteffaniak/go-ffmpeg/platform"
)
func TestEncoderBackendLabelDeviceAndEncoder(t *testing.T) {
caps := capabilities.NewCapabilities()
caps.SelectedGPU = &platform.GPUChoice{
Enabled: true,
Name: "NVIDIA RTX",
RenderDevice: "/dev/dri/renderD128",
}
if caps.Platform.Details == nil {
caps.Platform.Details = map[string]string{}
}
caps.Platform.Details["render_device"] = "/dev/dri/renderD128"
label := encoderBackendLabel(caps, capabilities.EncodeOption{
Accel: capabilities.AccelNVENC,
Encoder: "h264_nvenc",
})
if !strings.Contains(label, "/dev/dri/renderD128") || !strings.Contains(label, "h264_nvenc") {
t.Fatalf("label = %q, want device - encoder", label)
}
}
func TestEncoderBackendOptionIDUsesEncoderNameForHardware(t *testing.T) {
id := encoderBackendOptionID(capabilities.EncodeOption{
Accel: capabilities.AccelVAAPI,
Encoder: "h264_vaapi",
})
if id != "h264_vaapi" {
t.Fatalf("id = %q, want h264_vaapi", id)
}
}
+113
View File
@@ -0,0 +1,113 @@
package ffmpeg
import (
"context"
"fmt"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-ffmpeg/encode"
"github.com/gtsteffaniak/go-ffmpeg/ops"
)
// StreamInfo is re-exported for transcoding callers.
type StreamInfo = goffmpeg.StreamInfo
// HLSPipelineOptions configures remux/copy/transcode path selection.
type HLSPipelineOptions = goffmpeg.HLSPipelineOptions
// HLSSegmentParams holds resolved encode/remux settings for one HLS session.
type HLSSegmentParams = goffmpeg.HLSSegmentParams
// HLSSegmentBuildInput describes remux/copy/transcode path selection for one file.
type HLSSegmentBuildInput = goffmpeg.HLSSegmentBuildInput
// HLSCacheIdentity describes inputs for a stable on-demand HLS disk cache key.
type HLSCacheIdentity = goffmpeg.HLSCacheIdentity
// VideoDecodeProfile selects input decode settings.
type VideoDecodeProfile = encode.VideoDecodeProfile
// VideoProfile holds encode settings.
type VideoProfile = encode.VideoProfile
// ProbeMediaStream probes a local media file for transcoding.
func (s *Service) ProbeMediaStream(ctx context.Context, path string) (StreamInfo, error) {
if s == nil || s.inner == nil {
return StreamInfo{}, errUnavailable()
}
return s.inner.ProbeFile(ctx, path)
}
// BuildHLSSegmentBuildInput derives remux/copy/transcode flags.
func BuildHLSSegmentBuildInput(info StreamInfo, opts HLSPipelineOptions) HLSSegmentBuildInput {
return goffmpeg.BuildHLSSegmentBuildInput(info, opts)
}
// BuildHLSSegmentParams resolves GOP from fps when probeFPS is true.
func (s *Service) BuildHLSSegmentParams(ctx context.Context, path string, in HLSSegmentBuildInput, probeFPS bool) (HLSSegmentParams, error) {
if s == nil || s.inner == nil {
return HLSSegmentParams{}, errUnavailable()
}
defaults := goffmpeg.DefaultOnDemandHLSDefaults()
return s.inner.BuildHLSSegmentParams(ctx, path, in, defaults, probeFPS)
}
// HLSCacheFingerprint returns a stable directory name for a transcode cache entry.
func HLSCacheFingerprint(id HLSCacheIdentity) string {
return goffmpeg.HLSCacheFingerprint(id)
}
// DefaultOnDemandHLSDefaults returns on-demand segment defaults.
func DefaultOnDemandHLSDefaults() goffmpeg.OnDemandHLSDefaults {
return goffmpeg.DefaultOnDemandHLSDefaults()
}
// PrepareHLSMediaTimeline normalizes keyframes and duration for HLS segment
// planning: container start_time is subtracted and corrupt keyframe indexes
// are filtered out.
func PrepareHLSMediaTimeline(keyframes []float64, durationSec, startTimeSec float64) (sanitizedKeyframes []float64, mediaDurationSec float64) {
return goffmpeg.PrepareHLSMediaTimeline(keyframes, durationSec, startTimeSec)
}
// KeyframeSeekBefore returns the largest keyframe time <= sec, or 0 when none.
func KeyframeSeekBefore(keyframes []float64, sec float64) float64 {
return goffmpeg.KeyframeSeekBefore(keyframes, sec)
}
// HLSDecodeProfileForOnDemand selects input decode for short on-demand HLS segments.
func HLSDecodeProfileForOnDemand(info StreamInfo) VideoDecodeProfile {
return goffmpeg.HLSDecodeProfileForOnDemand(info)
}
// DefaultHLSVideoProfile returns safe H.264 transcode defaults.
func DefaultHLSVideoProfile(maxHeight int) VideoProfile {
return goffmpeg.DefaultHLSVideoProfile(maxHeight)
}
// ProbeVideoKeyframeTimes returns keyframe presentation times in seconds.
func (s *Service) ProbeVideoKeyframeTimes(ctx context.Context, path string) ([]float64, error) {
if s == nil || s.inner == nil {
return nil, errUnavailable()
}
return s.inner.ProbeVideoKeyframeTimes(ctx, path)
}
// ProbeVideoKeyframeTimesBounded returns keyframe times from the first windowSec
// of the file only. Used as a cheap pre-flight to prove stream copy impossible
// before paying for the full-file keyframe demux.
func (s *Service) ProbeVideoKeyframeTimesBounded(ctx context.Context, path string, windowSec float64) ([]float64, error) {
if s == nil || s.inner == nil {
return nil, errUnavailable()
}
return s.inner.ProbeVideoKeyframeTimesBounded(ctx, path, windowSec)
}
func errUnavailable() error {
return fmt.Errorf("ffmpeg service not available")
}
// InputSource describes ffmpeg input.
type InputSource = ops.InputSource
// StreamFile is the file stream type constant.
const StreamFile = goffmpeg.StreamFile
+128
View File
@@ -0,0 +1,128 @@
package ffmpeg
import (
"context"
"os"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-ffmpeg/ops"
)
// VODSegmentList is the segment timeline for Model A VOD playlists.
type VODSegmentList = ops.VODSegmentList
// VODPlaylistOptions configures URIs in BuildVODPlaylist.
type VODPlaylistOptions = ops.VODPlaylistOptions
// HLSVODJobOptions configures StartHLSVODSegmentJob.
type HLSVODJobOptions = ops.HLSVODJobOptions
// HLSVODSegmentJob runs a forward ffmpeg HLS encode for on-demand VOD.
type HLSVODSegmentJob = ops.HLSVODSegmentJob
// NOTE: the transcoding manager runs its own wait/restart loop; it only uses
// the session's AcquireServePath/ReleaseServePath leases and job installation.
//
// HLSVODSession coordinates one cache directory / transcode profile
// (Jellyfin DynamicHls-style): job tracking, wait/restart decisions, partial
// segment cleanup, and per-path read leases so in-flight readers survive
// segment rewrites.
type HLSVODSession = ops.HLSVODSession
// HLSVODSessionOptions configures NewHLSVODSession.
type HLSVODSessionOptions = ops.HLSVODSessionOptions
// HLSVODDecisionConfig tunes Jellyfin-style wait/restart thresholds.
type HLSVODDecisionConfig = ops.HLSVODDecisionConfig
// HLSVODJobView is a snapshot of the active encode job for segment decisions.
type HLSVODJobView = ops.HLSVODJobView
// HLSVODSegmentAction is the host/library decision for an on-demand segment request.
type HLSVODSegmentAction = ops.HLSVODSegmentAction
const (
// HLSVODActionServe serves the segment immediately.
HLSVODActionServe = ops.HLSVODActionServe
// HLSVODActionWait waits for production to reach the segment.
HLSVODActionWait = ops.HLSVODActionWait
// HLSVODActionRestart repositions the encode job to the segment.
HLSVODActionRestart = ops.HLSVODActionRestart
)
// DefaultHLSVODForwardGapSec is the forward wait window before restarting a
// job (Jellyfin DynamicHls default: 24s of media).
const DefaultHLSVODForwardGapSec = ops.DefaultHLSVODForwardGapSec
// NewHLSVODSession returns a session coordinating VOD output under outDir.
func NewHLSVODSession(opts HLSVODSessionOptions) *HLSVODSession {
return ops.NewHLSVODSession(opts)
}
// DecideHLSVODSegmentAction implements the Model A serve/wait/restart rules.
func DecideHLSVODSegmentAction(cfg HLSVODDecisionConfig, job HLSVODJobView, k int, ready bool) HLSVODSegmentAction {
return ops.DecideHLSVODSegmentAction(cfg, job, k, ready)
}
// HLSVODMaxWaitSegmentGap returns how many segment indices ahead of EncodeHead
// may wait instead of restarting (Jellyfin: 24s of media by default).
func HLSVODMaxWaitSegmentGap(segmentSec, gapMediaSec float64) int {
return ops.HLSVODMaxWaitSegmentGap(segmentSec, gapMediaSec)
}
// BuildVODSegmentList plans segment boundaries (fixed grid or keyframe snap).
func BuildVODSegmentList(mediaDurationSec, segmentSec, startTimeSec float64, keyframes []float64) VODSegmentList {
return goffmpeg.BuildVODSegmentList(mediaDurationSec, segmentSec, startTimeSec, keyframes)
}
// SegmentIndexForTime maps playlist seconds to segment index and start time.
func SegmentIndexForTime(timeSec float64, starts []float64) (index int, startSec float64) {
return goffmpeg.SegmentIndexForTime(timeSec, starts)
}
// DefaultVODPlaylistOptions returns standard init/segment URIs.
func DefaultVODPlaylistOptions() VODPlaylistOptions {
return goffmpeg.DefaultVODPlaylistOptions()
}
// BuildVODPlaylist renders a synthetic #EXT-X-PLAYLIST-TYPE:VOD media playlist.
func BuildVODPlaylist(list VODSegmentList, opts VODPlaylistOptions) string {
return goffmpeg.BuildVODPlaylist(list, opts)
}
// HLSVODSegmentPath returns the absolute path to a segment under a job directory.
func HLSVODSegmentPath(jobDir string, index int) string {
return ops.HLSVODSegmentPath(jobDir, index)
}
// HLSVODSegmentReady reports whether a segment file is a complete fMP4 fragment.
func HLSVODSegmentReady(path string) (bool, error) {
return ops.HLSVODSegmentReady(path)
}
// HLSVODOpenReadySegment opens path only when it is a publishable fMP4 media
// fragment. The caller must close the returned file.
func HLSVODOpenReadySegment(path string) (*os.File, bool, error) {
return ops.HLSVODOpenReadySegment(path)
}
// HLSVODExplicitCopySegmentReady reports readiness for explicit-cut (-f segment)
// copy jobs: segment k is complete only when k+1 is listed in the live CSV, or
// the encode job has ended.
func HLSVODExplicitCopySegmentReady(outDir string, index int, jobRunning bool) (bool, error) {
return ops.HLSVODExplicitCopySegmentReady(outDir, index, jobRunning)
}
// HLSVODOpenReadyExplicitCopySegment opens a publishable explicit-cut copy
// segment. The caller must close the returned file.
func HLSVODOpenReadyExplicitCopySegment(outDir string, index int, jobRunning bool) (*os.File, bool, error) {
return ops.HLSVODOpenReadyExplicitCopySegment(outDir, index, jobRunning)
}
// StartHLSVODSegmentJob launches ffmpeg writing numbered fMP4 segments (Model A).
func (s *Service) StartHLSVODSegmentJob(ctx context.Context, opts HLSVODJobOptions) (*HLSVODSegmentJob, error) {
if s == nil || s.inner == nil {
return nil, errUnavailable()
}
return s.inner.StartHLSVODSegmentJob(ctx, opts)
}
+11 -19
View File
@@ -4,7 +4,6 @@ import (
"context"
"fmt"
"os"
"strconv"
"strings"
"github.com/gtsteffaniak/filebrowser/backend/internal/adapters/fs/fileutils"
@@ -75,21 +74,25 @@ func (s *Service) ConvertHEICToJPEG(ctx context.Context, heicPath string, target
return nil, ctx.Err()
}
orientation, err := s.GetHEICOrientation(heicPath)
if err != nil {
orientation = "Horizontal (normal)"
}
jpegBytes, err := s.runHEICConversion(ctx, heicPath, targetWidth, targetHeight, quality, "")
if err != nil {
return nil, fmt.Errorf("ffmpeg image conversion failed: %w", err)
}
return s.CorrectHEICJPEGOrientation(ctx, heicPath, jpegBytes), nil
}
// CorrectHEICJPEGOrientation applies irot / display-matrix correction after a full-resolution
// HEIC decode when pixels are still in coded orientation (go-media and ffmpeg filter-free paths).
func (s *Service) CorrectHEICJPEGOrientation(ctx context.Context, heicPath string, jpegBytes []byte) []byte {
orientation, err := s.GetHEICOrientation(heicPath)
if err != nil {
orientation = "Horizontal (normal)"
}
displayRotation, displayKnown := s.GetHEICDisplayMatrixRotation(ctx, heicPath)
if orientationNeedsPostCorrection(orientation, displayRotation, displayKnown) {
jpegBytes = applyOrientationToJPEG(jpegBytes, orientation)
return applyOrientationToJPEG(jpegBytes, orientation)
}
return jpegBytes, nil
return jpegBytes
}
// ConvertImageToJPEG converts any supported image to JPEG using ffmpeg.
@@ -156,14 +159,3 @@ func (s *Service) runHEICConversion(ctx context.Context, inputPath string, targe
}
return os.ReadFile(outputFile)
}
func parseQuality(quality string) int {
n, err := strconv.Atoi(quality)
if err != nil || n < 1 {
return 5
}
if n > 10 {
return 10
}
return n
}
+3 -2
View File
@@ -161,10 +161,11 @@ func BenchmarkGetOrientationFilter(b *testing.B) {
"Unknown Orientation",
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
i := 0
for b.Loop() {
orientation := orientations[i%len(orientations)]
_ = service.GetOrientationFilter(orientation)
i++
}
}
+4 -5
View File
@@ -21,21 +21,20 @@ var blockedMediaPathPrefixes = []string{
// resolveLocalMediaPath restricts ffprobe to absolute local files (no remote protocols).
func resolveLocalMediaPath(mediaPath string) (string, os.FileInfo, error) {
trimmed := strings.TrimSpace(mediaPath)
if trimmed == "" {
if strings.TrimSpace(mediaPath) == "" {
return "", nil, fmt.Errorf("media path is empty")
}
if strings.Contains(trimmed, "://") {
if strings.Contains(mediaPath, "://") {
return "", nil, fmt.Errorf("remote media URLs are not allowed")
}
lower := strings.ToLower(trimmed)
lower := strings.ToLower(strings.TrimSpace(mediaPath))
for _, prefix := range blockedMediaPathPrefixes {
if strings.HasPrefix(lower, prefix) {
return "", nil, fmt.Errorf("remote media protocols are not allowed")
}
}
clean := filepath.Clean(trimmed)
clean := filepath.Clean(mediaPath)
if !filepath.IsAbs(clean) {
return "", nil, fmt.Errorf("media path must be absolute")
}
+10
View File
@@ -29,6 +29,16 @@ func TestResolveLocalMediaPath(t *testing.T) {
{name: "relative path", path: "video.mp4", wantErr: true},
}
trailingSpacePath := filepath.Join(filepath.Dir(tmpPath), "probe-trailing-space ")
if err := os.WriteFile(trailingSpacePath, []byte("x"), 0o644); err != nil {
t.Fatalf("WriteFile(trailing space) error = %v", err)
}
tests = append(tests, struct {
name string
path string
wantErr bool
}{name: "trailing space in filename", path: trailingSpacePath})
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
+65
View File
@@ -0,0 +1,65 @@
package ffmpeg
import (
"strconv"
"github.com/gtsteffaniak/go-ffmpeg/encode"
)
const (
// defaultLegacyMJPEGQScale is the historical 1–10 -q:v value used when no quality is given.
defaultLegacyMJPEGQScale = 5
)
// frameQualityForLegacyMJPEGQScale maps the pre–go-ffmpeg v1.5.4 scale where quality 1–10 was
// passed directly to ffmpeg -q:v (lower is better) onto the 1–100 frame quality scale.
func frameQualityForLegacyMJPEGQScale(legacy int) int {
if legacy < 1 {
legacy = 1
}
if legacy > 10 {
legacy = 10
}
if legacy == 1 {
// -q:v 1 is not reachable on the new scale; quality 100 is the best MJPEG output.
return 100
}
target := strconv.Itoa(legacy)
var matches []int
for q := 1; q <= 100; q++ {
if encode.MJPEGQualityQScale(q) == target {
matches = append(matches, q)
}
}
if len(matches) == 0 {
return encode.DefaultFrameQuality
}
idx := (legacy - 1) * (len(matches) - 1) / 9
return matches[idx]
}
// parseQuality interprets a quality string for go-ffmpeg ConvertHEIC / MJPEG encode.
// Values 1–10 are treated as the legacy -q:v scale (only used internally by preview today).
// Values 11–100 use the new frame quality scale. Invalid or empty input keeps the historical default.
// PreviewQualityFromLegacyQScale maps preview's legacy "1"–"10" quality strings to
// frame quality 1–100 (same mapping as ConvertHEICToJPEG / MJPEG encode).
func PreviewQualityFromLegacyQScale(quality string) int {
return parseQuality(quality)
}
func parseQuality(quality string) int {
if quality == "" {
return frameQualityForLegacyMJPEGQScale(defaultLegacyMJPEGQScale)
}
n, err := strconv.Atoi(quality)
if err != nil || n < 1 {
return frameQualityForLegacyMJPEGQScale(defaultLegacyMJPEGQScale)
}
if n <= 10 {
return frameQualityForLegacyMJPEGQScale(n)
}
if n > 100 {
return 100
}
return n
}
+38
View File
@@ -0,0 +1,38 @@
package ffmpeg
import (
"strconv"
"testing"
"github.com/gtsteffaniak/go-ffmpeg/encode"
"github.com/stretchr/testify/require"
)
func TestParseQuality_DefaultAndNewScale(t *testing.T) {
t.Parallel()
require.Equal(t, frameQualityForLegacyMJPEGQScale(5), parseQuality(""))
require.Equal(t, frameQualityForLegacyMJPEGQScale(5), parseQuality("not-a-number"))
require.Equal(t, 85, parseQuality("85"))
require.Equal(t, 100, parseQuality("150"))
}
func TestParseQuality_LegacyQScaleMapsToSameMJPEGQScale(t *testing.T) {
t.Parallel()
for legacy := 2; legacy <= 10; legacy++ {
got := parseQuality(itoa(legacy))
require.Equal(t, encode.MJPEGQualityQScale(got), itoa(legacy),
"legacy q:v %d", legacy)
}
require.Equal(t, 100, parseQuality("1"))
require.Equal(t, "2", encode.MJPEGQualityQScale(parseQuality("1")))
}
func TestFrameQualityForLegacyMJPEGQScale_KnownPoints(t *testing.T) {
t.Parallel()
require.Equal(t, 89, frameQualityForLegacyMJPEGQScale(5))
require.Equal(t, 74, frameQualityForLegacyMJPEGQScale(10))
}
func itoa(n int) string {
return strconv.Itoa(n)
}
+473 -42
View File
@@ -6,10 +6,13 @@ import (
"io"
"os"
"strings"
"sync"
"sync/atomic"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
"github.com/gtsteffaniak/go-ffmpeg/ops"
"github.com/gtsteffaniak/go-ffmpeg/platform"
"github.com/gtsteffaniak/go-logger/logger"
)
@@ -17,12 +20,30 @@ import (
type Service struct {
inner *goffmpeg.Service
cacheDir string
// gpus caches the host GPU enumeration, which shells out on some platforms
// (PowerShell on Windows) and so must not run per settings request.
gpusOnce sync.Once
gpus []platform.GPUDevice
}
// gpuList returns the cached host GPU enumeration. A Service is replaced on
// re-initialization, so the cache is refreshed whenever ffmpeg is re-detected.
func (s *Service) gpuList() []platform.GPUDevice {
if s == nil {
return nil
}
s.gpusOnce.Do(func() { s.gpus, _ = platform.ListGPUs() })
return s.gpus
}
// FFmpegService is kept for existing callers.
type FFmpegService = Service
var global *Service
// global holds the process-wide ffmpeg service. It is swapped wholesale by
// ApplyMediaSettings, so it is stored atomically to keep concurrent readers
// (transcode admission, previews, media handlers) race-free.
var global atomic.Pointer[Service]
// InitOptions configures startup initialization.
type InitOptions struct {
@@ -30,25 +51,50 @@ type InitOptions struct {
MaxConcurrent int // deprecated: use Concurrency
Concurrency goffmpeg.Concurrency
CacheDir string
SkipHWTests bool
HardwareAcceleration bool
// AccelDevice scopes hardware acceleration to one device: "auto" (default),
// "software"/"cpu", "igpu", "dgpu", a DRM render node, or a GPU name
// substring. Parsed by go-ffmpeg; an unavailable device falls back to auto.
AccelDevice string
// VerboseFFmpeg streams ffmpeg process output to stderr and raises the ffmpeg
// log level. Wired to the media debug setting.
VerboseFFmpeg bool
// SkipHWTests skips the hardware encoder probe. Only set this when hardware
// acceleration is pinned off in the config file, otherwise the admin cannot
// enable it without a restart.
SkipHWTests bool
// EncoderHierarchy restricts which acceleration backends are preferred.
// Pass []capabilities.AccelType{capabilities.AccelNone} to keep detection
// (so hardware availability stays known) while preferring software.
EncoderHierarchy []capabilities.AccelType
}
// Initialize creates the global ffmpeg service and runs capability detection.
// Hardware encoder smoke tests run so hardware acceleration can be enabled or
// disabled at runtime without a restart, unless the config file pins it off.
func Initialize(ctx context.Context, opts InitOptions) error {
if opts.CacheDir == "" {
opts.CacheDir = os.TempDir()
}
if opts.HardwareAcceleration {
logger.Info("Detecting ffmpeg hardware codec support...")
if opts.SkipHWTests {
logger.Info("Skipping ffmpeg hardware codec detection (hardware acceleration disabled in config)")
} else {
logger.Info("Detecting ffmpeg codec support...")
}
cfg := goffmpeg.Config{
FFmpegPath: opts.FFmpegPath,
Concurrency: opts.Concurrency,
Logger: goffmpeg.NopLogger(),
SkipHWTests: opts.SkipHWTests,
FFmpegPath: opts.FFmpegPath,
Concurrency: opts.Concurrency,
Logger: logger.GetGlobalLogger(),
VerboseFFmpeg: opts.VerboseFFmpeg,
SkipHWTests: opts.SkipHWTests,
EncoderHierarchy: opts.EncoderHierarchy,
}
if sel, selErr := goffmpeg.ParseGPUSelector(opts.AccelDevice); selErr == nil {
// GPUStrictness stays at the zero value: a selected device that is not
// present falls back to auto instead of failing ffmpeg init.
cfg.GPUSelector = sel
}
if opts.Concurrency.MaxProbe == 0 && opts.Concurrency.MaxDecode == 0 && opts.Concurrency.MaxEncode == 0 {
if opts.MaxConcurrent < 1 {
@@ -59,27 +105,307 @@ func Initialize(ctx context.Context, opts InitOptions) error {
svc, err := goffmpeg.New(ctx, cfg)
if err != nil {
global = nil
global.Store(nil)
return err
}
global = &Service{
global.Store(&Service{
inner: svc,
cacheDir: opts.CacheDir,
}
})
logCapabilities(svc, opts.HardwareAcceleration)
logCapabilities(svc, opts.HardwareAcceleration, opts.AccelDevice)
return nil
}
// HardwareAccelerationAvailable reports whether any hardware encoder is usable.
func HardwareAccelerationAvailable() bool {
return global.Load().HardwareAccelerationAvailable()
}
// HardwareAccelerationAvailable reports whether any hardware encoder is usable.
func (s *Service) HardwareAccelerationAvailable() bool {
if s == nil || s.inner == nil {
return false
}
caps := s.inner.Capabilities()
if caps == nil {
return false
}
for _, enc := range caps.Encoders {
if enc.Available && enc.Kind != "" && enc.Kind != "software" {
return true
}
}
return false
}
// Get returns the initialized service, or nil when ffmpeg is unavailable.
func Get() *Service {
return global
return global.Load()
}
// ResolveVideoEncoder returns the encoder selection for an encode profile so
// callers can surface which encoder/accel a transcode will use.
func (s *Service) ResolveVideoEncoder(profile goffmpeg.VideoProfile) (capabilities.EncoderSelection, error) {
if s == nil || s.inner == nil {
return capabilities.EncoderSelection{}, fmt.Errorf("ffmpeg service unavailable")
}
return s.inner.ResolveVideoEncoder(profile)
}
// Capabilities exposes the detected ffmpeg capability matrix, or nil when the
// service is unavailable.
func (s *Service) Capabilities() *capabilities.Capabilities {
if s == nil || s.inner == nil {
return nil
}
return s.inner.Capabilities()
}
// AccelDeviceOption is one selectable entry in the acceleration device list.
type AccelDeviceOption struct {
ID string `json:"id"`
Label string `json:"label"`
}
// AccelDeviceOptions lists concrete acceleration devices for the admin UI.
// The default auto-selected GPU (when stored as "auto") is included by name;
// there is no separate "auto" entry.
func (s *Service) AccelDeviceOptions() []AccelDeviceOption {
out := []AccelDeviceOption{}
if caps := s.Capabilities(); caps != nil {
defaultLabel, defaultID, gpuOpts := resolvedAccelDeviceOptions(caps, s.gpuList())
if defaultID != "" {
out = append(out, AccelDeviceOption{ID: defaultID, Label: defaultLabel})
}
out = append(out, gpuOpts...)
}
out = append(out, AccelDeviceOption{ID: "software", Label: "CPU (software)"})
return out
}
// ResolveAccelDevice maps stored "auto"/empty to the concrete device the UI
// should display; explicit software aliases and device IDs are unchanged.
func (s *Service) ResolveAccelDevice(device string) string {
dev := strings.TrimSpace(device)
switch strings.ToLower(dev) {
case "software", "cpu", "none":
return "software"
}
if dev != "" && !strings.EqualFold(dev, "auto") {
return dev
}
caps := s.Capabilities()
if caps == nil {
return "software"
}
defaultLabel, defaultID, gpuOpts := resolvedAccelDeviceOptions(caps, s.gpuList())
if defaultID != "" {
return defaultID
}
if len(gpuOpts) == 1 {
return gpuOpts[0].ID
}
for _, opt := range gpuOpts {
if opt.Label == defaultLabel {
return opt.ID
}
}
if id, _ := wslNVIDIAAccelDefault(caps); id != "" {
return id
}
return "software"
}
// resolvedAccelDeviceOptions labels the default auto-selected GPU and lists
// every other enumerated device.
func resolvedAccelDeviceOptions(caps *capabilities.Capabilities, gpus []platform.GPUDevice) (autoLabel, resolvedID string, opts []AccelDeviceOption) {
resolved := caps.Platform.Details["render_device"]
autoAccel := autoGPUAccel(caps)
wslID, wslLabel := wslNVIDIAAccelDefault(caps)
seen := map[string]bool{}
for _, g := range gpus {
label := gpuClassLabel(g) + " (" + gpuDisplayName(g) + ")"
if seen[accelDeviceID(g)] {
continue
}
seen[accelDeviceID(g)] = true
switch {
case resolvedID == "" && wslID != "" && g.Vendor == "nvidia" && g.RenderNode == "" && resolved == "" && autoAccel.Vendor == "":
// WSL exposes the NVIDIA GPU without a DRM node; the enumerated
// entry is the auto default, so it must not be listed twice.
autoLabel, resolvedID = label, accelDeviceID(g)
continue
case g.RenderNode != "" && g.RenderNode == resolved:
autoLabel, resolvedID = label, g.RenderNode
continue
case resolvedID == "" && autoAccel.Vendor != "" && g.Vendor == autoAccel.Vendor:
// Vendor-level default for platforms without a render node (macOS,
// synthetic GPUs): this device backs the preferred hardware encoder,
// so it is the resolved auto selection, not an extra option.
autoLabel, resolvedID = label, accelDeviceID(g)
continue
}
if id := accelDeviceID(g); id != "" {
opts = append(opts, AccelDeviceOption{ID: id, Label: label})
}
}
if autoLabel == "" {
autoLabel = autoAccel.Name
}
if resolvedID == "" {
if wslID != "" {
if autoLabel == "" {
autoLabel = wslLabel
}
resolvedID = wslID
}
}
return autoLabel, resolvedID, opts
}
// wslNVIDIAAccelDefault exposes dgpu when WSL has an NVIDIA driver but no DRM nodes.
func wslNVIDIAAccelDefault(caps *capabilities.Capabilities) (id, label string) {
if caps == nil || !caps.Platform.WSL || !caps.Platform.NVIDIA {
return "", ""
}
name := strings.TrimSpace(caps.Platform.Details["gpu"])
if name == "" {
name = platform.NVIDIAGPUName()
}
if name == "" {
name = "NVIDIA GPU"
}
return "dgpu", "Dedicated GPU (" + name + ")"
}
// accelDeviceID prefers the render node so the selection can be passed back to
// ffmpeg; GPU name is the fallback on platforms without DRM render nodes.
func accelDeviceID(g platform.GPUDevice) string {
if g.RenderNode != "" {
return g.RenderNode
}
return g.Name
}
// gpuClassLabel names the device class shown in the dropdown.
func gpuClassLabel(g platform.GPUDevice) string {
switch g.DeviceClass {
case "soc":
return "SoC GPU"
case "igpu":
return "Integrated GPU"
case "dgpu":
return "Dedicated GPU"
}
if g.Integrated {
return "Integrated GPU"
}
return "Dedicated GPU"
}
func gpuDisplayName(g platform.GPUDevice) string {
if g.Name != "" {
return g.Name
}
return g.RenderNode
}
// autoGPUAccel describes the GPU driving the detected default hardware
// encoder (e.g. Apple VideoToolbox → "Apple GPU"/vendor "apple").
func autoGPUAccel(caps *capabilities.Capabilities) capabilities.AccelGPU {
var fallback capabilities.AccelType
for _, opt := range caps.EncodeOptions {
if !opt.Available || opt.Accel == "" || opt.Accel == capabilities.AccelNone {
continue
}
if opt.Codec != capabilities.CodecH264 {
continue
}
if opt.Default {
if gpu := capabilities.GPUForAccel(caps, opt.Accel); gpu.Name != "" {
return gpu
}
}
fallback = opt.Accel
}
gpu := capabilities.GPUForAccel(caps, fallback)
if gpu.Name == "CPU" || gpu.Vendor == "software" {
return capabilities.AccelGPU{}
}
return gpu
}
// EncoderBackendOption is one selectable entry in the encoder backend list.
type EncoderBackendOption struct {
ID string `json:"id"`
Label string `json:"label"`
}
// EncoderBackendOptions lists concrete H.264 encode paths for the admin UI.
// Labels are "device - encoder" so multiple hardware backends stay distinct.
// An empty stored encoderBackend (Default in the UI) lets go-ffmpeg pick from
// the selected acceleration device.
func (s *Service) EncoderBackendOptions() []EncoderBackendOption {
caps := s.Capabilities()
if caps == nil {
return []EncoderBackendOption{{ID: "cpu", Label: encoderBackendLabel(nil, capabilities.EncodeOption{Encoder: "libx264"})}}
}
var out []EncoderBackendOption
seen := map[string]bool{}
for _, opt := range caps.EncodeOptions {
if opt.Codec != capabilities.CodecH264 || !opt.Available {
continue
}
id := encoderBackendOptionID(opt)
if seen[id] {
continue
}
seen[id] = true
out = append(out, EncoderBackendOption{ID: id, Label: encoderBackendLabel(caps, opt)})
}
if len(out) == 0 {
return []EncoderBackendOption{{ID: "cpu", Label: encoderBackendLabel(caps, capabilities.EncodeOption{Encoder: "libx264"})}}
}
return out
}
func encoderBackendOptionID(opt capabilities.EncodeOption) string {
if opt.Accel == "" || opt.Accel == capabilities.AccelNone {
return "cpu"
}
return opt.Encoder
}
func encoderBackendLabel(caps *capabilities.Capabilities, opt capabilities.EncodeOption) string {
device := "CPU (software)"
if opt.Accel != "" && opt.Accel != capabilities.AccelNone {
if caps != nil {
gpu := capabilities.GPUForAccel(caps, opt.Accel)
switch {
case gpu.RenderNode != "":
device = gpu.RenderNode
case gpu.Name != "" && gpu.Name != "CPU":
device = gpu.Name
default:
device = string(opt.Accel)
}
} else {
device = string(opt.Accel)
}
}
encoder := opt.Encoder
if encoder == "" {
encoder = "libx264"
}
return device + " - " + encoder
}
// Enabled reports whether ffmpeg initialized successfully.
func Enabled() bool {
return global != nil && global.inner != nil
svc := global.Load()
return svc != nil && svc.inner != nil
}
// FFmpegPath returns the resolved ffmpeg binary path.
@@ -98,27 +424,40 @@ func (s *Service) FFprobePath() string {
return s.inner.FFprobePath()
}
func (s *Service) Acquire(ctx context.Context) error {
// ProbeVideoFPS returns the average frame rate of the first video stream.
func (s *Service) ProbeVideoFPS(ctx context.Context, path string) (float64, error) {
if s == nil || s.inner == nil {
return fmt.Errorf("ffmpeg service not available")
return 0, fmt.Errorf("ffmpeg service not available")
}
return s.inner.Acquire(ctx)
}
func (s *Service) Release() {
if s == nil || s.inner == nil {
return
}
s.inner.Release()
return s.inner.ProbeVideoFPS(ctx, path)
}
// VideoPreview extracts a JPEG preview frame to w.
// go-ffmpeg Service methods acquire concurrency slots automatically; do not Acquire before calling.
//
// go-ffmpeg deprecated Service.VideoPreview in favor of CaptureFrame, so this
// wrapper keeps the PreviewOptions call shape used by the preview service while
// delegating to the supported capture API. go-ffmpeg Service methods acquire
// concurrency slots automatically; do not Acquire before calling.
func (s *Service) VideoPreview(ctx context.Context, w io.Writer, opts ops.PreviewOptions) error {
if s == nil || s.inner == nil {
return fmt.Errorf("ffmpeg service not available")
}
return s.inner.VideoPreview(ctx, w, opts)
frame := ops.PreviewOptionsToFrame(opts, 0)
// Mirror the legacy ResolvePreviewSeekSec behavior: an explicit SeekSec
// wins, otherwise SeekPercent (defaulting to 10%) is resolved against the
// duration inside CaptureFrame.
if opts.SeekSec > 0 {
frame.At = ops.AtSec(opts.SeekSec)
} else {
percent := opts.SeekPercent
if percent <= 0 || percent > 100 {
percent = 10
}
frame.At = ops.AtPercent(percent)
}
return s.inner.CaptureFrame(ctx, w, frame)
}
// ExtractSubtitle returns embedded subtitle content as WebVTT.
@@ -145,7 +484,7 @@ func (s *Service) ExtractSubtitle(ctx context.Context, videoPath string, streamI
return content, nil
}
func logCapabilities(svc *goffmpeg.Service, detectHardware bool) {
func logCapabilities(svc *goffmpeg.Service, wantsHardware bool, accelDevice string) {
caps := svc.Capabilities()
if caps == nil {
return
@@ -153,22 +492,104 @@ func logCapabilities(svc *goffmpeg.Service, detectHardware bool) {
logger.Debugf("Media Enabled: version %s @ %s", caps.FFmpegVersion, caps.FFmpegPath)
if !detectHardware {
return
}
warnIfAccelDeviceUnavailable(caps, accelDevice)
hw := hardwareCodecSummary(svc)
if hw == "" {
logger.Warning("no ffmpeg hardware codec support found")
if wantsHardware {
logger.Warning("hardware acceleration is enabled but no ffmpeg hardware codec support was found")
} else {
logger.Debug("no ffmpeg hardware codec support found")
}
logHWEncodeProbeFailures(caps)
return
}
if wantsHardware {
// go-ffmpeg already logs a single hardware support overview line at
// startup; keep the detailed per-codec list at debug only.
logger.Debugf("ffmpeg hardware codecs: %s", hw)
logHWEncodeProbeFailures(caps)
return
}
logger.Debugf("supported ffmpeg hardware codecs: %s", hw)
}
// warnIfAccelDeviceUnavailable surfaces a stale configured device. The stored
// device (e.g. a GPU name saved on another host) may no longer be present; the
// resolver falls back to auto, but silently, so the transcodes look like a
// plain software encode with no explanation. Resolve with no side effects and
// report the fallback at info.
func warnIfAccelDeviceUnavailable(caps *capabilities.Capabilities, accelDevice string) {
if reason, ok := accelDeviceFallbackReason(accelDevice); ok {
logger.Warningf("ffmpeg accel device %q is unavailable (%s); using automatic selection instead", strings.TrimSpace(accelDevice), reason)
}
}
// accelDeviceFallbackReason reports whether a configured accel device cannot be
// used on this host and why. A blank/"auto"/"software" value is not a device
// selection, so it never warns.
func accelDeviceFallbackReason(accelDevice string) (string, bool) {
dev := strings.TrimSpace(accelDevice)
switch strings.ToLower(dev) {
case "", "auto", "software", "cpu", "none":
return "", false
}
sel, err := goffmpeg.ParseGPUSelector(dev)
if err != nil {
return fmt.Sprintf("unparseable device (%v)", err), true
}
choice, err := goffmpeg.ResolveGPUSelector(sel, goffmpeg.GPUFallbackOnUnavailable)
if err != nil {
return err.Error(), true
}
if choice.Fallback {
if choice.FallbackReason != "" {
return choice.FallbackReason, true
}
return "not present on this host", true
}
return "", false
}
// logHWEncodeProbeFailures lists hardware encoders that failed their functional
// probe, distinctly from decoders, so a "hardware decode works but encode
// silently used libx264" case is visible. Only reports when there is at least
// one failure, and only for h264/hevc/av1 (the codecs transcode uses).
func logHWEncodeProbeFailures(caps *capabilities.Capabilities) {
var failures []string
for _, row := range caps.EncoderProbeResults() {
if row.Available || row.Accel == "" || row.Accel == capabilities.AccelNone {
continue
}
entry := fmt.Sprintf("%s (%s, %s)", row.Encoder, row.Accel, row.Code)
if row.Hint != "" {
entry += ": " + row.Hint
}
failures = append(failures, entry)
}
if len(failures) == 0 {
return
}
logger.Debugf("ffmpeg hardware encoders unavailable: %s", strings.Join(failures, "; "))
}
// hardwareCodecSummary lists detected hardware encode/decode paths. It reads
// the capability maps directly so hardware is still reported when detected
// but not preferred (e.g. hardware acceleration disabled).
func hardwareCodecSummary(svc *goffmpeg.Service) string {
if svc == nil {
return ""
}
return hardwareCodecSummaryFromCaps(svc.Capabilities())
}
func hardwareCodecSummaryFromCaps(caps *capabilities.Capabilities) string {
if caps == nil {
return ""
}
seen := make(map[string]struct{})
var parts []string
add := func(entry string) {
if entry == "" {
return
@@ -180,18 +601,28 @@ func hardwareCodecSummary(svc *goffmpeg.Service) string {
parts = append(parts, entry)
}
for _, opt := range svc.AvailableEncodeOptions() {
if opt.Accel == capabilities.AccelNone {
continue
}
add(fmt.Sprintf("%s encode via %s (%s)", opt.Codec, opt.Encoder, capabilities.AccelLabel(opt.Accel)))
for _, name := range caps.HardwareEncoders() {
add(fmt.Sprintf("%s encode via %s (%s)", encoderCodec(name), name,
capabilities.AccelLabel(capabilities.EncoderAccel(name))))
}
for _, opt := range svc.AvailableDecodeOptions() {
if opt.Accel == capabilities.AccelNone {
continue
}
add(fmt.Sprintf("%s decode via %s (%s)", opt.Codec, opt.Decoder, capabilities.AccelLabel(opt.Accel)))
for _, name := range caps.HardwareDecoders() {
add(fmt.Sprintf("%s decode via %s (%s)", caps.Decoders[name].Codec, name,
capabilities.AccelLabel(capabilities.DecoderAccel(name))))
}
return strings.Join(parts, ", ")
}
func encoderCodec(name string) capabilities.VideoCodec {
for _, codec := range []capabilities.VideoCodec{
capabilities.CodecH264,
capabilities.CodecHEVC,
capabilities.CodecAV1,
capabilities.CodecVP9,
} {
if capabilities.EncoderMatchesCodec(name, codec) {
return codec
}
}
return ""
}
@@ -10,6 +10,7 @@ import (
"time"
goffmpeg "github.com/gtsteffaniak/go-ffmpeg"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
)
func requireFFmpegBinaries(t *testing.T) {
@@ -58,13 +59,13 @@ func TestGetMediaDurationConcurrentNoDeadlock(t *testing.T) {
MaxEncode: 2,
},
CacheDir: t.TempDir(),
SkipHWTests: true,
HardwareAcceleration: false,
EncoderHierarchy: []capabilities.AccelType{capabilities.AccelNone},
})
if err != nil {
t.Fatalf("Initialize() error = %v", err)
}
t.Cleanup(func() { global = nil })
t.Cleanup(func() { global.Store(nil) })
svc := Get()
if svc == nil {
+55
View File
@@ -0,0 +1,55 @@
package ffmpeg
import (
"strings"
"testing"
"github.com/gtsteffaniak/go-ffmpeg/capabilities"
)
func TestHardwareCodecSummaryFromCaps(t *testing.T) {
t.Run("reports detected hardware independent of the preference hierarchy", func(t *testing.T) {
caps := capabilities.NewCapabilities()
caps.EncoderHierarchy = []capabilities.AccelType{capabilities.AccelNone}
caps.Encoders["libx264"] = capabilities.EncoderCapability{Name: "libx264", Kind: "software", Compiled: true, Available: true}
caps.Encoders["h264_videotoolbox"] = capabilities.EncoderCapability{Name: "h264_videotoolbox", Kind: "videotoolbox", Compiled: true, Available: true}
caps.Encoders["hevc_videotoolbox"] = capabilities.EncoderCapability{Name: "hevc_videotoolbox", Kind: "videotoolbox", Compiled: true, Available: true}
caps.Decoders["hwaccel:videotoolbox:h264"] = capabilities.DecoderCapability{
Name: "hwaccel:videotoolbox:h264", Kind: "videotoolbox", Codec: capabilities.CodecH264, Compiled: true, Available: true,
}
got := hardwareCodecSummaryFromCaps(caps)
if !strings.Contains(got, "h264 encode via h264_videotoolbox (Apple VideoToolbox)") {
t.Fatalf("summary = %q, want videotoolbox encode entry", got)
}
if !strings.Contains(got, "h264 decode via hwaccel:videotoolbox:h264 (Apple VideoToolbox)") {
t.Fatalf("summary = %q, want videotoolbox decode entry", got)
}
if strings.Contains(got, "libx264") {
t.Fatalf("summary = %q, must not list software encoders", got)
}
})
t.Run("skips hardware that failed detection", func(t *testing.T) {
caps := capabilities.NewCapabilities()
caps.Encoders["h264_nvenc"] = capabilities.EncoderCapability{Name: "h264_nvenc", Kind: "nvenc", Compiled: true, Available: false}
if got := hardwareCodecSummaryFromCaps(caps); got != "" {
t.Fatalf("summary = %q, want empty when hardware is unavailable", got)
}
})
t.Run("empty when only software is present", func(t *testing.T) {
caps := capabilities.NewCapabilities()
caps.Encoders["libx264"] = capabilities.EncoderCapability{Name: "libx264", Kind: "software", Compiled: true, Available: true}
caps.Decoders["h264"] = capabilities.DecoderCapability{Name: "h264", Kind: "software", Compiled: true, Available: true}
if got := hardwareCodecSummaryFromCaps(caps); got != "" {
t.Fatalf("summary = %q, want empty", got)
}
})
t.Run("nil capabilities", func(t *testing.T) {
if got := hardwareCodecSummaryFromCaps(nil); got != "" {
t.Fatalf("summary = %q, want empty", got)
}
})
}
+23 -4
View File
@@ -35,22 +35,41 @@ func DetectEmbeddedSubtitles(videoPath string, modtime time.Time) []utils.Subtit
return subtitles
}
func mapSubtitleTracks(tracks []goffmpeg.SubtitleTrack) []utils.SubtitleTrack {
subtitles := make([]utils.SubtitleTrack, 0, len(tracks))
// SubtitleStream is one embedded subtitle stream, independent of the probe
// backend (ffprobe via go-ffmpeg, or go-media).
type SubtitleStream struct {
Index int
Codec string
Language string
Title string
}
// BuildSubtitleTracks maps embedded subtitle streams to track entries. Shared by
// the ffmpeg and go-media paths so both produce identical track lists.
func BuildSubtitleTracks(streams []SubtitleStream) []utils.SubtitleTrack {
subtitles := make([]utils.SubtitleTrack, 0, len(streams))
mapper := &subtitleTrackMapper{
seenLang: make(map[string]int),
}
for _, stream := range tracks {
for _, stream := range streams {
subtitles = append(subtitles, mapper.track(stream))
}
return subtitles
}
func mapSubtitleTracks(tracks []goffmpeg.SubtitleTrack) []utils.SubtitleTrack {
streams := make([]SubtitleStream, 0, len(tracks))
for _, t := range tracks {
streams = append(streams, SubtitleStream{Index: t.Index, Codec: t.Codec, Language: t.Language, Title: t.Title})
}
return BuildSubtitleTracks(streams)
}
type subtitleTrackMapper struct {
seenLang map[string]int
}
func (m *subtitleTrackMapper) track(stream goffmpeg.SubtitleTrack) utils.SubtitleTrack {
func (m *subtitleTrackMapper) track(stream SubtitleStream) utils.SubtitleTrack {
index := stream.Index
lang := strings.TrimSpace(stream.Language)
return utils.SubtitleTrack{
+145
View File
@@ -0,0 +1,145 @@
package folio
import (
"fmt"
"io"
"os"
"path/filepath"
"strings"
"unicode/utf8"
"github.com/gen2brain/folio/doc"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
)
const folioTextCheckWindow = 4096 // html.readsAsText validation window
// maxTextPreviewLoad is the most plain-text we read for a first-page thumbnail.
const maxTextPreviewLoad = 32 << 10 // 32 KiB
// SupportsPreview reports whether folio should attempt this file (name + MIME).
// doc.Open sniffs content; extensions like .css are not in FolioConvertable but still render.
func SupportsPreview(name, mimetype string) bool {
ext := strings.ToLower(filepath.Ext(name))
if iteminfo.FolioConvertable[ext] {
return true
}
if strings.HasPrefix(mimetype, "text/") {
return true
}
return iteminfo.IsText(ext)
}
// useTextPreviewPrefix reports plain UTF-8 text previews (css, logs, .txt, …).
// These use doc.Load on a bounded prefix instead of doc.Open, which would read the whole file.
func useTextPreviewPrefix(name, mimetype string) bool {
ext := strings.ToLower(filepath.Ext(name))
if iteminfo.FolioConvertable[ext] {
switch ext {
case ".txt", ".md":
return true
default:
return false
}
}
if strings.HasPrefix(mimetype, "text/") {
return ext != ".html" && ext != ".htm"
}
return iteminfo.IsText(ext)
}
func openDocument(path, name, mimetype string) (doc.Document, error) {
if useTextPreviewPrefix(name, mimetype) {
return openTextPreviewPrefix(path)
}
return doc.Open(path)
}
func openTextPreviewPrefix(path string) (doc.Document, error) {
b, err := readTextPreviewPrefix(path, maxTextPreviewLoad)
if err != nil {
return nil, err
}
b = sanitizeFolioText(b)
b = repairFolioTextValidationWindow(b)
if len(b) == 0 {
return nil, fmt.Errorf("text document is empty or not valid utf-8")
}
return doc.Load(b)
}
func readTextPreviewPrefix(path string, max int) ([]byte, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
b, err := io.ReadAll(io.LimitReader(f, int64(max)))
if err != nil {
return nil, err
}
for len(b) > 0 && !utf8.Valid(b) {
b = b[:len(b)-1]
}
return b, nil
}
// sanitizeFolioText strips control characters folio/html.readsAsText rejects (except tab and common whitespace).
func sanitizeFolioText(b []byte) []byte {
if len(b) == 0 {
return b
}
out := make([]byte, 0, len(b))
for len(b) > 0 {
r, n := utf8.DecodeRune(b)
if r == utf8.RuneError && n == 1 {
break
}
if r < 0x20 && r != '\t' && r != '\n' && r != '\r' && r != '\f' {
out = append(out, ' ')
} else {
out = append(out, b[:n]...)
}
b = b[n:]
}
return out
}
// repairFolioTextValidationWindow fixes folio/html readsAsText, which validates only
// the first 4096 bytes as UTF-8. A valid UTF-8 file can fail when a rune spans that boundary.
func repairFolioTextValidationWindow(b []byte) []byte {
for len(b) > folioTextCheckWindow && !utf8.Valid(b[:folioTextCheckWindow]) {
next := removeRuneSpanningBoundary(b, folioTextCheckWindow)
if len(next) == len(b) {
break
}
b = next
}
return b
}
func removeRuneSpanningBoundary(b []byte, boundary int) []byte {
if boundary <= 0 || boundary > len(b) || utf8.Valid(b[:boundary]) {
return b
}
start := boundary - 1
for start > 0 && !utf8.RuneStart(b[start]) {
start--
}
if !utf8.RuneStart(b[start]) {
return b
}
_, size := utf8.DecodeRune(b[start:])
if start+size <= boundary {
return b
}
end := start + size
if end > len(b) {
end = len(b)
}
out := make([]byte, 0, len(b)-(end-start))
out = append(out, b[:start]...)
out = append(out, b[end:]...)
return out
}
+167
View File
@@ -0,0 +1,167 @@
// Package folio is the pure-Go document preview fast path using gen2brain/folio.
// When folio cannot handle a file, callers fall back to the office converter (when configured).
package folio
import (
"bytes"
"context"
"errors"
"fmt"
"image"
"image/jpeg"
"path/filepath"
"strings"
"sync"
"github.com/gen2brain/folio/doc"
"github.com/gtsteffaniak/filebrowser/backend/pkg/indexing/iteminfo"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-logger/logger"
"github.com/kovidgoyal/imaging"
)
var renderMu sync.Mutex
// RenderOptions controls thumbnail sizing for one rendered page.
type RenderOptions struct {
MaxWidth int
MaxHeight int
Fill bool
JpegQuality int
Original bool
}
// Enabled reports whether the folio fast path may run.
func Enabled() bool {
return !settings.AlwaysUseOnlyOfficeEnabled()
}
// SupportsExtension reports whether folio targets the file extension (including leading dot).
func SupportsExtension(ext string) bool {
ext = strings.ToLower(ext)
if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
return iteminfo.FolioConvertable[ext]
}
func logFallback(op, path string, err error) {
if errors.Is(err, doc.ErrUnsupported) {
logger.Warningf("folio does not support %s for %s (%v); using office converter if configured", op, path, err)
return
}
logger.Errorf("folio %s failed for %s: %v; falling back to office converter if configured", op, path, err)
}
// RenderPageJPEG renders one document page to JPEG. ok is true when folio returned bytes.
// name and mimetype should match the file being previewed; they control eligibility for
// folio (content sniffing does not depend on the path extension).
func RenderPageJPEG(ctx context.Context, path string, pageNumber int, opts RenderOptions, name, mimetype string) (data []byte, ok bool, err error) {
if !Enabled() {
return nil, false, nil
}
if name == "" {
name = filepath.Base(path)
}
if !SupportsPreview(name, mimetype) {
return nil, false, nil
}
if ctx.Err() != nil {
return nil, false, ctx.Err()
}
type result struct {
data []byte
err error
}
done := make(chan result, 1)
go func() {
defer func() {
if r := recover(); r != nil {
done <- result{err: fmt.Errorf("folio render panicked: %v", r)}
}
}()
data, err := renderPageJPEG(ctx, path, name, mimetype, pageNumber, opts)
done <- result{data: data, err: err}
}()
select {
case <-ctx.Done():
return nil, false, ctx.Err()
case res := <-done:
if res.err != nil {
if !errors.Is(res.err, context.Canceled) && !errors.Is(res.err, context.DeadlineExceeded) {
logFallback("render", path, res.err)
}
return nil, false, res.err
}
if len(res.data) == 0 {
return nil, false, fmt.Errorf("folio render returned empty jpeg")
}
return res.data, true, nil
}
}
func renderPageJPEG(ctx context.Context, path, name, mimetype string, pageNumber int, opts RenderOptions) ([]byte, error) {
renderMu.Lock()
defer renderMu.Unlock()
if err := ctx.Err(); err != nil {
return nil, err
}
d, err := openDocument(path, name, mimetype)
if err != nil {
return nil, err
}
defer d.Close()
if pageNumber < 0 || pageNumber >= d.NumPages() {
return nil, fmt.Errorf("invalid page number %d (pages: %d)", pageNumber, d.NumPages())
}
p, err := d.Page(pageNumber)
if err != nil {
return nil, err
}
var img image.Image
if opts.Original || opts.MaxWidth <= 0 || opts.MaxHeight <= 0 {
const maxEdge = 4096
rgba, err := p.ImageDPI(150)
if err != nil {
return nil, err
}
img = rgba
w, h := rgba.Bounds().Dx(), rgba.Bounds().Dy()
if w > maxEdge || h > maxEdge {
scale := min(float64(maxEdge)/float64(w), float64(maxEdge)/float64(h))
newW := max(1, int(float64(w)*scale))
newH := max(1, int(float64(h)*scale))
img = imaging.Resize(img, newW, newH, imaging.Lanczos)
}
} else {
rgba, err := p.ImageDPI(150)
if err != nil {
return nil, err
}
img = rgba
if opts.Fill {
img = imaging.Fill(img, opts.MaxWidth, opts.MaxHeight, imaging.Center, imaging.Lanczos)
} else {
b := img.Bounds()
if b.Dx() > opts.MaxWidth || b.Dy() > opts.MaxHeight {
img = imaging.Fit(img, opts.MaxWidth, opts.MaxHeight, imaging.Lanczos)
}
}
}
q := opts.JpegQuality
if q <= 0 {
q = 85
}
var buf bytes.Buffer
if err := jpeg.Encode(&buf, img, &jpeg.Options{Quality: q}); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
+34
View File
@@ -0,0 +1,34 @@
package folio
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
)
func TestRenderPageJPEG_txt(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "hello.txt")
require.NoError(t, os.WriteFile(path, []byte("Hello folio preview"), 0o644))
data, ok, err := RenderPageJPEG(t.Context(), path, 0, RenderOptions{
MaxWidth: 256,
MaxHeight: 256,
Fill: true,
JpegQuality: 80,
}, "hello.txt", "text/plain")
require.NoError(t, err)
require.True(t, ok)
require.NotEmpty(t, data)
require.Equal(t, byte(0xff), data[0])
require.Equal(t, byte(0xd8), data[1])
}
func TestRenderPageJPEG_unsupportedExtension(t *testing.T) {
data, ok, err := RenderPageJPEG(t.Context(), "/nonexistent/file.xyz", 0, RenderOptions{}, "file.xyz", "application/octet-stream")
require.NoError(t, err)
require.False(t, ok)
require.Nil(t, data)
}
+72
View File
@@ -0,0 +1,72 @@
package folio
import (
"os"
"path/filepath"
"strings"
"testing"
"unicode/utf8"
"github.com/stretchr/testify/require"
)
func TestRepairFolioTextValidationWindow_playwrightFixture(t *testing.T) {
realPath := filepath.Join("..", "..", "..", "frontend", "tests", "playwright-files", "utf8-truncated.txt")
b, err := os.ReadFile(realPath)
if err != nil {
t.Skip("playwright fixture missing")
}
require.True(t, utf8.Valid(b))
require.False(t, utf8.Valid(b[:folioTextCheckWindow]))
fixed := repairFolioTextValidationWindow(b)
require.True(t, utf8.Valid(fixed[:min(len(fixed), folioTextCheckWindow)]))
}
func TestOpenDocument_cssRealPath(t *testing.T) {
dir := t.TempDir()
cssPath := filepath.Join(dir, "styles.css")
require.NoError(t, os.WriteFile(cssPath, []byte("body { color: red; }\n"), 0o644))
d, err := openDocument(cssPath, "styles.css", "text/css")
require.NoError(t, err)
require.NoError(t, d.Close())
}
func TestSanitizeFolioText_stripsNUL(t *testing.T) {
raw := []byte("hello\x00world")
got := sanitizeFolioText(raw)
require.Equal(t, []byte("hello world"), got)
opts, err := folioRenderOptionsForTest()
require.NoError(t, err)
dir := t.TempDir()
path := filepath.Join(dir, "nul.txt")
require.NoError(t, os.WriteFile(path, raw, 0o600))
data, ok, err := RenderPageJPEG(t.Context(), path, 0, opts, "nul.txt", "text/plain")
require.NoError(t, err)
require.True(t, ok, "err=%v", err)
require.NotEmpty(t, data)
}
func folioRenderOptionsForTest() (RenderOptions, error) {
return RenderOptions{MaxWidth: 256, MaxHeight: 256, Fill: true, JpegQuality: 80}, nil
}
func TestReadTextPreviewPrefix_cap(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "big.log")
payload := strings.Repeat("x", maxTextPreviewLoad+1024)
require.NoError(t, os.WriteFile(path, []byte(payload), 0o644))
b, err := readTextPreviewPrefix(path, maxTextPreviewLoad)
require.NoError(t, err)
require.Len(t, b, maxTextPreviewLoad)
}
func TestUseTextPreviewPrefix(t *testing.T) {
require.True(t, useTextPreviewPrefix("styles.css", "text/css"))
require.True(t, useTextPreviewPrefix("readme.txt", "text/plain"))
require.False(t, useTextPreviewPrefix("page.html", "text/html"))
require.False(t, useTextPreviewPrefix("doc.pdf", "application/pdf"))
}
+678
View File
@@ -0,0 +1,678 @@
// Package gomedia is the go-media fast path for probing, duration, tags, lyrics,
// embedded subtitles, video thumbnails, HEIC/JXL decode, and camera RAW previews.
//
// Every function returns ok=false when the caller must use the existing ffmpeg
// (or dhowden) code instead. That happens when:
// - the admin "Always use ffmpeg" setting is on (no log, go-media is skipped);
// - go-media reports the input as unsupported (logged as a warning);
// - go-media fails for any other reason (logged as an error).
//
// Callers never resize or crop after go-media. Output sizes are requested from
// go-media directly.
package gomedia
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/gtsteffaniak/filebrowser/backend/internal/ffmpeg"
"github.com/gtsteffaniak/filebrowser/backend/internal/utils"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/gtsteffaniak/go-cache/cache"
"github.com/gtsteffaniak/go-logger/logger"
"github.com/gtsteffaniak/go-media/pkg/media"
"golang.org/x/sync/singleflight"
)
var (
subtitleListCache = cache.NewCache[[]utils.SubtitleTrack](24 * time.Hour)
subtitleContentCache = cache.NewCache[string](24 * time.Hour)
// probeCache caches successful probe results by file identity (path, modtime,
// size). A directory listing probes every video child, and the preview flow
// probes the focused file again; without this each request re-read the same
// containers. Keying on modtime+size means an edited file is re-probed.
probeCache = cache.NewCache[ProbeResult](24 * time.Hour)
// probeGroup coalesces concurrent probes of the same file so a burst of
// requests for one container reads it once.
probeGroup singleflight.Group
)
// Enabled reports whether the go-media fast path may run. The admin
// "Always use ffmpeg" setting turns it off.
func Enabled() bool {
return !settings.AlwaysUseFfmpegEnabled()
}
// videoDimensions returns the first video stream's width and height.
func videoDimensions(info *media.ProbeInfo) (w, h int, ok bool) {
for _, s := range info.Streams {
if s.Type == media.StreamVideo && s.Width > 0 && s.Height > 0 {
return s.Width, s.Height, true
}
}
return 0, 0, false
}
// videoContainerExts are the containers go-media parses for video probes,
// thumbnails, and embedded subtitles. Other video containers (AVI, WMV, TS, ...)
// are not go-media targets, so they go straight to ffmpeg with no file open and
// no log line. go-media reads MOV as MP4.
var videoContainerExts = map[string]bool{
".mp4": true,
".m4v": true,
".mov": true,
".mkv": true,
".webm": true,
}
// heifImageExts are still-image containers go-media decodes for HEIC preview.
var heifImageExts = map[string]bool{
".heic": true,
".heif": true,
".heics": true,
}
const heicDecodeMaxEdge = 16384
// cameraRawExts match go-media rawpreview formats (also iteminfo raw extensions).
var cameraRawExts = map[string]bool{
".cr2": true, ".cr3": true, ".nef": true, ".nrw": true, ".arw": true, ".srf": true, ".sr2": true,
".orf": true, ".rw2": true, ".raw": true, ".dng": true, ".raf": true, ".pef": true, ".ptx": true,
".rwl": true, ".3fr": true, ".fff": true, ".erf": true, ".mrw": true, ".dcr": true, ".kdc": true,
".dc2": true, ".x3f": true, ".iiq": true, ".nkc": true, ".r3d": true,
}
// supportsVideoContainer reports whether go-media targets path's container.
func supportsVideoContainer(path string) bool {
return videoContainerExts[strings.ToLower(filepath.Ext(path))]
}
func supportsHEIFImage(path string) bool {
return heifImageExts[strings.ToLower(filepath.Ext(path))]
}
func supportsCameraRAW(path string) bool {
return cameraRawExts[strings.ToLower(filepath.Ext(path))]
}
// logFallback records why go-media was not used for op on path, for operations
// that fall back to ffmpeg.
func logFallback(op, path string, err error) {
logFallbackTo(op, path, err, "using ffmpeg", "falling back to ffmpeg")
}
// logFallbackTo records a go-media failure. next is the outcome for an
// unsupported input and failed is the outcome for a real error; both are
// phrases appended to the message. JPEG XL has no fallback, so it passes
// phrases that do not mention ffmpeg.
func logFallbackTo(op, path string, err error, next, failed string) {
// ErrNotMedia means the file is not a container go-media reads. That is an
// expected fallback, not a failure, so it logs as a warning.
if errors.Is(err, media.ErrUnsupported) || errors.Is(err, media.ErrUnsupportedProfile) || errors.Is(err, media.ErrNotMedia) {
logger.Warningf("go-media does not support %s for %s (%v); %s", op, path, err, next)
return
}
logger.Errorf("go-media %s failed for %s: %v; %s", op, path, err, failed)
}
// logUnsupported records a container or codec that go-media does not handle.
func logUnsupported(op, path, reason string) {
logger.Warningf("go-media does not support %s for %s (%s); using ffmpeg", op, path, reason)
}
// openFile opens path for go-media, which reads through io.ReaderAt.
func openFile(path string) (*os.File, int64, error) {
f, err := os.Open(path)
if err != nil {
return nil, 0, err
}
info, err := f.Stat()
if err != nil {
_ = f.Close()
return nil, 0, err
}
return f, info.Size(), nil
}
// ffprobeFormatNames maps go-media containers to the format_name strings ffprobe
// reports, so the stored container value does not change for existing users.
// go-media appends the ftyp brand to MP4 names (for example ",isom"), which
// ffprobe does not include, so MP4 is normalized as well.
var ffprobeFormatNames = map[media.Format]string{
media.FormatMP4: "mov,mp4,m4a,3gp,3g2,mj2",
media.FormatMKV: "matroska,webm",
media.FormatWebM: "matroska,webm",
media.FormatWAV: "wav",
media.FormatFLAC: "flac",
media.FormatOGG: "ogg",
media.FormatMP3: "mp3",
media.FormatAIFF: "aiff",
media.FormatAU: "au",
}
// ProbeResult is the same shape as the ffprobe result, so callers can use one
// code path for both backends.
type ProbeResult = ffmpeg.FileProbeInfo
// Probe returns duration, codecs, and container name for a local media file.
// legacy is the ffmpeg probe, run only in shadow mode.
func Probe(path string, legacy Legacy[ProbeResult]) (ProbeResult, bool) {
return timed("probe", path, legacy, probeDiff, func() (ProbeResult, bool) { return probeFast(path) })
}
func probeFast(path string) (ProbeResult, bool) {
if !Enabled() {
return ProbeResult{}, false
}
if !supportsVideoContainer(path) {
return ProbeResult{}, false
}
f, size, err := openFile(path)
if err != nil {
logFallback("probe", path, err)
return ProbeResult{}, false
}
defer f.Close()
info, err := f.Stat()
if err != nil {
logFallback("probe", path, err)
return ProbeResult{}, false
}
cacheKey := probeCacheKey(path, info.ModTime().UnixNano(), size)
if res, ok := probeCache.Get(cacheKey); ok {
return res, true
}
// Coalesce concurrent probes of the same file into one container read.
v, err, _ := probeGroup.Do(cacheKey, func() (any, error) {
res, ok := parseProbe(path, f, size)
if !ok {
return nil, errProbeUnsupported
}
probeCache.Set(cacheKey, res)
return res, nil
})
if err != nil {
return ProbeResult{}, false
}
res, ok := v.(ProbeResult)
if !ok {
return ProbeResult{}, false
}
return res, true
}
// errProbeUnsupported marks a probe that go-media chose not to answer. It carries
// no detail because parseProbe already logged the specific reason.
var errProbeUnsupported = errors.New("probe unsupported")
// probeCacheKey identifies a file version for caching: path plus modtime and size.
// An edited file gets a new key, so stale metadata is never served.
func probeCacheKey(path string, modTimeNanos, size int64) string {
return fmt.Sprintf("probe:%s:%d:%d", path, modTimeNanos, size)
}
// parseProbe reads a container and builds a ProbeResult. It reports false when
// go-media should defer to ffmpeg, having logged why.
func parseProbe(path string, f *os.File, size int64) (ProbeResult, bool) {
info, err := media.Probe(f, size)
if err != nil {
logFallback("probe", path, err)
return ProbeResult{}, false
}
if info.Duration <= 0 {
// Fragmented or unusual files can report no length. ffprobe answers instead,
// the same as durationFast does.
logUnsupported("probe", path, "duration not in container header")
return ProbeResult{}, false
}
res := ProbeResult{Duration: info.Duration.Seconds(), FormatName: info.FormatName}
if name, ok := ffprobeFormatNames[info.Format]; ok {
res.FormatName = name
}
for _, s := range info.Streams {
switch s.Type {
case media.StreamVideo:
if s.Codec == "" {
// The UI would show no codec, so let ffprobe answer instead.
logUnsupported("probe", path, "video codec not identified")
return ProbeResult{}, false
}
if res.VideoCodec == "" {
res.VideoCodec = s.Codec
}
case media.StreamAudio:
if s.Codec == "" {
logUnsupported("probe", path, "audio codec not identified")
return ProbeResult{}, false
}
if res.AudioCodec == "" {
res.AudioCodec = s.Codec
}
}
}
return res, true
}
// Duration returns the media length in seconds from container headers.
// go-media may report an estimate (MP3 without a VBR header); the source is not
// surfaced to callers.
// legacy is the ffmpeg duration probe, run only in shadow mode.
func Duration(path string, legacy Legacy[float64]) (float64, bool) {
return timed("duration", path, legacy, durationDiff, func() (float64, bool) { return durationFast(path) })
}
func durationFast(path string) (float64, bool) {
if !Enabled() {
return 0, false
}
f, size, err := openFile(path)
if err != nil {
logFallback("duration", path, err)
return 0, false
}
defer f.Close()
res, err := media.Duration(f, size)
if err != nil {
logFallback("duration", path, err)
return 0, false
}
if res.Duration <= 0 {
// Unknown duration is not an error; ffmpeg may still find one.
return 0, false
}
return res.Duration.Seconds(), true
}
// AudioInfo is the tag, lyrics availability, and cover art FileBrowser reads
// from an audio file.
type AudioInfo struct {
Tags media.Tags
HasLyrics bool
Artwork []byte
}
// Audio reads tags and lyrics availability. Artwork is read only when withArt
// is true. A missing cover is not an error.
// legacy is the dhowden tag read, run only in shadow mode.
func Audio(path string, withArt bool, legacy Legacy[AudioInfo]) (AudioInfo, bool) {
return timed("tags", path, legacy, audioDiffer(withArt), func() (AudioInfo, bool) { return audioFast(path, withArt) })
}
func audioFast(path string, withArt bool) (AudioInfo, bool) {
if !Enabled() {
return AudioInfo{}, false
}
f, size, err := openFile(path)
if err != nil {
logFallback("tags", path, err)
return AudioInfo{}, false
}
defer f.Close()
// One container parse gives tags, the embedded lyrics flag, and cover bytes.
// Separate ReadTags/HasLyrics/EmbeddedArtwork calls each re-parsed the file.
audio, err := media.InspectAudio(f, size, media.AudioOptions{WithArtwork: withArt})
if err != nil {
logFallback("tags", path, err)
return AudioInfo{}, false
}
info := AudioInfo{Tags: audio.Tags, HasLyrics: audio.HasEmbeddedLyrics}
if !info.HasLyrics {
// Sidecar lyrics are only checked when the file has no embedded lyrics.
_, _, info.HasLyrics = media.LyricsSidecar(path)
}
if withArt {
info.Artwork = audio.Artwork
}
return info, true
}
// Lyrics returns lyrics from a sidecar file, or else from embedded tags. The
// format is "embedded" for tags or the sidecar extension. An empty result with
// ok=true means go-media checked and found no lyrics.
// legacy is the sidecar and dhowden lyrics read, run only in shadow mode.
func Lyrics(path string, legacy Legacy[LyricsResult]) (text, format string, ok bool) {
res, ok := timed("lyrics", path, legacy, lyricsDiff, func() (LyricsResult, bool) {
t, f, found := lyricsFast(path)
return LyricsResult{Text: t, Format: f}, found
})
return res.Text, res.Format, ok
}
func lyricsFast(path string) (text, format string, ok bool) {
if !Enabled() {
return "", "", false
}
if sidecarText, ext, found := media.LyricsSidecar(path); found {
return sidecarText, ext, true
}
f, size, err := openFile(path)
if err != nil {
logFallback("lyrics", path, err)
return "", "", false
}
defer f.Close()
// InspectAudio puts embedded lyrics in Tags.Raw["lyrics"] from the same parse.
audio, err := media.InspectAudio(f, size, media.AudioOptions{})
if err != nil {
logFallback("lyrics", path, err)
return "", "", false
}
text = audio.Tags.Raw["lyrics"]
if text == "" {
return "", "", false
}
return text, "embedded", true
}
// EmbeddedSubtitles lists embedded subtitle tracks. go-media supports MKV, WebM,
// and MP4 only. Other containers are reported as unsupported, not as "no
// subtitles", because go-media returns an empty list for them.
// legacy is the ffmpeg subtitle detection, run only in shadow mode.
func EmbeddedSubtitles(path string, modtime time.Time, legacy Legacy[[]utils.SubtitleTrack]) ([]utils.SubtitleTrack, bool) {
return timed("subtitle list", path, legacy, subtitleListDiff, func() ([]utils.SubtitleTrack, bool) {
return embeddedSubtitlesFast(path, modtime)
})
}
func embeddedSubtitlesFast(path string, modtime time.Time) ([]utils.SubtitleTrack, bool) {
if !Enabled() {
return nil, false
}
if !supportsVideoContainer(path) {
return nil, false
}
key := fmt.Sprintf("embedded_subtitles:%s:%d", path, modtime.Unix())
if cached, ok := subtitleListCache.Get(key); ok {
return cached, true
}
f, size, err := openFile(path)
if err != nil {
logFallback("subtitle list", path, err)
return nil, false
}
defer f.Close()
info, err := media.Probe(f, size)
if err != nil {
logFallback("subtitle list", path, err)
return nil, false
}
switch info.Format {
case media.FormatMKV, media.FormatWebM, media.FormatMP4:
default:
logUnsupported("subtitle list", path, "container "+string(info.Format))
return nil, false
}
streams := make([]ffmpeg.SubtitleStream, 0, len(info.SubtitleTracks))
for _, t := range info.SubtitleTracks {
streams = append(streams, ffmpeg.SubtitleStream{
Index: t.Index,
Codec: t.Codec,
Language: t.Language,
Title: t.Title,
})
}
tracks := ffmpeg.BuildSubtitleTracks(streams)
subtitleListCache.Set(key, tracks)
return tracks, true
}
// ExtractSubtitle returns the content of an embedded subtitle stream as WebVTT.
// streamIndex is the same index EmbeddedSubtitles reports.
// legacy is the ffmpeg subtitle extraction, run only in shadow mode.
func ExtractSubtitle(path string, streamIndex int, legacy Legacy[string]) (string, bool) {
return timed("subtitle extract", path, legacy, subtitleExtractDiff, func() (string, bool) {
return extractSubtitleFast(path, streamIndex)
})
}
func extractSubtitleFast(path string, streamIndex int) (string, bool) {
if !Enabled() {
return "", false
}
if !supportsVideoContainer(path) {
return "", false
}
st, err := os.Stat(path)
if err != nil {
logFallback("subtitle extract", path, err)
return "", false
}
key := fmt.Sprintf("subtitle_content:%s:%d:%d", path, streamIndex, st.ModTime().Unix())
if content, ok := subtitleContentCache.Get(key); ok {
return content, true
}
f, size, err := openFile(path)
if err != nil {
logFallback("subtitle extract", path, err)
return "", false
}
defer f.Close()
vtt, err := media.ExtractEmbeddedSubtitle(f, size, streamIndex)
if err != nil {
logFallback("subtitle extract", path, err)
return "", false
}
if strings.TrimSpace(vtt) == "" {
logFallback("subtitle extract", path, errors.New("empty subtitle output"))
return "", false
}
subtitleContentCache.Set(key, vtt)
return vtt, true
}
// VideoThumbnail returns a JPEG frame at seekPercent that fits inside
// width x height with aspect ratio kept. Crop is never applied. Quality is JPEG
// quality 1-100.
//
// A zero width or height means "original size". go-media defaults a zero bound
// to 320px, so the source's own dimensions are passed as the bound instead.
// Then no scaling happens in go-media either.
// legacy is the ffmpeg preview frame, run only in shadow mode.
func VideoThumbnail(path string, width, height, quality int, seekPercent float64, legacy Legacy[[]byte]) ([]byte, bool) {
return timed("video thumbnail", path, legacy, thumbnailDiff, func() ([]byte, bool) {
return videoThumbnailFast(path, width, height, quality, seekPercent)
})
}
func videoThumbnailFast(path string, width, height, quality int, seekPercent float64) ([]byte, bool) {
if !Enabled() {
return nil, false
}
if !supportsVideoContainer(path) {
return nil, false
}
f, size, err := openFile(path)
if err != nil {
logFallback("video thumbnail", path, err)
return nil, false
}
defer f.Close()
if width <= 0 || height <= 0 {
info, probeErr := media.Probe(f, size)
if probeErr != nil {
logFallback("video thumbnail", path, probeErr)
return nil, false
}
srcW, srcH, found := videoDimensions(info)
if !found {
logUnsupported("video thumbnail", path, "source dimensions not found")
return nil, false
}
if width <= 0 {
width = srcW
}
if height <= 0 {
height = srcH
}
}
res, err := media.Thumbnail(f, size, media.ThumbnailOptions{
MaxWidth: width,
MaxHeight: height,
SeekPercent: seekPercent,
Format: "jpeg",
Quality: quality,
})
if err != nil {
logFallback("video thumbnail", path, err)
return nil, false
}
if len(res.Data) == 0 {
logFallback("video thumbnail", path, errors.New("empty thumbnail output"))
return nil, false
}
return res.Data, true
}
// HEICJPEG decodes HEIC/HEIF to JPEG at full resolution (bounds heicDecodeMaxEdge).
// Resize for small/large/xlarge previews is done later by the preview service.
// legacy is ffmpeg full decode, run only in shadow mode.
func HEICJPEG(path string, quality int, legacy Legacy[[]byte]) ([]byte, bool) {
return timed("heic jpeg", path, legacy, thumbnailDiff, func() ([]byte, bool) {
return heicJPEGFast(path, quality)
})
}
func heicJPEGFast(path string, quality int) ([]byte, bool) {
return gatedStillImageJPEG(path, heicDecodeMaxEdge, heicDecodeMaxEdge, quality, "heic jpeg", supportsHEIFImage)
}
// JXLJPEG decodes JPEG XL to JPEG at full resolution (bounds heicDecodeMaxEdge).
// JPEG XL is decoded only by go-media, so the "Always use ffmpeg" setting does
// not disable it.
func JXLJPEG(path string, quality int, legacy Legacy[[]byte]) ([]byte, bool) {
return timed("jxl jpeg", path, legacy, thumbnailDiff, func() ([]byte, bool) {
return jxlJPEGFast(path, quality)
})
}
func jxlJPEGFast(path string, quality int) ([]byte, bool) {
// No extension check: the caller's display name can be .jxl while the resolved
// path (for example a symlink target) has another name. go-media sniffs the
// content, so a non-JPEG XL file fails in decode and is logged there.
return stillImageJPEG(path, heicDecodeMaxEdge, heicDecodeMaxEdge, quality, "jxl jpeg", acceptAnyPath, false)
}
// CameraRAWJPEG decodes camera RAW to JPEG at the requested bounds (0 means full
// resolution up to heicDecodeMaxEdge). legacy is ffmpeg ConvertImageToJPEG, shadow only.
func CameraRAWJPEG(path string, width, height, quality int, legacy Legacy[[]byte]) ([]byte, bool) {
return timed("raw jpeg", path, legacy, thumbnailDiff, func() ([]byte, bool) {
return cameraRAWJPEGFast(path, width, height, quality)
})
}
func cameraRAWJPEGFast(path string, width, height, quality int) ([]byte, bool) {
maxW, maxH := width, height
if maxW <= 0 || maxH <= 0 {
maxW, maxH = heicDecodeMaxEdge, heicDecodeMaxEdge
}
return gatedStillImageJPEG(path, maxW, maxH, quality, "raw jpeg", supportsCameraRAW)
}
// gatedStillImageJPEG applies the "Always use ffmpeg" setting before decoding.
// Used for formats ffmpeg can also decode (HEIC, camera RAW). JPEG XL skips it.
func gatedStillImageJPEG(path string, maxW, maxH, quality int, op string, supported func(string) bool) ([]byte, bool) {
if !Enabled() {
if ShadowEnabled() {
logger.Debugf("go-media skipped for %s (%s): always use ffmpeg enabled", op, path)
}
return nil, false
}
return stillImageJPEG(path, maxW, maxH, quality, op, supported, true)
}
// acceptAnyPath is the format gate for decoders that identify the file by content.
func acceptAnyPath(string) bool { return true }
// stillImageJPEG decodes a still image with go-media. ffmpegFallback selects the
// log wording: formats ffmpeg can also decode say so; JPEG XL, which ffmpeg
// cannot decode, does not.
func stillImageJPEG(path string, maxW, maxH, quality int, op string, supported func(string) bool, ffmpegFallback bool) ([]byte, bool) {
logFail := func(err error) {
if ffmpegFallback {
logFallback(op, path, err)
return
}
logFallbackTo(op, path, err, "JPEG XL has no fallback", "JPEG XL has no fallback")
}
if !supported(path) {
return nil, false
}
f, size, err := openFile(path)
if err != nil {
logFail(err)
return nil, false
}
defer f.Close()
if quality <= 0 {
quality = 85
}
res, err := media.Thumbnail(f, size, media.ThumbnailOptions{
MaxWidth: maxW,
MaxHeight: maxH,
Format: "jpeg",
Quality: quality,
})
if err != nil {
logFail(err)
return nil, false
}
if len(res.Data) == 0 {
logFail(fmt.Errorf("empty %s output", op))
return nil, false
}
return res.Data, true
}
// SidecarSubtitleExts are the sidecar formats go-media lists and converts.
// Other subtitle extensions in iteminfo.SubtitleExts keep the directory scan.
var SidecarSubtitleExts = map[string]bool{
".srt": true,
".vtt": true,
".ass": true,
".ssa": true,
}
// ExternalSubtitles lists sidecar subtitles next to videoName, for the formats in
// SidecarSubtitleExts, using go-media. siblingNames are the other file names in
// the same directory. legacy is the directory scan, run only in shadow mode.
func ExternalSubtitles(videoName string, siblingNames []string, legacy Legacy[[]utils.SubtitleTrack]) ([]utils.SubtitleTrack, bool) {
return timed("external subtitle list", videoName, legacy, subtitleListDiff, func() ([]utils.SubtitleTrack, bool) {
return externalSubtitlesFast(videoName, siblingNames)
})
}
func externalSubtitlesFast(videoName string, siblingNames []string) ([]utils.SubtitleTrack, bool) {
if !Enabled() {
return nil, false
}
tracks := media.ExternalSubtitleTracks(videoName, siblingNames)
out := make([]utils.SubtitleTrack, 0, len(tracks))
for _, t := range tracks {
if !SidecarSubtitleExts["."+t.Codec] {
continue
}
out = append(out, utils.SubtitleTrack{Name: t.Name, Language: t.Language, Codec: t.Codec})
}
return out, true
}
@@ -0,0 +1,38 @@
package gomedia
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
)
// TestAudioSidecarLyricsSetsFlag checks that a sidecar .lrc alone sets HasLyrics.
func TestAudioSidecarLyricsSetsFlag(t *testing.T) {
withGate(t, false)
mp3 := makeMP3(t)
require.NoError(t, os.WriteFile(filepath.Join(filepath.Dir(mp3), "tone.lrc"), []byte("[00:01.00]hello"), 0o644))
info, ok := Audio(mp3, false, nil)
require.True(t, ok)
require.True(t, info.HasLyrics)
text, format, ok := Lyrics(mp3, nil)
require.True(t, ok)
require.Equal(t, "lrc", format)
require.Contains(t, text, "hello")
}
// TestAudioNoLyricsNoFlag checks that a plain file reports no lyrics.
func TestAudioNoLyricsNoFlag(t *testing.T) {
withGate(t, false)
mp3 := makeMP3(t)
info, ok := Audio(mp3, false, nil)
require.True(t, ok)
require.False(t, info.HasLyrics)
_, _, ok = Lyrics(mp3, nil)
require.False(t, ok)
}
+277
View File
@@ -0,0 +1,277 @@
package gomedia
import (
"bytes"
"image/jpeg"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"github.com/gtsteffaniak/filebrowser/backend/pkg/settings"
"github.com/stretchr/testify/require"
)
// Fixtures shipped with the Playwright suite. Paths are relative to this package.
const (
fixtureDir = "../../../frontend/tests/playwright-files/myfolder/testdata"
fixtureMP4 = fixtureDir + "/sample-640x480.mp4"
fixtureWebM = fixtureDir + "/clip-a.webm"
)
// withGate sets the admin "Always use ffmpeg" setting for one test.
func withGate(t *testing.T, alwaysFfmpeg bool) {
t.Helper()
prev := settings.CurrentMediaSettings()
cur := prev
cur.AlwaysUseFfmpeg = alwaysFfmpeg
settings.ApplyMediaSettings(cur)
t.Cleanup(func() { settings.ApplyMediaSettings(prev) })
}
// requireTool skips the test when an ffmpeg binary is not on PATH.
func requireTool(t *testing.T, name string) {
t.Helper()
if _, err := exec.LookPath(name); err != nil {
t.Skipf("%s not installed", name)
}
}
// ffprobeValue returns one ffprobe value for path. args select the field.
func ffprobeValue(t *testing.T, path string, args ...string) string {
t.Helper()
requireTool(t, "ffprobe")
full := append([]string{"-v", "error"}, args...)
full = append(full, "-of", "default=nw=1:nk=1", path)
out, err := exec.Command("ffprobe", full...).Output()
require.NoError(t, err)
return strings.TrimSpace(string(out))
}
// ffprobeMedia returns the ffprobe duration, format name, and first video and
// audio codec names. One ffprobe call per field keeps the parsing simple.
func ffprobeMedia(t *testing.T, path string) (dur float64, format, video, audio string) {
t.Helper()
dur, _ = strconv.ParseFloat(ffprobeValue(t, path, "-show_entries", "format=duration"), 64)
format = ffprobeValue(t, path, "-show_entries", "format=format_name")
video = ffprobeValue(t, path, "-select_streams", "v:0", "-show_entries", "stream=codec_name")
audio = ffprobeValue(t, path, "-select_streams", "a:0", "-show_entries", "stream=codec_name")
return dur, format, video, audio
}
// makeMP3 generates a 2s MP3 with an ID3 title tag. The repo's MP3 fixture is
// empty, so ffmpeg makes a real one.
func makeMP3(t *testing.T) string {
t.Helper()
requireTool(t, "ffmpeg")
path := filepath.Join(t.TempDir(), "tone.mp3")
out, err := exec.Command("ffmpeg", "-v", "error", "-y",
"-f", "lavfi", "-i", "sine=frequency=440:duration=2",
"-metadata", "title=Gomedia Title",
"-c:a", "libmp3lame", path).CombinedOutput()
require.NoError(t, err, string(out))
return path
}
func TestEnabledFollowsAdminGate(t *testing.T) {
withGate(t, true)
require.False(t, Enabled(), "admin gate on must disable go-media")
withGate(t, false)
require.True(t, Enabled(), "default must keep go-media on")
}
func TestGateOffSkipsEveryPath(t *testing.T) {
withGate(t, true)
mp3 := makeMP3(t)
_, ok := Probe(fixtureMP4, nil)
require.False(t, ok, "Probe must skip go-media when the gate is on")
_, ok = Duration(fixtureMP4, nil)
require.False(t, ok, "Duration must skip go-media when the gate is on")
_, ok = Audio(mp3, true, nil)
require.False(t, ok, "Audio must skip go-media when the gate is on")
_, _, ok = Lyrics(mp3, nil)
require.False(t, ok, "Lyrics must skip go-media when the gate is on")
_, ok = VideoThumbnail(fixtureMP4, 256, 256, 72, 10, nil)
require.False(t, ok, "VideoThumbnail must skip go-media when the gate is on")
_, ok = EmbeddedSubtitles(fixtureMP4, time.Now(), nil)
require.False(t, ok, "EmbeddedSubtitles must skip go-media when the gate is on")
_, ok = ExtractSubtitle(fixtureMP4, 0, nil)
require.False(t, ok, "ExtractSubtitle must skip go-media when the gate is on")
}
func TestProbeMP4MatchesFFprobe(t *testing.T) {
withGate(t, false)
wantDur, wantFormat, wantVideo, wantAudio := ffprobeMedia(t, fixtureMP4)
got, ok := Probe(fixtureMP4, nil)
require.True(t, ok)
require.InDelta(t, wantDur, got.Duration, 0.1)
require.Equal(t, wantFormat, got.FormatName)
require.Equal(t, wantVideo, got.VideoCodec)
require.Equal(t, wantAudio, got.AudioCodec)
}
// TestProbeUsesCache checks that a populated cache entry is returned without
// re-reading the container.
func TestProbeUsesCache(t *testing.T) {
withGate(t, false)
path := filepath.Join(t.TempDir(), "sample.mp4")
data, err := os.ReadFile(fixtureMP4)
require.NoError(t, err)
require.NoError(t, os.WriteFile(path, data, 0o644))
st, err := os.Stat(path)
require.NoError(t, err)
key := probeCacheKey(path, st.ModTime().UnixNano(), st.Size())
sentinel := ProbeResult{Duration: 123, VideoCodec: "sentinel", AudioCodec: "sentinel", FormatName: "sentinel"}
probeCache.Set(key, sentinel)
t.Cleanup(func() { probeCache.Delete(key) })
got, ok := Probe(path, nil)
require.True(t, ok)
require.Equal(t, sentinel, got, "probe must return the cached value for an unchanged file")
}
// TestProbeCacheKeyTracksFileIdentity checks that path, modtime, and size all
// participate in the cache key, so an edited file is never served stale metadata.
func TestProbeCacheKeyTracksFileIdentity(t *testing.T) {
base := probeCacheKey("f.mp4", 100, 200)
require.Equal(t, base, probeCacheKey("f.mp4", 100, 200), "identical identity must hash the same")
require.NotEqual(t, base, probeCacheKey("f.mp4", 101, 200), "modtime change must change the key")
require.NotEqual(t, base, probeCacheKey("f.mp4", 100, 201), "size change must change the key")
require.NotEqual(t, base, probeCacheKey("g.mp4", 100, 200), "path change must change the key")
require.NotEqual(t, base, probeCacheKey("f.mp4", 100, 201), "key must not collide across fields")
}
func TestProbeWebMMatchesFFprobe(t *testing.T) {
withGate(t, false)
wantDur, _, wantVideo, wantAudio := ffprobeMedia(t, fixtureWebM)
got, ok := Probe(fixtureWebM, nil)
if !ok {
t.Skip("go-media does not handle this WebM file; ffmpeg fallback covers it")
}
// ffprobe reports "matroska,webm" for WebM; go-media's normalized name must match it.
require.Equal(t, "matroska,webm", got.FormatName)
require.InDelta(t, wantDur, got.Duration, 0.1)
require.Equal(t, wantVideo, got.VideoCodec)
require.Equal(t, wantAudio, got.AudioCodec)
}
func TestDurationMP3MatchesFFprobe(t *testing.T) {
withGate(t, false)
mp3 := makeMP3(t)
wantDur, _, _, _ := ffprobeMedia(t, mp3)
got, ok := Duration(mp3, nil)
if !ok {
t.Skip("go-media returned no duration for this MP3; ffmpeg fallback covers it")
}
// MP3 durations may be estimates; allow a looser tolerance.
require.InDelta(t, wantDur, got, 0.5)
}
func TestAudioTagsFromMP3(t *testing.T) {
withGate(t, false)
mp3 := makeMP3(t)
info, ok := Audio(mp3, true, nil)
require.True(t, ok)
require.Equal(t, "Gomedia Title", info.Tags.Title)
require.False(t, info.HasLyrics)
require.Empty(t, info.Artwork, "generated MP3 has no cover art")
}
func TestVideoThumbnailFitsBoxAndKeepsAspect(t *testing.T) {
withGate(t, false)
data, ok := VideoThumbnail(fixtureMP4, 256, 256, 72, 10, nil)
require.True(t, ok)
img, err := jpeg.Decode(bytes.NewReader(data))
require.NoError(t, err)
b := img.Bounds()
require.LessOrEqual(t, b.Dx(), 256)
require.LessOrEqual(t, b.Dy(), 256)
// 640x480 source is 4:3, so a 256 box gives 256x192. Allow 1px rounding.
require.InDelta(t, 256, b.Dx(), 1)
require.InDelta(t, 192, b.Dy(), 1)
}
func TestVideoThumbnailOriginalSizeIsNotScaled(t *testing.T) {
withGate(t, false)
// Zero bounds mean "original": the output must match the source size exactly.
data, ok := VideoThumbnail(fixtureMP4, 0, 0, 90, 10, nil)
require.True(t, ok)
img, err := jpeg.Decode(bytes.NewReader(data))
require.NoError(t, err)
require.Equal(t, 640, img.Bounds().Dx())
require.Equal(t, 480, img.Bounds().Dy())
}
func TestSubtitleListSkipsUnsupportedContainer(t *testing.T) {
withGate(t, false)
// MP3 has no subtitle streams in go-media. Report unsupported so the caller
// uses ffmpeg. Do not return an empty list, which would hide real subtitles.
_, ok := EmbeddedSubtitles(makeMP3(t), time.Now(), nil)
require.False(t, ok)
}
func TestGarbageFileFallsBack(t *testing.T) {
withGate(t, false)
path := filepath.Join(t.TempDir(), "not-media.mp4")
require.NoError(t, os.WriteFile(path, bytes.Repeat([]byte("x"), 4096), 0o600))
_, ok := Probe(path, nil)
require.False(t, ok)
_, ok = Duration(path, nil)
require.False(t, ok)
_, ok = VideoThumbnail(path, 256, 256, 72, 10, nil)
require.False(t, ok)
}
// TestEmbeddedSubtitleIndexMatchesFFprobe builds an MKV with a known subtitle
// track and checks that go-media's index is the one ffmpeg uses. A wrong index
// would extract the wrong track.
func TestEmbeddedSubtitleIndexMatchesFFprobe(t *testing.T) {
withGate(t, false)
requireTool(t, "ffmpeg")
dir := t.TempDir()
srt := filepath.Join(dir, "in.srt")
require.NoError(t, os.WriteFile(srt, []byte("1\n00:00:00,000 --> 00:00:01,000\nHello gomedia\n"), 0o600))
mkv := filepath.Join(dir, "subs.mkv")
out, err := exec.Command("ffmpeg", "-v", "error", "-y",
"-f", "lavfi", "-i", "color=c=black:s=64x64:d=1",
"-i", srt,
"-map", "0:v", "-map", "1:s",
"-c:v", "mpeg4", "-c:s", "srt", mkv).CombinedOutput()
require.NoError(t, err, string(out))
tracks, ok := EmbeddedSubtitles(mkv, time.Now(), nil)
require.True(t, ok, "go-media must list MKV subtitle tracks")
require.Len(t, tracks, 1)
require.NotNil(t, tracks[0].Index)
// ffprobe's subtitle stream index is the ground truth.
ffIndex := ffprobeValue(t, mkv, "-select_streams", "s:0", "-show_entries", "stream=index")
require.Equal(t, ffIndex, strconv.Itoa(*tracks[0].Index))
vtt, ok := ExtractSubtitle(mkv, *tracks[0].Index, nil)
require.True(t, ok)
require.True(t, strings.HasPrefix(strings.TrimSpace(vtt), "WEBVTT"), "output must be WebVTT")
require.Contains(t, vtt, "Hello gomedia")
}
Loaded 100 of 559 files, more files were not shown because too many files have changed in this diff. Show more