Every file the scripts write stays in .tools/: temporary files, downloads, Meteor, Node.js, npm.

build.sh and the releases/ scripts wrote all over the machine: temporary
files to /tmp and macOS's /var/folders, downloads and release zips to
~/Downloads or ~/Lataukset, Meteor to ~/.meteor, Node.js to /usr/local
(sudo n) or ~/.nvm, the npm cache to ~/.npm, Volta to ~/.volta, the
Sandstorm toolchain to ~/projects, and PATH lines into ~/.zshrc and
~/.bashrc. A sandboxed editor (Flatpak on Linux, sandbox-exec on macOS)
can reach only the checkout, and a container's own /tmp is a few hundred
megabytes. Everything is in .tools/ now, which git and Meteor ignore:

- releases/tools-env.sh, sourced by build.sh and every script that writes
  files, sets TMPDIR/TMP/TEMP to .tools/tmp (also downloads and release
  zips), NVM_DIR, npm_config_cache, VOLTA_HOME and the meteor-spk
  directory to .tools/, METEOR_WAREHOUSE_DIR to .tools/.meteor once Meteor
  is installed there, and puts those tools first on PATH, so no shell rc
  file is edited. HOME is not moved: git needs ~/.gitconfig and ~/.ssh.
  A TMPDIR already inside a .tools/ is kept, so a test's own directory
  there stays its own; without a writable one it stops rather than fall
  back to /tmp. It never fails a caller running set -euo pipefail.
- macOS's mktemp ignores TMPDIR when given no template, so every mktemp
  names one: mktemp "$TMPDIR/name.XXXXXX". Fixed /tmp files are in
  $TMPDIR.
- build.sh installs Node.js on Linux into .tools/n (N_PREFIX, no sudo),
  nvm on macOS into .tools/nvm (PROFILE=/dev/null: no rc file edited), and
  Meteor with HOME=.tools for the installer only, into .tools/.meteor.
  build.sh is replaced, not edited in place, while it may be running.
- Containers get this checkout's .tools/tmp as their /tmp: every
  db-conformance database (MariaDB's temporary tables were verified to land
  there, and MongoDB's socket) and the repack container, which empty it
  themselves before they are removed. install-node-for-arch.sh requires
  the mounted TMPDIR.
- tests/run-node-suites.cjs gives every suite TMPDIR in .tools/tmp, and the
  tests that copied a script into a fixture copy tools-env.sh too; tests
  that pinned ~/.meteor, the rc file edits, sudo n or /tmp/manifest.json
  pin the .tools/ places now and say why.

tests/toolsDirOnly.test.cjs runs tools-env.sh against a TMPDIR already
set elsewhere and under set -euo pipefail, and fails on any script that
makes a temporary file without naming .tools/tmp, writes to the home
directory, or edits a shell rc file, or on a container whose /tmp is its
own.

Thanks to xet7 !
This commit is contained in:
Lauri Ojansivu committed 2026-10-27 23:49:55 +02:00
1 parent d1b96f65c3
commit aeff4900df
54 files changed
+475 -164

No files matched your search

+33 -30
View File
@@ -103,6 +103,10 @@ WEKAN_TOOLS_DIR="$WEKAN_DIR/.tools"
export WEKAN_DIR WEKAN_TOOLS_DIR
export PATH="$WEKAN_TOOLS_DIR/bin:${GOBIN:+$GOBIN:}$PATH"
. "$WEKAN_DIR/releases/ensure-tools.sh"
# Everything this script and the releases/ scripts write - temporary files,
# downloads, Meteor, Node.js, the npm cache - stays in .tools/, on Linux, macOS
# and Windows alike (releases/tools-env.sh).
. "$WEKAN_DIR/releases/tools-env.sh"
# Prefer the repository-local toolchain installed under .tools. A fresh shell
# does not normally have .tools/.meteor on PATH, which made unattended test and
@@ -1216,7 +1220,7 @@ function run_all_tests(){
PW_FAILURES=""
TEST_SERVER_PID=""
MONGOD_PID=""
STATDIR="$(mktemp -d)"
STATDIR="$(mktemp -d "$TMPDIR/wekan-stat.XXXXXX")"
BPIDS=""
# STOP WHAT THIS RUN STARTED, on EVERY way out - not only the happy path.
@@ -2972,15 +2976,20 @@ for _once in 1; do
;;
*) echo "Unsupported Linux distribution; install a C/C++ toolchain, git, curl, wget, 7zip, zip, unzip, file and npm." >&2; exit 1 ;;
esac
sudo npm -g install n
sudo n "$_wekan_node_version"
sudo npm install -g "npm@${_wekan_npm_version:-latest}"
# Node.js into .tools/n with n's N_PREFIX, as this user: not into
# /usr/local with sudo. releases/tools-env.sh puts .tools/n/bin on
# PATH, and npm's global installs go to that same prefix.
N_PREFIX="$WEKAN_TOOLS_DIR/n" npx -y n "$_wekan_node_version"
export PATH="$WEKAN_TOOLS_DIR/n/bin:$PATH"
hash -r
npm install -g "npm@${_wekan_npm_version:-latest}"
hash -r
# Meteor the way https://docs.meteor.com/about/install.html installs
# it on Linux, macOS and Windows alike: `npx meteor`, as this user and
# never with sudo. The meteor npm package is only an installer; npx
# runs its cli.js, which installs into ~/.meteor and adds that to
# PATH in the shell rc files. `sudo npm -g install meteor` ran the
# runs its cli.js, which installs into $HOME/.meteor and adds that to
# PATH in the shell rc files - so HOME is .tools for the installer
# only, and Meteor and those rc lines land in .tools/. `sudo npm -g install meteor` ran the
# same installer as an install script instead, which npm 12 refuses
# to run (and which also refused --unsafe-perm and the documented
# --foreground-script).
@@ -2988,16 +2997,14 @@ for _once in 1; do
# The global meteor package such a setup left behind is only that
# installer, never Meteor itself (~/.meteor is not touched).
sudo npm -g uninstall meteor >/dev/null 2>&1 || true
# npx caches in ~/.npm, which an old `sudo npm` could leave root-owned.
if [ -d "$HOME/.npm" ] && [ ! -w "$HOME/.npm" ]; then
sudo chown -R "$(id -u):$(id -g)" "$HOME/.npm"
fi
if [ -x "$HOME/.meteor/meteor" ]; then
echo "Meteor is already installed in $HOME/.meteor"
# npx caches in npm_config_cache (.tools/npm-cache).
mkdir -p "$npm_config_cache"
if [ -x "$WEKAN_TOOLS_DIR/.meteor/meteor" ]; then
echo "Meteor is already installed in $WEKAN_TOOLS_DIR/.meteor"
else
npx -y meteor
HOME="$WEKAN_TOOLS_DIR" npx -y meteor
fi
export PATH="$HOME/.meteor:$PATH"
. "$WEKAN_DIR/releases/tools-env.sh"
elif [[ "$OSTYPE" == "darwin"* ]]; then
echo "macOS"
# Node comes from nvm, not from Homebrew's node@26 keg.
@@ -3009,13 +3016,14 @@ for _once in 1; do
# puts that on PATH itself, so this does not go stale the way a
# pinned version does. npm comes with the Node it installs, so
# there is no `brew install npm` either.
export NVM_DIR="${NVM_DIR:-$HOME/.nvm}"
# NVM_DIR is .tools/nvm (releases/tools-env.sh).
mkdir -p "$NVM_DIR"
if [ ! -s "$NVM_DIR/nvm.sh" ]; then
echo "Installing nvm into $NVM_DIR ..."
# A PINNED tag, not master: this pipes a downloaded script into
# a shell, so it must be a revision somebody has looked at.
curl -fsSL https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.6/install.sh | bash
# PROFILE=/dev/null: the installer edits no shell rc file.
curl -fsSL https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.6/install.sh | PROFILE=/dev/null bash
fi
# nvm is a shell FUNCTION, not a binary, so it has to be sourced -
# `command -v nvm` finds nothing until this line has run.
@@ -3037,23 +3045,18 @@ for _once in 1; do
nvm use 26
npm install -g "npm@${_wekan_npm_version:-latest}"
echo "Node $(node --version), npm $(npm --version)"
# Let new shells find nvm too. Its installer appends these itself,
# but only to the rc file it detects and only when IT did the
# install - so a machine that already had nvm keeps working.
touch "$HOME/.zshrc"
for rc in "$HOME/.zshrc" "$HOME/.bashrc"; do
[ -e "$rc" ] || continue
grep -qxF 'export NVM_DIR="$HOME/.nvm"' "$rc" || echo 'export NVM_DIR="$HOME/.nvm"' >> "$rc"
grep -qxF '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' "$rc" || echo '[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"' >> "$rc"
done
# No shell rc file is edited: build.sh and the releases/ scripts put
# .tools/nvm's Node.js and .tools/.meteor on PATH themselves
# (releases/tools-env.sh).
# Meteor as https://docs.meteor.com/about/install.html installs it:
# `npx meteor`, as this user, never sudo (see the Linux branch).
if [ -x "$HOME/.meteor/meteor" ]; then
echo "Meteor is already installed in $HOME/.meteor"
# `npx meteor`, as this user, never sudo, with HOME=.tools for the
# installer only (see the Linux branch).
if [ -x "$WEKAN_TOOLS_DIR/.meteor/meteor" ]; then
echo "Meteor is already installed in $WEKAN_TOOLS_DIR/.meteor"
else
npx -y meteor
HOME="$WEKAN_TOOLS_DIR" npx -y meteor
fi
export PATH="$HOME/.meteor:$PATH"
. "$WEKAN_DIR/releases/tools-env.sh"
exit;
elif [[ "$OSTYPE" == "cygwin" ]]; then
# POSIX compatibility layer and Linux environment emulation for Windows
+4 -3
View File
@@ -46,6 +46,7 @@
# defaults to .tools/charts in this repo.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
APPLY=false
PUSH=false
@@ -97,7 +98,7 @@ fi
# its answer can change the outcome.
mkdir -p "$REPO_DIR/.tools/tmp"
export TMPDIR="$REPO_DIR/.tools/tmp"
PLAN_JSON="$(mktemp)"
PLAN_JSON="$(mktemp "$TMPDIR/wekan.XXXXXX")"
trap 'rm -f "$PLAN_JSON"' EXIT
CHART_BASE_URL="$CHART_BASE_URL" LIMIT="$LIMIT" python3 - "$PLAN_JSON" <<'PYEOF'
@@ -265,7 +266,7 @@ cd "$CHARTS_DIR"
# are needed, so the source is copied out before the branch is switched.
git checkout main
git pull --ff-only || true
SRC="$(mktemp -d)"
SRC="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
trap 'rm -f "$PLAN_JSON"; rm -rf "$SRC"' EXIT
cp -a wekan "$SRC/wekan"
@@ -284,7 +285,7 @@ for VERSION in $(jq -r '.build[]' "$PLAN_JSON"); do
if [ -f "wekan-${CHART_VERSION}.tgz" ]; then
continue
fi
work="$(mktemp -d)"
work="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
cp -a "$SRC/wekan" "$work/wekan"
sedi -E "s|^appVersion: \"[^\"]*\"|appVersion: \"${VERSION}\"|" "$work/wekan/Chart.yaml"
sedi -E "s|^version: [0-9]+\.[0-9]+\.[0-9]+|version: ${CHART_VERSION}|" "$work/wekan/Chart.yaml"
+2 -1
View File
@@ -5,9 +5,10 @@
#
# bash releases/bsd/smoke.sh <bundle dir> [port]
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/../tools-env.sh" # temporary files and downloads go to .tools/tmp
bundle="$(cd "${1:?usage: smoke.sh <bundle dir> [port]}" && pwd)"
port="${2:-3123}"
work="$(mktemp -d "${TMPDIR:-/tmp}/wekan-smoke.XXXXXX")"
work="$(mktemp -d "$TMPDIR/wekan-smoke.XXXXXX")"
log="$work/start-wekan.log"
(
cd "$bundle"
+4 -3
View File
@@ -1,5 +1,6 @@
#!/bin/bash
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ $# -ne 1 ]
then
@@ -9,9 +10,9 @@ if [ $# -ne 1 ]
fi
# Lataukset is the Finnish Linux name of Downloads; macOS keeps ~/Downloads
# in every language (Finder only shows it translated).
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$([ -d "$HOME/Lataukset" ] && echo "$HOME/Lataukset" || echo "$HOME/Downloads")}"
# Downloads and release zips stay in .tools/tmp (releases/tools-env.sh), not
# in ~/Downloads or ~/Lataukset.
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$WEKAN_DOWNLOADS}"
git pull
cd .build
+1
View File
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/../tools-env.sh" # temporary files and downloads go to .tools/tmp
root=$(pwd)
export TMPDIR="$root/.tools/tmp"
mkdir -p "$TMPDIR"
+1
View File
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
# Temporary opt-in port of meteor/meteor#14751; normal builds never use it.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/../tools-env.sh" # temporary files and downloads go to .tools/tmp
root=$(pwd)
export TMPDIR="$root/.tools/tmp"
mkdir -p "$TMPDIR"
+2 -1
View File
@@ -27,6 +27,7 @@
# the question the two crashes above were.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
BUNDLE="${1:?usage: bundle-smoke-boot.sh <bundle-dir> [node-binary]}"
NODE_BIN="${2:-node}"
@@ -73,7 +74,7 @@ run_with_timeout() {
# Verify the architecture-independent engine and WASM/database asset before boot.
run_with_timeout "$TIMEOUT" "$NODE_BIN" "$(dirname "$0")/verify-mime-runtime.cjs" "$BUNDLE" || exit 1
log="$(mktemp)"
log="$(mktemp "$TMPDIR/wekan.XXXXXX")"
trap 'rm -f "$log"' EXIT
echo "--- smoke: starting the bundle with a database that cannot answer"
+5 -4
View File
@@ -42,6 +42,7 @@
# is a warning.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
arch="${1:?arch is required}"
node_arch="${2:?node_arch is required}"
@@ -110,10 +111,10 @@ if [ -n "$image" ] && [ -n "$platform" ]; then
# manifest. The bounded loop still fails closed when the registry remains
# unavailable.
manifest_ok=false
manifest_tmp="/tmp/manifest.$$.json"
manifest_tmp="$TMPDIR/manifest.$$.json"
for attempt in 1 2 3 4 5; do
if docker manifest inspect "$image" >"$manifest_tmp" 2>/dev/null; then
mv "$manifest_tmp" /tmp/manifest.json
mv "$manifest_tmp" "$TMPDIR/manifest.json"
manifest_ok=true
break
fi
@@ -121,13 +122,13 @@ if [ -n "$image" ] && [ -n "$platform" ]; then
done
rm -f "$manifest_tmp"
if [ "$manifest_ok" = true ]; then
if ! WANT_ARCH="$want_arch" WANT_VARIANT="$want_variant" python3 - <<'PYEOF'
if ! MANIFEST="$TMPDIR/manifest.json" WANT_ARCH="$want_arch" WANT_VARIANT="$want_variant" python3 - <<'PYEOF'
import json, os, sys
want_arch = os.environ["WANT_ARCH"]
want_variant = os.environ["WANT_VARIANT"]
with open("/tmp/manifest.json", encoding="utf-8") as fh:
with open(os.environ["MANIFEST"], encoding="utf-8") as fh:
doc = json.load(fh)
# A single-platform image has no "manifests" list; its own config names the
+2 -1
View File
@@ -89,6 +89,7 @@ if [ -n "${ZSH_VERSION:-}" ]; then exec /bin/bash "$0" "$@"; fi
# ─────────────────────────────────────────────────────────────────────────────
set -u
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
REPO_DIR="$(cd "$(dirname "$0")/.." && pwd)"
TIMESTAMP="$(date +%Y-%m-%d_%H-%M-%S)"
@@ -189,7 +190,7 @@ create_snap() {
echo " snapcraft login failed; skipping SNAP_AUTH."
return
fi
local tmp; tmp="$(mktemp)"
local tmp; tmp="$(mktemp "$TMPDIR/wekan.XXXXXX")"
if snapcraft export-login --snaps=wekan \
--acls=package_access,package_push,package_update,package_release \
--channels=edge,beta "$tmp"; then
+34 -14
View File
@@ -54,6 +54,7 @@
# speed test, and docs/Databases/FerretDB/1/SpeedTest/speedtest.sh, set it.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
cd "$(dirname "$0")/.." || exit 1
WEKAN_DIR="$(pwd)"
@@ -335,22 +336,41 @@ fi
# stopped or reused by this.
CONTAINER="wekan-conformance-db-$RUN_TS"
# A container's own /tmp is a few hundred megabytes of its writable layer, and
# a database writes its temporary files there (MySQL's and MariaDB's on-disk
# temporary tables, sort files). Each container's /tmp is this run's own
# directory under .tools/tmp instead, mode 1777 like /tmp, so the database's
# own user can write it. The container empties it before it is removed: on
# Linux the files belong to that user, whom the host cannot remove them as.
container_tmp() {
local dir="$WEKAN_DIR/.tools/tmp/$PREFIX-$1-tmp"
mkdir -p "$dir" && chmod 1777 "$dir"
printf '%s\n' "$dir"
}
CONTAINER_TMP="$(container_tmp db)"
remove_container() {
docker_exec exec "$1" sh -c 'rm -rf /tmp/* /tmp/.[!.]* 2>/dev/null' >/dev/null 2>&1
docker_exec rm -fv "$1" >/dev/null 2>&1
}
echo "Ports: FerretDB on 127.0.0.1:$FERRET_PORT (nothing else is using it)"
echo
cleanup() {
[ -n "${FERRET_PID:-}" ] && kill "$FERRET_PID" 2>/dev/null
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
remove_ferretdb2
remove_sqlite_data
rmdir "$CONTAINER_TMP" "$FERRET2_TMP" 2>/dev/null
}
# FerretDB 2 runs as a container of its own beside its database, on a network
# of this run's own: wekan-conformance-db-<run>-ferretdb, -net.
FERRET2_CONTAINER="$CONTAINER-ferretdb"
FERRET2_NETWORK="$CONTAINER-net"
FERRET2_TMP="$(container_tmp ferretdb2)"
remove_ferretdb2() {
docker_exec rm -fv "$FERRET2_CONTAINER" >/dev/null 2>&1
remove_container "$FERRET2_CONTAINER"
docker_exec network rm "$FERRET2_NETWORK" >/dev/null 2>&1
}
@@ -506,7 +526,7 @@ for entry in "${BACKENDS[@]}"; do
echo
echo "---- $name: starting the database ----"
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
# Start the container, and when the published port turns out to be taken after
# all, move to the next free one and try again. `free_port` looks a moment
@@ -516,7 +536,7 @@ for entry in "${BACKENDS[@]}"; do
start_db_container() {
local attempt
for attempt in 1 2 3 4 5; do
if docker_exec run -d --label "$OWNER_LABEL" --name "$CONTAINER" -p "127.0.0.1:$hostport:$port" "$@" >>"$log" 2>&1; then
if docker_exec run -d --label "$OWNER_LABEL" --name "$CONTAINER" -v "$CONTAINER_TMP:/tmp" -p "127.0.0.1:$hostport:$port" "$@" >>"$log" 2>&1; then
return 0
fi
@@ -524,7 +544,7 @@ for entry in "${BACKENDS[@]}"; do
return 1 # a real failure: report it as one
fi
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
hostport=$((hostport + 1))
hostport="$(free_port "$hostport")"
echo " port was taken; retrying on 127.0.0.1:$hostport" | tee -a "$log"
@@ -632,7 +652,7 @@ for entry in "${BACKENDS[@]}"; do
echo "SKIP $name: MongoDB refuses to start on this Linux kernel ($kernel), 6.19-7.0.13"
echo "SKIP $name MongoDB refuses Linux kernel $kernel (6.19-7.0.13; upgrade the kernel)" >> "$SUMMARY"
docker_exec logs "$CONTAINER" >>"$log" 2>&1
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
skipped=$((skipped + 1))
continue
fi
@@ -641,7 +661,7 @@ for entry in "${BACKENDS[@]}"; do
docker_exec logs "$CONTAINER" >>"$log" 2>&1
echo "ERROR $name database never ready" >> "$SUMMARY"
stop_on_failure
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
continue
fi
fi
@@ -650,12 +670,12 @@ for entry in "${BACKENDS[@]}"; do
# port this run's FerretDB would use. Then it is asked like MongoDB is.
if [ "$handler" = ferretdb2 ]; then
if ! docker_exec run -d --label "$OWNER_LABEL" --name "$FERRET2_CONTAINER" --network "$FERRET2_NETWORK" \
-p "127.0.0.1:$FERRET_PORT:27017" \
-v "$FERRET2_TMP:/tmp" -p "127.0.0.1:$FERRET_PORT:27017" \
-e "FERRETDB_POSTGRESQL_URL=postgres://ferretdb:ferretdb_secret@$CONTAINER:5432/postgres" \
-e FERRETDB_TELEMETRY=disable "$FERRET2_IMAGE" >>"$log" 2>&1; then
echo "ERROR $name FerretDB 2 container did not start" >> "$SUMMARY"
stop_on_failure
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1; remove_ferretdb2
remove_container "$CONTAINER"; remove_ferretdb2
continue
fi
echo -n "waiting for FerretDB 2 "
@@ -675,7 +695,7 @@ for entry in "${BACKENDS[@]}"; do
docker_exec logs "$FERRET2_CONTAINER" >>"$log" 2>&1
echo "ERROR $name FerretDB 2 did not start" >> "$SUMMARY"
stop_on_failure
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1; remove_ferretdb2
remove_container "$CONTAINER"; remove_ferretdb2
continue
fi
engine="$(engine_name)"
@@ -691,7 +711,7 @@ for entry in "${BACKENDS[@]}"; do
echo "---- $name: stopping ----"
docker_exec logs "$FERRET2_CONTAINER" >>"$log" 2>&1
docker_exec logs "$CONTAINER" >>"$log" 2>&1
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
remove_ferretdb2
echo
continue
@@ -711,7 +731,7 @@ for entry in "${BACKENDS[@]}"; do
fi
echo "---- $name: stopping ----"
docker_exec logs "$CONTAINER" >>"$log" 2>&1
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
echo
if [ "${WEKAN_TEST_BAIL:-0}" = 1 ]; then
compare_suite 2>&1 | tee -a "$SUMMARY"
@@ -761,7 +781,7 @@ for entry in "${BACKENDS[@]}"; do
echo "ERROR $name FerretDB did not start on this backend" >> "$SUMMARY"
stop_on_failure
kill "$FERRET_PID" 2>/dev/null; FERRET_PID=""
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
continue
fi
@@ -781,7 +801,7 @@ for entry in "${BACKENDS[@]}"; do
remove_sqlite_data
if [ -n "$service" ]; then
docker_exec logs "$CONTAINER" >>"$log" 2>&1
docker_exec rm -fv "$CONTAINER" >/dev/null 2>&1
remove_container "$CONTAINER"
fi
echo
# Compare each completed backend before starting the next one in bail mode.
+4 -6
View File
@@ -8,11 +8,9 @@ WEKAN_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
FERRET_DIR="$WEKAN_DIR/.tools/FerretDB"
METEOR_BIN="${METEOR_BIN:-}"
if [ -z "$METEOR_BIN" ]; then
if [ -x "$HOME/.meteor/meteor" ]; then
METEOR_BIN="$HOME/.meteor/meteor"
elif [ -x "$WEKAN_DIR/.tools/.meteor/meteor" ]; then
# The repository sandbox instructions keep HOME-owned toolchains isolated
# here. It is the same Meteor distribution, including its development MongoDB.
# Meteor lives in .tools/.meteor (releases/tools-env.sh); an older install
# still in ~/.meteor is used until build.sh's install-dependencies moves it.
if [ -x "$WEKAN_DIR/.tools/.meteor/meteor" ]; then
METEOR_BIN="$WEKAN_DIR/.tools/.meteor/meteor"
else
METEOR_BIN="$HOME/.meteor/meteor"
@@ -37,7 +35,7 @@ fi
if [ ! -x "$METEOR_BIN" ]; then
echo "Meteor executable not found at $METEOR_BIN" >&2
echo 'Install Meteor under ~/.meteor or set METEOR_BIN.' >&2
echo 'Install Meteor with build.sh (Install dependencies), into .tools/.meteor, or set METEOR_BIN.' >&2
exit 1
fi
+2 -1
View File
@@ -39,6 +39,7 @@
# and 1 when something went wrong.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
dest="${1:?dest path is required}"
platform="${2:?platform is required (e.g. amd64, win64, mac-arm64)}"
@@ -65,7 +66,7 @@ if [ "$node_found" != "true" ] || [ -z "$node_url" ] || [ -z "$node_sha256" ]; t
fi
mkdir -p "$(dirname "$dest")"
work="$(mktemp -d)"
work="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
trap 'rm -rf "$work"' EXIT
# ── 2. Download it ────────────────────────────────────────────────────────────
+1
View File
@@ -22,6 +22,7 @@
# The notes go file to file, never through a shell variable or argv: see
# releases/release-notes.sh for the two ways that broke a release.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ "$#" -ne 2 ]; then
echo "Usage: $0 <repository> <version>" >&2
exit 2
+1
View File
@@ -1,4 +1,5 @@
#!/bin/bash
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
#
# Shared helper: install required CLI tools for the current platform if missing.
# Supports Debian/Ubuntu, Fedora, RHEL/Oracle Linux, Alpine, Arch, and macOS.
+3 -2
View File
@@ -60,6 +60,7 @@
# step and its embed step agree and only one of them asks.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_SLUG="wekan/FerretDB"
@@ -100,7 +101,7 @@ full_asset="ferretdb-${platform}${ext}"
valid_tag() { case "$1" in v[0-9]*) case "$1" in *[!A-Za-z0-9._+-]*) return 1 ;; esac; return 0 ;; *) return 1 ;; esac; }
cache_dir="${FERRETDB_ASSET_CACHE:-${RUNNER_TEMP:-${TMPDIR:-/tmp}}}"
cache_dir="${FERRETDB_ASSET_CACHE:-${RUNNER_TEMP:-$TMPDIR}}"
cache="$cache_dir/ferretdb-asset-${variant}-${platform}${FERRETDB_TAG:+-$FERRETDB_TAG}"
# url_for <tag> <asset>: that release's asset, or /latest/ when the tag is not
@@ -243,7 +244,7 @@ if [ "$mode" = download ]; then
exit 3
fi
chmod +x "$out" 2>/dev/null || true
sumfile="$(mktemp "${TMPDIR:-/tmp}/ferretdb-sha.XXXXXX")"
sumfile="$(mktemp "$TMPDIR/ferretdb-sha.XXXXXX")"
if bash "$here/fetch.sh" --optional -o "$sumfile" "${FERRET_URL}.sha256sum"; then
want="$(awk '{print $1; exit}' "$sumfile")"
got="$(sha_of "$out")"
+2 -1
View File
@@ -25,8 +25,9 @@
# one request rather than six.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
cache="${FERRETDB_TAG_CACHE:-${RUNNER_TEMP:-${TMPDIR:-/tmp}}/ferretdb-latest-tag}"
cache="${FERRETDB_TAG_CACHE:-${RUNNER_TEMP:-$TMPDIR}/ferretdb-latest-tag}"
if [ -s "$cache" ]; then
cat "$cache"
+2 -1
View File
@@ -38,6 +38,7 @@
# Tunables (env): FETCH_ATTEMPTS (default 7), FETCH_SLEEPS (default
# "15 30 60 120 240 480"; set to "0" in tests), FETCH_CONNECT_TIMEOUT (30).
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
attempts=${FETCH_ATTEMPTS:-7}
sleeps=${FETCH_SLEEPS:-"15 30 60 120 240 480"}
@@ -67,7 +68,7 @@ if [ -z "$url" ] || { [ "$mode" = download ] && [ -z "$out" ]; }; then
exit 2
fi
tmp=$(mktemp)
tmp=$(mktemp "$TMPDIR/wekan.XXXXXX")
trap 'rm -f "$tmp"' EXIT
attempt=1
+3 -2
View File
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# ── Repoint stale commit links in the CHANGELOG ──────────────────────────────
#
# Every CHANGELOG bullet links the commit it describes. Those links are written
@@ -65,7 +66,7 @@ CHANGELOG_FILE="${ARG_FILE:-${CHANGELOG_FILE:-CHANGELOG.md}}"
# The set is listed ONCE (`git rev-list --all` walks the history once; asking
# `for-each-ref --contains` per link would walk it again for every link, which
# --all-sections turns into thousands of walks).
REACHABLE_LIST="$(mktemp)"
REACHABLE_LIST="$(mktemp "$TMPDIR/wekan.XXXXXX")"
trap 'rm -f "$REACHABLE_LIST"' EXIT
git rev-list --all 2>/dev/null | sort > "$REACHABLE_LIST"
@@ -171,7 +172,7 @@ fix_commit_hashes() {
fixed=$((fixed + 1))
continue
fi
_tmp="$(mktemp)"
_tmp="$(mktemp "$TMPDIR/wekan.XXXXXX")"
awk -v s="$start" -v e="$end" -v old="$h" -v repl="$short" '
NR>=s && NR<=e { gsub("commit/" old, "commit/" repl) } { print }
' "$CHANGELOG_FILE" > "$_tmp" && mv "$_tmp" "$CHANGELOG_FILE"
+1
View File
@@ -14,6 +14,7 @@
# `timeout` on PATH can be System32's "wait N seconds" command, which takes
# different arguments.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ "$#" -lt 3 ]; then
echo "Usage: $0 <repository> <tag> <artifact> [artifact ...]" >&2
exit 2
+16 -11
View File
@@ -35,6 +35,9 @@
# the newest release: a CPU can lag a version or two behind.
set -eux
# The caller (repack-bundle-for-arch.sh) mounts this repository's .tools/tmp as
# the container's /tmp and sets TMPDIR: the container's own /tmp is tiny.
: "${TMPDIR:?TMPDIR must be the mounted .tools/tmp}"
# Older macOS has no sha256sum; shasum -a 256 prints the same "hash file" lines and takes -c.
command -v sha256sum >/dev/null 2>&1 || sha256sum() { shasum -a 256 "$@"; }
@@ -118,22 +121,22 @@ download_and_verify() {
# distribution, node-patches publishes the BARE binary. NODE_KIND says which and
# NODE_MEMBER names the node inside an archive - both resolved on the host, so
# nothing is guessed from the URL here.
download_and_verify "$NODE_URL" /tmp/node-download
download_and_verify "$NODE_URL" "$TMPDIR/node-download"
case "${NODE_KIND:-binary}" in
binary)
cp /tmp/node-download /opt/node/bin/node
cp "$TMPDIR/node-download" /opt/node/bin/node
;;
tar.xz|tar.gz|tar)
mkdir -p /tmp/nodedist
mkdir -p "$TMPDIR/nodedist"
# -xf and let tar detect the compression: unofficial-builds publishes
# .tar.gz beside .tar.xz and either may be what was resolved.
if ! tar -xf /tmp/node-download -C /tmp/nodedist "${NODE_MEMBER}"; then
if ! tar -xf "$TMPDIR/node-download" -C "$TMPDIR/nodedist" "${NODE_MEMBER}"; then
echo "install-node-for-arch.sh: ${NODE_MEMBER} is not in the archive" \
"from ${NODE_FROM} (${NODE_URL})." >&2
exit 1
fi
cp "/tmp/nodedist/${NODE_MEMBER}" /opt/node/bin/node
cp "$TMPDIR/nodedist/${NODE_MEMBER}" /opt/node/bin/node
;;
*)
echo "install-node-for-arch.sh: unknown NODE_KIND '${NODE_KIND}'. The" \
@@ -143,16 +146,18 @@ case "${NODE_KIND:-binary}" in
;;
esac
chmod +x /opt/node/bin/node
rm -f /tmp/node-download
rm -f "$TMPDIR/node-download"
# npm from the official amd64 tarball of the same version - JavaScript, so it runs
# on this arch's node just as well. The tarball is a build-time tool, not what
# WeKan ships; the shipped node is the target CPU's, above.
bash "$(dirname "$0")/fetch.sh" -o /tmp/npm.tar.xz \
bash "$(dirname "$0")/fetch.sh" -o "$TMPDIR/npm.tar.xz" \
"https://nodejs.org/dist/${NODE_FULL}/node-${NODE_FULL}-linux-x64.tar.xz"
mkdir -p /tmp/npm
tar -xJf /tmp/npm.tar.xz -C /tmp/npm --strip-components=1
cp -a /tmp/npm/lib /opt/node/
cp -a /tmp/npm/bin/npm /tmp/npm/bin/npx /opt/node/bin/
mkdir -p "$TMPDIR/npm"
tar -xJf "$TMPDIR/npm.tar.xz" -C "$TMPDIR/npm" --strip-components=1
cp -a "$TMPDIR/npm"/lib /opt/node/
cp -a "$TMPDIR/npm"/bin/npm "$TMPDIR/npm"/bin/npx /opt/node/bin/
# Removed here, in the container: on Linux the host could not, they are root's.
rm -rf "$TMPDIR/nodedist" "$TMPDIR/npm" "$TMPDIR/npm.tar.xz"
export PATH=/opt/node/bin:$PATH
+5 -3
View File
@@ -1,5 +1,6 @@
#!/bin/bash
if [ -n "${ZSH_VERSION:-}" ]; then exec /bin/bash "$0" "$@"; fi
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
echo "INSTALLING WEKAN SANDSTORM VERSION RELATED FILES:"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -8,12 +9,13 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
. "$SCRIPT_DIR/ensure-tools.sh"
ensure_archive_tools
cd ~
cd "$WEKAN_TOOLS_DIR"
wget https://releases.wekan.team/dev/meteor-spk/projects.7z
7z x projects.7z
rm projects.7z
echo "export PATH=\$PATH:~/projects/meteor-spk/meteor-spk-0.6.0" >> ~/.bashrc
source ~/.bashrc
# meteor-spk is in .tools/projects/meteor-spk now: on PATH for this script,
# and found there by sandstorm-slim-spk.sh. No shell rc file is edited.
export PATH="$PATH:$WEKAN_METEOR_SPK_DIR/meteor-spk-0.6.0"
echo "INSTALL DEV VERSION OF SANDSTORM:"
curl https://install.sandstorm.io | bash
cd "$REPO_DIR"
+2 -1
View File
@@ -14,6 +14,7 @@
# Never packed: .git, .tools, .env, node_modules. COPYFILE_DISABLE keeps
# macOS's tar from adding ._* AppleDouble files for extended attributes.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ "$#" -ne 3 ]; then
echo "Usage: $0 <source dir> <app id> <archive.tar.gz>" >&2
@@ -36,7 +37,7 @@ case "$archive" in
esac
export COPYFILE_DISABLE=1
stage=$(mktemp -d "${TMPDIR:-/tmp}/nextcloud-pack.XXXXXX")
stage=$(mktemp -d "$TMPDIR/nextcloud-pack.XXXXXX")
trap 'rm -rf -- "$stage"' EXIT
mkdir "$stage/$app_id"
tar --exclude=.git --exclude=.tools --exclude=.env --exclude=node_modules \
+6 -5
View File
@@ -1,5 +1,6 @@
#!/bin/bash
if [ -n "${ZSH_VERSION:-}" ]; then exec /bin/bash "$0" "$@"; fi
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# Update node version.
@@ -24,9 +25,9 @@ else
ensure_tools curl
if ! command -v volta >/dev/null 2>&1; then
echo "volta not found. Installing Volta with its official installer..."
curl -fsSL https://get.volta.sh | bash
curl -fsSL https://get.volta.sh | bash -s -- --skip-setup
fi
export VOLTA_HOME="${VOLTA_HOME:-$HOME/.volta}"
export VOLTA_HOME="${VOLTA_HOME:-$WEKAN_TOOLS_DIR/volta}"
export PATH="$VOLTA_HOME/bin:$PATH"
fi
if [ $NODE_VERSION != v$2 ]; then
@@ -37,7 +38,7 @@ else
fi
# If Sandstorm node does not exist
SANDSTORM_NODE=~/projects/meteor-spk/meteor-spk-0.5.1/meteor-spk.deps/bin/node
SANDSTORM_NODE="$WEKAN_METEOR_SPK_DIR/meteor-spk-0.5.1/meteor-spk.deps/bin/node"
#if [[ -f "$SANDSTORM_NODE" ]]; then
# echo "9) Installing local Sandstorm develoment version"
@@ -47,12 +48,12 @@ SANDSTORM_NODE=~/projects/meteor-spk/meteor-spk-0.5.1/meteor-spk.deps/bin/node
#fi
SANDSTORM_NODE_VERSION=$($SANDSTORM_NODE -v)
PROJECTS_ARCHIVE=~/projects.7z
PROJECTS_ARCHIVE="$WEKAN_TOOLS_DIR/projects.7z"
# If installed Sandstorm node is not newest version
if [ $SANDSTORM_NODE != v$2 ]; then
echo "9) Copy previously updated local node to Sandstorm node"
cp ~/.volta/bin/node ~/projects/meteor-spk/meteor-spk-0.5.1/meteor-spk.deps/bin/
cp "$VOLTA_HOME/bin/node" "$WEKAN_METEOR_SPK_DIR/meteor-spk-0.5.1/meteor-spk.deps/bin/"
# echo "11) Install 7zip"
# sudo apt-get -y install p7zip-full
# # If projects.7z exists, delete it
+2 -1
View File
@@ -28,6 +28,7 @@
# Tunables (env): NPM_RETRY_ATTEMPTS (default 5), NPM_RETRY_SLEEPS (the backoff,
# default "15 30 60 120" seconds; set to "0" in tests).
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ "$#" -eq 0 ]; then
echo "npm-retry.sh: give it a command to run, e.g. npm install" >&2
@@ -49,7 +50,7 @@ export npm_config_fetch_retry_maxtimeout="${npm_config_fetch_retry_maxtimeout:-1
# Kept as one grep -E so it can be read as the list it is.
TRANSIENT='E(50[0-9]|429)|"?(429|50[0-24])"? ?(Too Many Requests|Internal Server Error|Bad Gateway|Service Unavailable|Gateway Time-?out)|ECONNRESET|ECONNREFUSED|ETIMEDOUT|ESOCKETTIMEDOUT|ERR_SOCKET_TIMEOUT|EAI_AGAIN|ENOTFOUND|EPROTO|socket hang up|network socket disconnected|network timeout|Could not resolve host|TLS connection|remote end hung up|early EOF|RPC failed'
log=$(mktemp)
log=$(mktemp "$TMPDIR/wekan.XXXXXX")
trap 'rm -f "$log"' EXIT
attempt=1
+1
View File
@@ -2,6 +2,7 @@
# Snapcraft hashes every file in its project directory, including .git. Keep
# mutable Git metadata outside that directory so --recover finds the same recipe.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
arch=${1:?Usage: prepare-launchpad-source.sh <architecture>}
case "$arch" in armhf|ppc64el|s390x|riscv64) ;; *) echo "Unsupported Launchpad architecture: $arch" >&2; exit 2 ;; esac
root=$(git rev-parse --show-toplevel)
+2 -1
View File
@@ -3,6 +3,7 @@
# Replace a variant repository's working tree with WeKan while retaining the
# package identity by which users install that variant.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ "$#" -ne 4 ]; then
echo "Usage: $0 SOURCE_DIR TARGET_DIR VARIANT_NAME VARIANT_TITLE" >&2
@@ -29,7 +30,7 @@ esac
# running WeKan's release pipeline, retaining them means the ordinary Contents
# token can update source without GitHub requiring the additional `workflow`
# PAT scope merely because WeKan changed a workflow file.
WORKFLOWS_BACKUP="$(mktemp -d)"
WORKFLOWS_BACKUP="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
trap 'rm -rf "$WORKFLOWS_BACKUP"' EXIT
HAD_VARIANT_WORKFLOWS=false
if [ -d "$TARGET_DIR/.github/workflows" ]; then
+8 -5
View File
@@ -4,6 +4,7 @@ if [ -n "${ZSH_VERSION:-}" ]; then exec /bin/bash "$0" "$@"; fi
# Check dependencies for every supported host family.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
. "$SCRIPT_DIR/ensure-tools.sh"
. "$SCRIPT_DIR/tools-env.sh" # Meteor, npm cache and temporary files in .tools
if [ "$(_et_os)" = macos ]; then
_et_brew_ensure
command -v zip >/dev/null 2>&1 || brew install zip
@@ -11,11 +12,12 @@ else
ensure_tools npm zip
fi
# Meteor as https://docs.meteor.com/about/install.html installs it: `npx meteor`,
# as this user, never sudo, into ~/.meteor (see build.sh's Install dependencies).
if ! command -v meteor >/dev/null 2>&1 && [ ! -x "$HOME/.meteor/meteor" ]; then
npx -y meteor
# as this user, never sudo, into .tools/.meteor (see build.sh's Install dependencies):
# HOME is pointed at .tools for the installer only.
if [ ! -x "$WEKAN_TOOLS_DIR/.meteor/meteor" ]; then
HOME="$WEKAN_TOOLS_DIR" npx -y meteor
. "$SCRIPT_DIR/tools-env.sh"
fi
export PATH="$HOME/.meteor:$PATH"
echo "Note: If you use other locale than en_US.UTF-8 , you need to additionally install en_US.UTF-8"
case "$(_et_linux_family)" in
@@ -27,7 +29,8 @@ esac
echo " You can still use any other locale as your main locale."
echo "Building Wekan."
sudo chown -R $(id -u):$(id -g) $HOME/.npm $HOME/.meteor
mkdir -p "$npm_config_cache"
[ -w "$npm_config_cache" ] || sudo chown -R "$(id -u):$(id -g)" "$npm_config_cache"
rm -rf node_modules .meteor/local .build
(meteor update --npm 2>/dev/null || true) && meteor npm install
#METEOR_PROFILE=100 meteor build .build --directory
+5 -4
View File
@@ -1,4 +1,5 @@
#!/bin/bash
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# The checkout is wherever this script is: ~/repos/wekan on Linux,
# ~/Documents/repos/wekan on macOS, Downloads\repos\wekan on Windows.
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -12,7 +13,7 @@ if [ $# -ne 1 ]
fi
mkdir -p ~/rel$1
mv "$REPO_DIR"/wekan-$1*.zip ~/rel$1/
mv ~/Julkinen/wekan-$1*.zip ~/rel$1/
mv "$REPO_DIR"/.build/wekan-$1-amd64.zip ~/rel$1/wekan-$1-amd64.zip
mkdir -p "$TMPDIR/rel$1"
mv "$REPO_DIR"/wekan-$1*.zip "$TMPDIR/rel$1/"
mv "$TMPDIR"/wekan-$1*.zip "$TMPDIR/rel$1/"
mv "$REPO_DIR"/.build/wekan-$1-amd64.zip "$TMPDIR/rel$1/wekan-$1-amd64.zip"
+2 -1
View File
@@ -56,6 +56,7 @@ if [ -n "${ZSH_VERSION:-}" ]; then exec /bin/bash "$0" "$@"; fi
# documented in, and can be created with: ./releases/create-github-secrets.sh
set -e
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
CHECK_ONLY=false
if [ "${1:-}" = --check ]; then CHECK_ONLY=true; shift; fi
@@ -124,7 +125,7 @@ bash "$(dirname "$0")/fix-changelog-hashes.sh" || true
# ── Determine PREVIOUS (OLD) and NEW version — no version argument needed ────
DATE="$(date +%F)"
echo "--- Renaming '# Upcoming WeKan ® release' -> '# v$NEW $DATE WeKan ® release' ---"
_tmp="$(mktemp)"
_tmp="$(mktemp "$TMPDIR/wekan.XXXXXX")"
sed "s|^# Upcoming WeKan ® release.*|# v$NEW $DATE WeKan ® release|" CHANGELOG.md > "$_tmp" && mv "$_tmp" CHANGELOG.md
# Keep CHANGELOG.md small enough for GitHub to show it: older releases move to
+4 -3
View File
@@ -1,4 +1,5 @@
#!/bin/bash
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# The checkout is wherever this script is: ~/repos/wekan on Linux,
# ~/Documents/repos/wekan on macOS, Downloads\repos\wekan on Windows.
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -8,9 +9,9 @@ if [ "$#" -ne 1 ]; then
exit 1
fi
# Lataukset is the Finnish Linux name of Downloads; macOS keeps ~/Downloads
# in every language (Finder only shows it translated).
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$([ -d "$HOME/Lataukset" ] && echo "$HOME/Lataukset" || echo "$HOME/Downloads")}"
# Downloads and release zips stay in .tools/tmp (releases/tools-env.sh), not
# in ~/Downloads or ~/Lataukset.
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$WEKAN_DOWNLOADS}"
cd "$REPO_DIR"
#sudo apt-get -y install parallel
+3 -3
View File
@@ -1,12 +1,12 @@
#!/bin/bash
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# The checkout is wherever this script is: ~/repos/wekan on Linux,
# ~/Documents/repos/wekan on macOS, Downloads\repos\wekan on Windows.
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
# Usage: ./release.sh 1.36
# Delete old stuff
cd ~/
sudo rm -rf .npm
sudo rm -rf "$npm_config_cache"
cd "$REPO_DIR"
sudo rm -rf parts prime stage .meteor-spk
@@ -14,7 +14,7 @@ sudo rm -rf parts prime stage .meteor-spk
cd "$(dirname "$REPO_DIR")"
sudo chown -R $USER wekan
cd ~/
sudo chown -R $USER ~/.meteor
[ -d "$WEKAN_TOOLS_DIR/.meteor" ] && sudo chown -R $USER "$WEKAN_TOOLS_DIR/.meteor"
sudo rm -rf .cache/snapcraft
sudo chown -R $USER .config
+2 -1
View File
@@ -26,6 +26,7 @@
# file, and only file PATHS are passed around - which has no size limit, and
# leaves the text as data that no shell ever parses.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
VERSION="${1:-}"
CHANGELOG_FILE="${2:-CHANGELOG.md}"
@@ -42,7 +43,7 @@ fi
# Prefer a version-specific "# v<version> …" section; otherwise the topmost
# "# Upcoming/Upcomig WeKan ® release" one. The python reads the file itself and
# writes to stdout, so the notes never pass through the shell.
NOTES_FILE="$(mktemp)"
NOTES_FILE="$(mktemp "$TMPDIR/wekan.XXXXXX")"
trap 'rm -f "$NOTES_FILE"' EXIT
VERSION="$VERSION" CHANGELOG_FILE="$CHANGELOG_FILE" python3 - > "$NOTES_FILE" <<'PYEOF'
+7 -4
View File
@@ -1,6 +1,7 @@
#!/bin/bash
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# The checkout is wherever this script is: ~/repos/wekan on Linux,
# ~/Documents/repos/wekan on macOS, Downloads\repos\wekan on Windows.
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -20,9 +21,9 @@ fi
# 2) Commit and push version number changes
cd "$REPO_DIR"
# Lataukset is the Finnish Linux name of Downloads; macOS keeps ~/Downloads
# in every language (Finder only shows it translated).
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$([ -d "$HOME/Lataukset" ] && echo "$HOME/Lataukset" || echo "$HOME/Downloads")}"
# Downloads and release zips stay in .tools/tmp (releases/tools-env.sh), not
# in ~/Downloads or ~/Lataukset.
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$WEKAN_DOWNLOADS}"
if compgen -G "$DOWNLOAD_DIR/node-v*-linux-*.tar.*" > /dev/null && compgen -G "$DOWNLOAD_DIR/mongodb-*.tgz" > /dev/null; then
read -r -p "Check the latest Node.js and MongoDB versions from the internet? (y/n) " CHECK_LATEST
case "$CHECK_LATEST" in
@@ -43,7 +44,9 @@ fi
# are ready before commit/tag steps.
"$REPO_DIR/releases/release-bundle.sh" "$2"
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$([ -d "$HOME/Lataukset" ] && echo "$HOME/Lataukset" || echo "$HOME/Downloads")}"
# Downloads and release zips stay in .tools/tmp (releases/tools-env.sh), not
# in ~/Downloads or ~/Lataukset.
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$WEKAN_DOWNLOADS}"
ZIP_PATH="$DOWNLOAD_DIR/wekan-$2-amd64.zip"
if [ ! -f "$ZIP_PATH" ]; then
echo "Error: Expected bundle not found: $ZIP_PATH"
+14 -5
View File
@@ -27,6 +27,7 @@
# Produces wekan-$VERSION-$ARCH.zip, its .sha256sum, and provenance/$ARCH.tsv.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# Older macOS has no sha256sum; shasum -a 256 prints the same "hash file" lines and takes -c.
command -v sha256sum >/dev/null 2>&1 || sha256sum() { shasum -a 256 "$@"; }
@@ -49,12 +50,12 @@ command -v sha256sum >/dev/null 2>&1 || sha256sum() { shasum -a 256 "$@"; }
# in the middle of an npm install.
bash releases/check-arch-binaries.sh \
"$ARCH" "$NODE_ARCH" "$FERRETDB_ARCH" "$NODE_VERSION" "$IMAGE" "$PLATFORM" \
| tee /tmp/preflight.txt
| tee "$TMPDIR/preflight.txt"
NODE_FULL="$(sed -n 's/^node_full=//p' /tmp/preflight.txt)"
NODE_FROM="$(sed -n 's/^node_from=//p' /tmp/preflight.txt)"
NODE_URL="$(sed -n 's/^node_url=//p' /tmp/preflight.txt)"
NODE_SHA256="$(sed -n 's/^node_sha256=//p' /tmp/preflight.txt)"
NODE_FULL="$(sed -n 's/^node_full=//p' "$TMPDIR/preflight.txt")"
NODE_FROM="$(sed -n 's/^node_from=//p' "$TMPDIR/preflight.txt")"
NODE_URL="$(sed -n 's/^node_url=//p' "$TMPDIR/preflight.txt")"
NODE_SHA256="$(sed -n 's/^node_sha256=//p' "$TMPDIR/preflight.txt")"
bash releases/record-provenance.sh "$ARCH" 'Node.js' \
"$NODE_FROM" "$NODE_FULL" "$NODE_URL" "${NODE_SHA256:-}"
@@ -83,6 +84,11 @@ echo "Using Node ${NODE_FULL} (${NODE_ARCH}) for ${ARCH}, from ${NODE_FROM}"
# "node-v24.18.1-linux-.tar.xz", with an empty architecture, and died on the 404
# - four of the six failures in the v10.57 run. A file has no quoting layer to
# get wrong, and `bash -n` can check it.
# The container's own /tmp is a small part of its writable layer: its /tmp is
# this repository's .tools/tmp/repack-<arch>-tmp, where the Node.js download
# and its unpacking happen (releases/install-node-for-arch.sh uses $TMPDIR).
CONTAINER_TMP="$TMPDIR/repack-$ARCH-tmp"
mkdir -p "$CONTAINER_TMP" && chmod 1777 "$CONTAINER_TMP"
docker run --rm --platform "$PLATFORM" \
-e NODE_FULL="$NODE_FULL" \
-e NODE_FROM="$NODE_FROM" \
@@ -90,11 +96,14 @@ docker run --rm --platform "$PLATFORM" \
-e NODE_SHA256="$NODE_SHA256" \
-e ARCH="$ARCH" \
-e NODE_ARCH="$NODE_ARCH" \
-e TMPDIR=/tmp \
-v "$CONTAINER_TMP:/tmp" \
-v "$(pwd)/bundle:/bundle" \
-v "$(pwd)/releases:/releases:ro" \
-w /bundle/programs/server \
"$IMAGE" \
bash /releases/install-node-for-arch.sh
rmdir "$CONTAINER_TMP" 2>/dev/null || true
# ── FerretDB, this architecture's own ────────────────────────────────────────
# These arches have no MongoDB server, so FerretDB is their default DB. The
+2 -1
View File
@@ -71,6 +71,7 @@
# are curl, grep, sed and awk.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
platform="${1:?usage: resolve-node-source.sh <platform> <major|version>}"
want="${2:?usage: resolve-node-source.sh <platform> <major|version>}"
@@ -200,7 +201,7 @@ esac
# It is fetched at most once per version per source; the cache matters because a
# platform nobody publishes (i386, armhf, win32) would otherwise re-fetch both
# files for every version it walks back through.
cache="$(mktemp -d)"
cache="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
trap 'rm -rf "$cache"' EXIT
shasums() { # <base-url> <version> <tag-for-the-cache-file>
+3 -2
View File
@@ -25,6 +25,7 @@
# `capnp` schema compiler and the C++ RPC libraries
# .tools/sandstorm sandstorm-io/sandstorm at the commit pinned below
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
# Older macOS has no sha256sum; shasum -a 256 prints the same "hash file" lines and takes -c.
command -v sha256sum >/dev/null 2>&1 || sha256sum() { shasum -a 256 "$@"; }
@@ -47,7 +48,7 @@ MOCK_SCHEMAS="sandstorm-http-bridge grain identity powerbox util activity"
CAPNP_PREFIX="$TOOLS/capnproto"
if [ ! -x "$CAPNP_PREFIX/bin/capnp" ] || [ ! -f "$CAPNP_PREFIX/lib/libcapnp-rpc.a" ]; then
echo "==> Building Cap'n Proto $CAPNP_VERSION into $CAPNP_PREFIX"
work="$(mktemp -d)"
work="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
bash "$FETCH" -o "$work/capnproto.tar.gz" "https://capnproto.org/capnproto-c++-$CAPNP_VERSION.tar.gz"
echo "$CAPNP_SHA256 $work/capnproto.tar.gz" | sha256sum -c -
tar -xzf "$work/capnproto.tar.gz" -C "$work"
@@ -75,7 +76,7 @@ SRC="$TOOLS/sandstorm/src"
# Its type check cannot see the other generated files (it resolves imports in
# memory only), so its diagnostics are reported but do not stop the emit; the
# load check at the end is what decides whether the output is usable.
gen="$(mktemp -d)"
gen="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
trap 'rm -rf "$gen"' EXIT
cp -r "$REPO/npm-packages/capnp-es" "$gen/capnp-es"
(cd "$gen/capnp-es" && npm install --no-save --no-package-lock --ignore-scripts \
+2 -1
View File
@@ -29,6 +29,7 @@
# METEOR the meteor command [meteor]
# SANDSTORM_SLIM_NO_PACK=1 stop after the trimmed bundle is verified
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
REPO="$(cd "$(dirname "$0")/.." && pwd)"
cd "$REPO"
@@ -40,7 +41,7 @@ if [ -z "$OUT" ] && [ "${SANDSTORM_SLIM_NO_PACK:-}" != "1" ]; then
fi
[ $# -gt 0 ] && shift
DEPS="${DEPS:-$HOME/projects/meteor-spk/meteor-spk-0.6.0/meteor-spk.deps}"
DEPS="${DEPS:-$WEKAN_METEOR_SPK_DIR/meteor-spk-0.6.0/meteor-spk.deps}"
METEOR="${METEOR:-meteor}"
SPK_BIN="${SPK:-}"
if [ -z "$SPK_BIN" ]; then
+2 -1
View File
@@ -3,12 +3,13 @@
# (.github/workflows/bun-single-file.yml). Run on the target's own platform:
# ZIP=<bundle.zip> BUN=<the bundle's own bun> OUT=<file> VERSION=<v> TARGET=<t>
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/../../tools-env.sh" # temporary files and downloads go to .tools/tmp
: "${ZIP:?}" "${BUN:?}" "${OUT:?}" "${VERSION:?}" "${TARGET:?}"
"$BUN" releases/single-file/bun/build.mjs --zip "$ZIP" --target "$TARGET" --version "$VERSION" --out "$OUT"
exe="./$OUT"
cache="$(mktemp -d)"
cache="$(mktemp -d "$TMPDIR/wekan.XXXXXX")"
trap 'rm -rf "$cache"' EXIT
# It names itself.
+1
View File
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then
echo "Usage: $0 <snap-file> [channels]" >&2
+86
View File
@@ -0,0 +1,86 @@
#!/bin/bash
#
# tools-env.sh - keep every file WeKan's scripts write inside .tools/.
#
# Usage (source it): . "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh"
#
# .tools/ is ignored by .gitignore and .meteorignore, so nothing written there
# can reach a commit or trigger a Meteor rebuild. The scripts used to write
# all over the machine instead: downloads and release zips to ~/Downloads or
# ~/Lataukset, temporary files to /tmp (or macOS's /var/folders), Meteor to
# ~/.meteor, nvm and Node.js to ~/.nvm, the npm cache to ~/.npm, Volta to
# ~/.volta, the Sandstorm toolchain to ~/projects, and PATH lines into
# ~/.zshrc and ~/.bashrc. Each now lives under .tools/, on Linux, macOS and
# Windows (Git Bash, WSL) alike:
#
# .tools/tmp TMPDIR, TMP, TEMP - temporary files, downloads, release zips
# .tools/.meteor Meteor (METEOR_WAREHOUSE_DIR once installed there; on PATH)
# .tools/nvm nvm and the Node.js it installs (NVM_DIR), macOS
# .tools/n the Node.js n installs (N_PREFIX), Linux
# .tools/npm-cache npm's cache (npm_config_cache)
# .tools/volta Volta (VOLTA_HOME)
# .tools/projects/meteor-spk the Sandstorm packaging toolchain (as projects.7z unpacks)
#
# HOME itself is NOT moved: git reads ~/.gitconfig (the commit author) and
# ~/.ssh from it, and gh its login. Only what WeKan's scripts write is moved.
# Sourcing twice is harmless.
WEKAN_DIR="${WEKAN_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
WEKAN_TOOLS_DIR="${WEKAN_TOOLS_DIR:-$WEKAN_DIR/.tools}"
# Only a real checkout gets a .tools/: scripts mounted into a build container
# without one (/releases) must not make a /.tools/tmp in the container's layer.
if [ -e "$WEKAN_DIR/.git" ] || [ -d "$WEKAN_TOOLS_DIR" ]; then
mkdir -p "$WEKAN_TOOLS_DIR/tmp" 2>/dev/null || true
fi
# Not "if unset": macOS always sets TMPDIR (to /var/folders/...), and a CI
# runner sets its own - and those are exactly what is being replaced. Inside a
# build container, where the scripts are mounted without the checkout, the
# caller mounts .tools/tmp as the container's /tmp and sets TMPDIR to it
# (repack-bundle-for-arch.sh): a container's own /tmp is too small to use.
# macOS's mktemp ignores TMPDIR when it is given no template, so every script
# names one: mktemp "$TMPDIR/name.XXXXXX".
case "${TMPDIR:-}/" in
*/.tools/*) ;; # already inside a .tools/ - a test's own directory there is kept
*) if [ -w "$WEKAN_TOOLS_DIR/tmp" ]; then TMPDIR="$WEKAN_TOOLS_DIR/tmp"; fi ;;
esac
if [ -z "${TMPDIR:-}" ] || [ ! -w "$TMPDIR" ]; then
echo "tools-env.sh: $WEKAN_TOOLS_DIR/tmp cannot be written and no writable TMPDIR is set" >&2
return 1 2>/dev/null || exit 1
fi
TMP="$TMPDIR"
TEMP="$TMPDIR"
WEKAN_DOWNLOADS="$TMPDIR"
# Meteor reads its packages from METEOR_WAREHOUSE_DIR (default ~/.meteor). It
# is pointed at .tools/.meteor only once Meteor is installed there - build.sh's
# install-dependencies option does that - so a Meteor still in ~/.meteor keeps
# working until then instead of finding an empty warehouse.
if [ -x "$WEKAN_TOOLS_DIR/.meteor/meteor" ]; then
METEOR_WAREHOUSE_DIR="$WEKAN_TOOLS_DIR/.meteor"
export METEOR_WAREHOUSE_DIR
fi
NVM_DIR="$WEKAN_TOOLS_DIR/nvm"
npm_config_cache="$WEKAN_TOOLS_DIR/npm-cache"
VOLTA_HOME="$WEKAN_TOOLS_DIR/volta"
WEKAN_METEOR_SPK_DIR="$WEKAN_TOOLS_DIR/projects/meteor-spk"
export WEKAN_DIR WEKAN_TOOLS_DIR TMPDIR TMP TEMP WEKAN_DOWNLOADS \
NVM_DIR npm_config_cache VOLTA_HOME WEKAN_METEOR_SPK_DIR
# The tools installed there come first on PATH, so no shell rc file has to be
# edited for a new shell to find them: Meteor, n's Node.js, the newest nvm put in
# .tools/nvm, Volta's, the Sandstorm meteor-spk, and .tools/bin.
_wekan_env_path="$WEKAN_TOOLS_DIR/bin:$WEKAN_TOOLS_DIR/.meteor:$WEKAN_TOOLS_DIR/n/bin:$VOLTA_HOME/bin"
if [ -d "$NVM_DIR/versions/node" ]; then
_wekan_env_node="$(ls -d "$NVM_DIR"/versions/node/v*/bin 2>/dev/null | sort -V | tail -n 1 || true)"
if [ -n "$_wekan_env_node" ]; then _wekan_env_path="$_wekan_env_path:$_wekan_env_node"; fi
fi
# Written to never fail under a caller's `set -euo pipefail`: ls of a missing
# directory fails, and pipefail makes that the substitution's status.
_wekan_env_spk="$(ls -d "$WEKAN_METEOR_SPK_DIR"/meteor-spk-* 2>/dev/null | sort -V | tail -n 1 || true)"
if [ -n "$_wekan_env_spk" ]; then _wekan_env_path="$_wekan_env_path:$_wekan_env_spk"; fi
case ":$PATH:" in
*":$WEKAN_TOOLS_DIR/bin:$WEKAN_TOOLS_DIR/.meteor:"*) ;;
*) PATH="$_wekan_env_path:$PATH" ;;
esac
export PATH
unset _wekan_env_path _wekan_env_node _wekan_env_spk
@@ -1,5 +1,6 @@
#!/bin/sh
set -eu
. "$(dirname "${BASH_SOURCE[0]}")/../tools-env.sh" # temporary files and downloads go to .tools/tmp
translation_repo_dir=$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)
cd "$translation_repo_dir"
+4 -3
View File
@@ -5,6 +5,7 @@
# =================================================================
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/tools-env.sh" # temporary files and downloads go to .tools/tmp
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
@@ -12,9 +13,9 @@ cd "$REPO_ROOT"
# Local copy of dependency artifacts, used only for the optional
# USE_LOCAL_DEP_VERSIONS offline mode. Builds themselves fetch from upstream.
# Lataukset is the Finnish Linux name of Downloads; macOS keeps ~/Downloads
# in every language (Finder only shows it translated).
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$([ -d "$HOME/Lataukset" ] && echo "$HOME/Lataukset" || echo "$HOME/Downloads")}"
# Downloads and release zips stay in .tools/tmp (releases/tools-env.sh), not
# in ~/Downloads or ~/Lataukset.
DOWNLOAD_DIR="${DOWNLOAD_DIR:-$WEKAN_DOWNLOADS}"
# Portable sedi function for macOS/Linux compatibility
sedi() {
+1
View File
@@ -430,6 +430,7 @@ test('every script in releases/ is reachable from BOTH menus', () => {
// container for musl, Termux-like Android builders, the Bun builders).
SKIP['musl/build-bundle.sh'] = 'musl bundle build run by musl.yml inside Alpine';
SKIP['android/build-bundle.sh'] = 'Android bundle build run by android.yml';
SKIP['tools-env.sh'] = 'sourced by build.sh and the releases/ scripts: keeps every file they write in .tools/';
SKIP['nextcloud-pack.sh'] = 'Nextcloud ExApp packer run by nextcloud.yml before signing; tested by tests/nextcloudRelease.test.py';
SKIP['single-file/bun/build-and-check.sh'] = 'Bun single-file build run by bun-single-file.yml';
SKIP['single-file/bun/build.mjs'] = 'Bun single-file builder run by bun-single-file.yml and releases/bun/build-target.sh';
+3 -1
View File
@@ -121,7 +121,9 @@ test('it is db-conformance.sh\'s databases, started the same way, and both menus
assert.ok(sh.includes('SUITE="${WEKAN_DB_SUITE:-conformance}"'));
assert.match(sh, /node docs\/Databases\/FerretDB\/1\/SpeedTest\/run\.cjs --uri "\$uri" --label "\$label"/);
assert.match(sh, /node docs\/Databases\/FerretDB\/1\/SpeedTest\/report\.cjs --dir "\$LOGDIR"/);
assert.match(sh, /docker_exec rm -fv "\$CONTAINER"/, 'containers go with their volumes - the speed test writes gigabytes');
// remove_container empties the container's /tmp (.tools/tmp, mounted) first.
assert.match(sh, /remove_container\(\) \{[\s\S]*?docker_exec rm -fv "\$1"/, 'containers go with their volumes - the speed test writes gigabytes');
assert.match(sh, /remove_container "\$CONTAINER"/);
assert.match(sh, /SQLITE_DATA="\$\(mktemp -d "\$WEKAN_DIR\/\.tools\/tmp\//, 'SQLite data on disk under .tools/tmp, not /tmp');
assert.ok(read('docs/Databases/FerretDB/1/SpeedTest/speedtest.sh').includes('export WEKAN_DB_SUITE=speedtest'));
assert.ok(read('build.sh').includes('./docs/Databases/FerretDB/1/SpeedTest/speedtest.sh'));
+1 -1
View File
@@ -14,7 +14,7 @@ function fixture() {
for (const file of ['Dockerfile', 'snapcraft.yaml', 'Stackerfile.yml', 'package.json', 'package-lock.json',
'sandstorm-pkgdef.capnp', '.meteor/release', 'docs/Platforms/Propietary/OS/Windows/Offline.md',
'releases/version.sh', 'releases/dependency-versions.py', 'releases/update-website-version-info.sh',
'releases/use-release-npm.sh']) {
'releases/use-release-npm.sh', 'releases/tools-env.sh']) {
fs.mkdirSync(path.dirname(path.join(dir, file)), {recursive: true});
fs.copyFileSync(path.join(root, file), path.join(dir, file));
}
+24 -19
View File
@@ -17,6 +17,7 @@ const assert = require('assert');
const fs = require('fs');
const path = require('path');
const read = rel => fs.readFileSync(path.join(__dirname, '..', rel), 'utf8');
const sh = fs.readFileSync(path.join(__dirname, '..', 'build.sh'), 'utf8');
// The INSTALL branch specifically: `darwin` also appears in the branches that
@@ -68,8 +69,11 @@ test('nvm is sourced before it is used', () => {
const sourced = code.indexOf('. "$NVM_DIR/nvm.sh"');
assert.ok(sourced !== -1, 'nvm.sh must be sourced');
assert.ok(sourced < code.indexOf('nvm install 26'), 'before the first nvm call');
assert.ok(/NVM_DIR="\$\{NVM_DIR:-\$HOME\/\.nvm\}"/.test(code),
'and an NVM_DIR the caller already set must be honoured');
// NVM_DIR is .tools/nvm (releases/tools-env.sh, sourced at the top of
// build.sh): every file WeKan's scripts write stays in .tools/ (2026-10-27).
assert.ok(/NVM_DIR="\$WEKAN_TOOLS_DIR\/nvm"/.test(read('releases/tools-env.sh')),
'nvm and its Node.js live in .tools/nvm');
assert.ok(/mkdir -p "\$NVM_DIR"/.test(code));
assert.ok(/if ! command -v nvm/.test(code), 'a failed install is reported, not run into');
});
@@ -93,29 +97,30 @@ test('the Homebrew node@24 keg and its hand-exported flags are gone', () => {
}
});
test('a new shell finds nvm too', () => {
assert.ok(/export NVM_DIR="\$HOME\/\.nvm"/.test(code), 'NVM_DIR is exported in the rc file');
assert.ok(/for rc in "\$HOME\/\.zshrc" "\$HOME\/\.bashrc"/.test(code),
'zsh (the macOS default) and bash both');
// Appended once, however many times this menu entry is run.
assert.strictEqual((code.match(/grep -qxF/g) || []).length, 2,
'each line is added only if it is not already there');
assert.ok(/touch "\$HOME\/\.zshrc"/.test(code),
'and .zshrc is created if the machine has none');
test('a new shell finds nvm\'s Node.js without any rc file being edited', () => {
// It used to append NVM_DIR to ~/.zshrc and ~/.bashrc. Every file WeKan
// writes stays in .tools/ now (2026-10-27): the installer is told to edit no
// profile, and build.sh and the releases/ scripts put the newest Node.js in
// .tools/nvm on PATH themselves (releases/tools-env.sh).
assert.ok(/install\.sh \| PROFILE=\/dev\/null bash/.test(code), 'the nvm installer edits no rc file');
assert.ok(!/>>\s*"\$rc"|\.zshrc|\.bashrc/.test(code.replace(/^\s*#.*$/gm, '')), 'nor does build.sh');
assert.ok(/\$NVM_DIR"\/versions\/node\/v\*\/bin/.test(read('releases/tools-env.sh')), 'tools-env.sh puts it on PATH');
});
test('Meteor is still installed the same way afterwards', () => {
// The point of the branch is Node; everything after it is unchanged.
assert.ok(/npx -y meteor/.test(code));
// Spelled with "$HOME" since 0dbf9d8502, the same line as the Linux branch.
assert.ok(/export PATH="\$HOME\/\.meteor:\$PATH"/.test(code));
test('Meteor is still installed with npx, into .tools/.meteor', () => {
// `npx meteor` installs into $HOME/.meteor, so HOME is .tools for the
// installer only, the same line as the Linux branch; tools-env.sh then puts
// .tools/.meteor on PATH and points METEOR_WAREHOUSE_DIR at it.
assert.ok(/HOME="\$WEKAN_TOOLS_DIR" npx -y meteor/.test(code));
assert.ok(/\. "\$WEKAN_DIR\/releases\/tools-env\.sh"/.test(code));
});
test('and Linux is untouched - it installs Node its own way', () => {
// Only macOS was asked to move to nvm. Linux uses `n` from npm, which needs
// sudo and is what the Debian/Ubuntu instructions say.
// Only macOS was asked to move to nvm. Linux uses `n`, now into .tools/n with
// N_PREFIX and without sudo, as everything WeKan writes stays in .tools/.
const linux = sh.slice(sh.indexOf('if [[ "$OSTYPE" == "linux-gnu" ]]; then'), start);
assert.ok(/sudo npm -g install n\b/.test(linux), 'Linux still uses n');
assert.ok(/N_PREFIX="\$WEKAN_TOOLS_DIR\/n" npx -y n "\$_wekan_node_version"/.test(linux), 'Linux still uses n');
assert.ok(!/sudo n /.test(linux), 'not into /usr/local with sudo');
assert.ok(!/nvm/.test(linux), 'and not nvm');
});
+5 -3
View File
@@ -81,11 +81,13 @@ assert.deepStrictEqual(meteorInstalls.filter(({ text }) => !/--allow-scripts=met
'npm 12 blocks the meteor installer\'s install script without --allow-scripts=meteor');
// Positive: every script that installs Meteor does it with `npx -y meteor`.
// build.sh has it twice: the Linux and the macOS branch.
// build.sh has it twice: the Linux and the macOS branch. On Linux and macOS
// HOME is .tools for the installer only, so Meteor lands in .tools/.meteor:
// every file WeKan's scripts write stays in .tools/ (releases/tools-env.sh).
for (const [file, line, count] of [
['build.sh', 'npx -y meteor', 2],
['build.sh', 'HOME="$WEKAN_TOOLS_DIR" npx -y meteor', 2],
['build.bat', 'call npx -y meteor', 1],
['releases/rebuild-release.sh', 'npx -y meteor', 1],
['releases/rebuild-release.sh', 'HOME="$WEKAN_TOOLS_DIR" npx -y meteor', 1],
]) {
assert.strictEqual(lines(path.join(root, file)).filter(({ text }) => text.trim() === line).length,
count, `${file} installs Meteor with: ${line}`);
+7 -7
View File
@@ -164,7 +164,8 @@ test('the base-image preflight compares the ARM variant, not just the architectu
test('the base-image manifest probe retries transient registry failures safely', () => {
assert.ok(/for attempt in 1 2 3 4 5/.test(preflight),
'one transient Docker Hub response must not fail an otherwise complete build');
assert.ok(/>"\$manifest_tmp"[\s\S]*?mv "\$manifest_tmp" \/tmp\/manifest\.json/.test(preflight),
// In .tools/tmp, not /tmp: every file the scripts write stays in .tools/.
assert.ok(/>"\$manifest_tmp"[\s\S]*?mv "\$manifest_tmp" "\$TMPDIR\/manifest\.json"/.test(preflight),
'only a complete successful response may replace the manifest being parsed');
assert.ok(/\[ "\$manifest_ok" = true \]/.test(preflight),
'five failed attempts must still fail closed');
@@ -179,7 +180,7 @@ test('the preflight platform match answers correctly for the manifests it meets'
const os = require('os');
const script = preflight
.split("if ! WANT_ARCH=\"$want_arch\" WANT_VARIANT=\"$want_variant\" python3 - <<'PYEOF'\n")[1]
.split("if ! MANIFEST=\"$TMPDIR/manifest.json\" WANT_ARCH=\"$want_arch\" WANT_VARIANT=\"$want_variant\" python3 - <<'PYEOF'\n")[1]
.split('\nPYEOF\n')[0];
assert.ok(script.includes('want_variant'), 'could not extract the preflight python block');
@@ -187,15 +188,16 @@ test('the preflight platform match answers correctly for the manifests it meets'
const scriptPath = path.join(tmp, 'match.py');
fs.writeFileSync(scriptPath, script);
const manifestPath = '/tmp/manifest.json';
const saved = fs.existsSync(manifestPath) ? fs.readFileSync(manifestPath) : null;
// The script reads the manifest from MANIFEST ($TMPDIR/manifest.json), in
// .tools/tmp: this test writes its own copy there, never into /tmp.
const manifestPath = path.join(tmp, 'manifest.json');
function matches(manifest, platform) {
fs.writeFileSync(manifestPath, JSON.stringify(manifest));
const [, arch, variant = ''] = platform.split('/');
try {
execFileSync('python3', [scriptPath], {
env: { ...process.env, WANT_ARCH: arch, WANT_VARIANT: variant },
env: { ...process.env, MANIFEST: manifestPath, WANT_ARCH: arch, WANT_VARIANT: variant },
stdio: 'pipe',
});
return true;
@@ -232,8 +234,6 @@ test('the preflight platform match answers correctly for the manifests it meets'
assert.strictEqual(matches(singlePlatform, 'linux/arm/v7'), false,
'and it must not answer for a variant it is not');
} finally {
if (saved !== null) fs.writeFileSync(manifestPath, saved);
else fs.rmSync(manifestPath, { force: true });
fs.rmSync(tmp, { recursive: true, force: true });
}
});
+1 -1
View File
@@ -13,7 +13,7 @@ function fixture(fn) {
try {
fs.mkdirSync(path.join(dir, 'releases'));
fs.mkdirSync(path.join(dir, 'bin'));
for (const name of ['latest-release-version.sh', 'check-upcoming-release.sh']) {
for (const name of ['latest-release-version.sh', 'check-upcoming-release.sh', 'tools-env.sh']) {
fs.copyFileSync(path.join(root, 'releases', name), path.join(dir, 'releases', name));
}
// Run the actual preflight only. Never include the publishing portion.
+2 -1
View File
@@ -160,7 +160,8 @@ test('the sync preserves snap, npm and container package identities', () => {
assert.ok(/rm -f "\$TARGET_DIR\/\.github\/dependabot\.yml"/.test(prepareVariant)
&& /"\$TARGET_DIR\/\.github\/dependabot\.yaml"/.test(prepareVariant),
'variant mirrors do not inherit duplicate Dependabot version updates');
assert.ok(/WORKFLOWS_BACKUP="\$\(mktemp -d\)"/.test(prepareVariant),
// In .tools/tmp: macOS's mktemp ignores TMPDIR without a template.
assert.ok(/WORKFLOWS_BACKUP="\$\(mktemp -d "\$TMPDIR\/[a-z.-]+XXXXXX"\)"/.test(prepareVariant),
'the variant workflow directory is saved before replacing its tree');
assert.ok(/cp -a "\$WORKFLOWS_BACKUP\/\." "\$TARGET_DIR\/\.github\/workflows\/"/.test(prepareVariant),
'and the variant-owned workflows are restored after copying WeKan');
+5 -5
View File
@@ -5,8 +5,8 @@
// macOS, Downloads\repos\wekan on Windows. The release scripts hard-coded
// Linux's ~/repos/wekan, so on a Mac they cd'd into nothing - or into a stale
// copy. They find the checkout from their own location now, which is right on
// every system, and the Finnish Linux download folder "Lataukset" is no longer
// assumed on macOS, where ~/Downloads keeps its English name.
// every system. Downloads and release zips are in .tools/tmp, not ~/Downloads
// or the Finnish Linux ~/Lataukset (tests/toolsDirOnly.test.cjs).
// Run: node tests/repoLocationPerOs.test.cjs
const assert = require('assert');
@@ -57,10 +57,10 @@ test('a script that needs the checkout finds it from its own location', () => {
}
});
test('the download folder is not assumed to be the Finnish Linux one', () => {
const offenders = scripts().filter(rel => /DOWNLOAD_DIR="\$\{DOWNLOAD_DIR:-\$HOME\/Lataukset\}"/.test(read(rel)));
test('downloads go to .tools/tmp, not a folder in the home directory', () => {
const offenders = scripts().filter(rel => /DOWNLOAD_DIR="\$\{DOWNLOAD_DIR:-[^}]*\$HOME/.test(read(rel)));
assert.deepStrictEqual(offenders, []);
assert.match(read('releases/version.sh'), /\[ -d "\$HOME\/Lataukset" \] && echo "\$HOME\/Lataukset" \|\| echo "\$HOME\/Downloads"/);
assert.match(read('releases/version.sh'), /DOWNLOAD_DIR="\$\{DOWNLOAD_DIR:-\$WEKAN_DOWNLOADS\}"/);
});
test('negative: the pattern catches each system\'s hard-coded path', () => {
+7
View File
@@ -48,6 +48,12 @@ const EXCLUDED = new Map([
// A suite that hangs would otherwise hang the whole run. Generous, because some
// suites parse the entire client tree.
const SUITE_TIMEOUT_MS = Number(process.env.WEKAN_SUITE_TIMEOUT_MS) || 300000;
// Every file a suite writes goes to .tools/tmp, never /tmp or macOS's
// /var/folders: os.tmpdir() reads TMPDIR (TEMP and TMP on Windows), the same
// variables releases/tools-env.sh sets for the shell scripts.
const TOOLS_TMP = path.join(ROOT, '.tools', 'tmp');
fs.mkdirSync(TOOLS_TMP, { recursive: true });
const SUITE_ENV = { ...process.env, TMPDIR: TOOLS_TMP, TMP: TOOLS_TMP, TEMP: TOOLS_TMP };
function discoverSuites() {
const found = [];
@@ -100,6 +106,7 @@ function main(argv) {
const r = spawnSync(process.execPath, [suite], {
cwd: ROOT,
env: SUITE_ENV,
stdio: 'inherit',
timeout: SUITE_TIMEOUT_MS,
});
+130
View File
@@ -0,0 +1,130 @@
'use strict';
// Every file WeKan's scripts write stays in .tools/, which git and Meteor
// ignore: temporary files, downloads and release zips in .tools/tmp, Meteor in
// .tools/.meteor, Node.js in .tools/n (Linux) or .tools/nvm (macOS), the npm
// cache, Volta and the Sandstorm toolchain beside them - on Linux, macOS and
// Windows alike (releases/tools-env.sh). They used to write to /tmp, macOS's
// /var/folders, ~/Downloads, ~/Lataukset, ~/.meteor, ~/.nvm, ~/.npm, ~/.volta,
// ~/projects, and PATH lines into ~/.zshrc and ~/.bashrc - which is also what
// a sandboxed editor (Flatpak on Linux, sandbox-exec on macOS) cannot allow.
// Run: node tests/toolsDirOnly.test.cjs
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const { spawnSync } = require('child_process');
const root = path.join(__dirname, '..');
const read = rel => fs.readFileSync(path.join(root, rel), 'utf8');
let passed = 0;
const test = (name, fn) => { fn(); passed += 1; console.log(' ok -', name); };
function scripts() {
const out = fs.readdirSync(root).filter(f => f.endsWith('.sh'));
const walk = dir => {
for (const e of fs.readdirSync(path.join(root, dir), { withFileTypes: true })) {
const rel = path.join(dir, e.name);
if (e.isDirectory()) walk(rel);
else if (/\.sh$/.test(e.name) || rel === 'releases/rel.sh') out.push(rel);
}
};
walk('releases');
return [...new Set(out)];
}
const code = src => src.split('\n').filter(l => !/^\s*#/.test(l)).join('\n');
// The Linux VM appliance (virtualbox/) has its own fixed /home/wekan layout.
const ELSEWHERE = /^releases\/virtualbox\//;
// A container's /tmp named on the host side - the mount point of .tools/tmp
// (-v ...:/tmp), the TMPDIR given to the container, or the container emptying
// its own /tmp - is .tools/tmp, not the container's own small /tmp.
const containerSide = c => c.split('\n')
.filter(l => !/:\/tmp"|-e TMPDIR=\/tmp\b|exec "\$1" sh -c 'rm -rf \/tmp\//.test(l)).join('\n');
test('tools-env.sh puts each kind of file in .tools/', () => {
const env = read('releases/tools-env.sh');
for (const line of [
'TMPDIR="$WEKAN_TOOLS_DIR/tmp"', 'TMP="$TMPDIR"', 'TEMP="$TMPDIR"', 'WEKAN_DOWNLOADS="$TMPDIR"',
'METEOR_WAREHOUSE_DIR="$WEKAN_TOOLS_DIR/.meteor"', 'NVM_DIR="$WEKAN_TOOLS_DIR/nvm"',
'npm_config_cache="$WEKAN_TOOLS_DIR/npm-cache"', 'VOLTA_HOME="$WEKAN_TOOLS_DIR/volta"',
'WEKAN_METEOR_SPK_DIR="$WEKAN_TOOLS_DIR/projects/meteor-spk"',
]) assert.ok(env.includes(line), line);
assert.doesNotMatch(code(env), /\bHOME=/, 'HOME is not moved: git needs ~/.gitconfig and ~/.ssh');
});
test('sourced, it replaces a TMPDIR that was already set, and mktemp lands in .tools/tmp', () => {
const r = spawnSync('bash', ['-c', '. releases/tools-env.sh; printf "%s\\n%s\\n" "$TMPDIR" "$npm_config_cache"; f="$(mktemp "$TMPDIR/wekan-test.XXXXXX")"; echo "$f"; rm -f "$f"'],
{ cwd: root, encoding: 'utf8', env: { ...process.env, TMPDIR: '/var/folders/elsewhere', WEKAN_DIR: '', WEKAN_TOOLS_DIR: '' } });
assert.strictEqual(r.status, 0, r.stderr);
const [tmpdir, cache, made] = r.stdout.trim().split('\n');
assert.strictEqual(tmpdir, path.join(root, '.tools', 'tmp'));
assert.strictEqual(cache, path.join(root, '.tools', 'npm-cache'));
assert.ok(made.startsWith(path.join(root, '.tools', 'tmp') + path.sep), made);
});
test('negative: sourcing never stops a script running under set -euo pipefail', () => {
// With no .tools/projects/meteor-spk, `ls` failed, pipefail made that the
// substitution's status, and set -e stopped every release script there.
const r = spawnSync('bash', ['-c', 'set -euo pipefail; WEKAN_METEOR_SPK_DIR=; . releases/tools-env.sh; echo survived'],
{ cwd: root, encoding: 'utf8' });
assert.strictEqual(r.status, 0, r.stderr);
assert.strictEqual(r.stdout.trim(), 'survived');
});
test('every script that makes temporary files names .tools/tmp, sourcing tools-env.sh', () => {
const bad = [];
for (const rel of scripts().filter(r => !ELSEWHERE.test(r) && r !== 'releases/tools-env.sh')) {
const c = containerSide(code(read(rel)));
// macOS's mktemp ignores TMPDIR without a template: one must be named.
if (/\bmktemp(\s+-d)?\s*\)/.test(c) || /\bmktemp(\s+-d)?\s*$/m.test(c)) bad.push(`${rel}: mktemp with no template`);
if (/(^|[\s"=(])\/tmp\b/m.test(c)) bad.push(`${rel}: /tmp`);
// In a container the caller's mounted TMPDIR is required: ${TMPDIR:?...}.
if (/\bmktemp\b|\$TMPDIR\b/.test(c) && !/tools-env\.sh|WEKAN_TOOLS_DIR\/tmp|WEKAN_DIR\/\.tools\/tmp|\$\{TMPDIR:\?/.test(c)) {
bad.push(`${rel}: TMPDIR not set to .tools/tmp`);
}
}
assert.deepStrictEqual(bad, []);
});
test('no script writes to a folder in the home directory or edits a shell rc file', () => {
const HOME_WRITE = /(~|\$HOME|\$\{HOME\})\/(\.meteor|\.npm|\.nvm|\.volta|projects|Downloads|Lataukset|Julkinen|rel\$)|>>\s*(~|\$HOME)\/\.(bash|zsh)rc/;
const bad = scripts().filter(r => !ELSEWHERE.test(r))
.filter(rel => HOME_WRITE.test(code(read(rel)).replace(/METEOR_BIN="\$HOME\/\.meteor\/meteor"/g, '')));
assert.deepStrictEqual(bad, []);
});
test('build.sh sources tools-env.sh and installs Node.js and Meteor into .tools', () => {
const sh = read('build.sh');
assert.match(sh, /^\. "\$WEKAN_DIR\/releases\/tools-env\.sh"$/m);
assert.match(sh, /N_PREFIX="\$WEKAN_TOOLS_DIR\/n" npx -y n "\$_wekan_node_version"/);
assert.match(sh, /HOME="\$WEKAN_TOOLS_DIR" npx -y meteor/);
assert.match(sh, /install\.sh \| PROFILE=\/dev\/null bash/, 'the nvm installer edits no rc file');
assert.doesNotMatch(code(sh), /sudo n "|>>\s*"\$rc"/);
});
test('a build container\'s /tmp is this checkout\'s .tools/tmp, never its own', () => {
const repack = read('releases/repack-bundle-for-arch.sh');
assert.match(repack, /-v "\$CONTAINER_TMP:\/tmp"/);
assert.match(repack, /CONTAINER_TMP="\$TMPDIR\/repack-\$ARCH-tmp"/);
assert.match(read('releases/install-node-for-arch.sh'), /: "\$\{TMPDIR:\?TMPDIR must be the mounted \.tools\/tmp\}"/);
const conf = read('releases/db-conformance.sh');
assert.match(conf, /--name "\$CONTAINER" -v "\$CONTAINER_TMP:\/tmp"/, 'every database container');
assert.match(conf, /-v "\$FERRET2_TMP:\/tmp"/, 'and FerretDB 2');
const env = read('releases/tools-env.sh');
assert.doesNotMatch(code(env), /TMPDIR:-\/tmp/, 'never a /tmp fallback');
});
test('the node suites run with TMPDIR in .tools/tmp', () => {
assert.match(read('tests/run-node-suites.cjs'), /env: SUITE_ENV,/);
assert.match(read('tests/run-node-suites.cjs'), /TMPDIR: TOOLS_TMP, TMP: TOOLS_TMP, TEMP: TOOLS_TMP/);
});
test('negative: the checks catch what they are for', () => {
assert.match('x="$(mktemp)"', /\bmktemp(\s+-d)?\s*\)/);
assert.match('d="$(mktemp -d)"', /\bmktemp(\s+-d)?\s*\)/);
assert.doesNotMatch('d="$(mktemp -d "$TMPDIR/x.XXXXXX")"', /\bmktemp(\s+-d)?\s*\)/);
assert.match('cp a /tmp/b', /(^|[\s"=(])\/tmp\b/m);
assert.doesNotMatch('cp a "$TMPDIR/b"', /(^|[\s"=(])\/tmp\b/m);
});
console.log(`\n${passed} tests passed`);
+2
View File
@@ -18,6 +18,8 @@ const oldValue = '{"board":"Монгол самбар"}\n';
try {
fs.copyFileSync(path.join(root, 'releases/translations/pull-translations.sh'),
path.join(scripts, 'pull-translations.sh'));
// It sources ../tools-env.sh, which keeps its temporary files in .tools/tmp.
fs.copyFileSync(path.join(root, 'releases/tools-env.sh'), path.join(scripts, '..', 'tools-env.sh'));
fs.writeFileSync(path.join(data, 'mn.i18n.json'), oldValue);
fs.writeFileSync(path.join(data, 'en.i18n.json'), '{"board":"Board"}\n');
const tx = path.join(fixture, 'tx');