T-487: the blocklist updater tested on the device; EasyPrivacy's truncation guard and the skipped early look fixed (cavekit-content-blocking R5)

On the Enyo card: a forced update (lastUpdate cleared) fetched EasyList 202610191826 and EasyPrivacy
202610191828 over HTTPS, logged the versions and the counts, and the next page's image from
beacon.cdnpure.com, a rule only in the new EasyPrivacy, was blocked (render/goanna/test/
jihad-blocking-update.html, docs/rounds/t487-update.png). With the network off (a dead proxy) the update
logged "no download" for both lists and changed nothing, with no dialog (t487-netoff.png). Across daemon
restarts no download within 4 days, nor within a day of a failed one.

Two bugs the runs found, fixed in components/jihadContentBlocker.js: EasyPrivacy was refused as
"truncated" on every update (its 57,139 lines held against both lists' rule count; each download is now
held against its own list's copy), and a due update skipped at the timer's one early look (a page had
started within 20 s) waited six hours (it now looks again 60-90 s later).

tools/device/set-prefs.sh waits for the daemon to exit before it edits prefs.js (an exiting daemon saved
its prefs over the edit).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

context/kits/cavekit-content-blocking.md

@@ -131,9 +131,9 @@ place; on any failure it keeps what it has and waits a day before the next try;
 cannot postpone it. Whichever of the bundled snapshot and the download is newer is used, so a
 package upgrade's snapshot is not shadowed by an older download. Updates are logged.
 **Acceptance Criteria:**
-- [ ] A forced update on the device (the button, or `jihad.blocking.lastUpdate` cleared) downloads both files over HTTPS, logs the versions and the counts, and the next page is blocked by the new set (a rule present only in the new list is honoured).
-- [ ] With the network off, the update fails cleanly: a log line, no change, no dialog.
-- [ ] The download is not attempted more often than every 4 days (the timestamp pref, checked across a daemon restart), and a failed attempt not more often than daily (`jihad.blocking.lastAttempt`).
+- [x] A forced update on the device (the button, or `jihad.blocking.lastUpdate` cleared) downloads both files over HTTPS, logs the versions and the counts, and the next page is blocked by the new set (a rule present only in the new list is honoured). *(2026-10-19, the Enyo card, `lastUpdate` cleared: EasyList 202610151429 -> 202610191826 and EasyPrivacy 202610151429 -> 202610191828 over HTTPS, hosts 91,924 -> 93,360, logged with the counts; the next page's image from `beacon.cdnpure.com`, a rule only in the new EasyPrivacy, was blocked (`docs/rounds/t487-update.png`). The run found EasyPrivacy refused as "truncated" on every update (its 57,139 lines held against both lists' rules together): each download is now held against its own list's copy.)*
+- [x] With the network off, the update fails cleanly: a log line, no change, no dialog. *(2026-10-19: a dead proxy (127.0.0.1:9) and `lastUpdate` 1: `update: easylist: no download`, `easyprivacy: no download`, the versions and the compiled set unchanged, `lastAttempt` set, no dialog (`docs/rounds/t487-netoff.png`).)*
+- [x] The download is not attempted more often than every 4 days (the timestamp pref, checked across a daemon restart), and a failed attempt not more often than daily (`jihad.blocking.lastAttempt`). *(2026-10-19, five minutes after a daemon restart each: with `lastUpdate` fresh, 5 card connects and 0 update lines; after the failed attempt above (`lastUpdate` 1, `lastAttempt` today), 4 connects and 0 update lines. The runs found the timer's one early look skipped when a page had started within 20 s, the next look six hours away: a due update now looks again 60-90 s later.)*
 **Dependencies:** R2
 
 ### R6: It fits the device

context/plans/build-site.md

@@ -530,7 +530,7 @@ kit's table says what was rejected and why).
 |------|-------|---------|-------------|-----------|--------|--------|--------|
 | T-485 | The lists in the tree: `third_party/easylist/{easylist.txt,easyprivacy.txt,README.md}` pinned (upstream `! Version:` kept, sha256 in `make-device-bundle.sh`, bundled raw under `blocklists/`); NOTICE, CREDITS.md, about:jihad, `licenses/` name them under GPLv3+ | cavekit-content-blocking.md | R2, R7 | — | no | S | DONE (`third_party/easylist/` snapshots 202610151429 with README, sha256-pinned and bundled raw under `blocklists/` by `make-device-bundle.sh`; NOTICE, CREDITS.md, about:jihad and the About pane name them under GPLv3+) |
 | T-486 | `render/goanna/components/jihadContentBlocker.js`: the compiler (host set, third-party set, exceptions, generic cosmetic selectors, skip counts), the profile cache, `nsIContentPolicy` in the `content-policy` category (top-level and same-host never blocked, eTLD for third-party), the user sheet, prefs `jihad.blocking.*`, the counter, the log lines; registered by `make-device-bundle.sh`, created at daemon start; test pages `jihad-blocking.html`, `jihad-blocking-many.html` | cavekit-content-blocking.md | R1, R2, R3 | T-485 | yes | L | DONE on all three (v5.1 packages: Enyo 14:20, Mochi 14:36, Mojo 14:53 on the fifth try; the http test page right on Mochi; MDN clean); memory +40 MB idle open (R6: the memory reporter figures being read; jemalloc is off in this build) |
-| T-487 | The updater: every 4 days from easylist.to over HTTPS, newer `! Version:` only, compile and swap in place, keep-what-you-have on failure, `jihad.blocking.lastUpdate`; `tools/host/count-blocklist.py` for the independent count | cavekit-content-blocking.md | R5, R2 | T-486 | yes | M | BUILT, untested (the 4-day timer, `update()` for the button, newer-version-only swap, keep-what-you-have on failure; the forced-update and network-off tests pending) |
+| T-487 | The updater: every 4 days from easylist.to over HTTPS, newer `! Version:` only, compile and swap in place, keep-what-you-have on failure, `jihad.blocking.lastUpdate`; `tools/host/count-blocklist.py` for the independent count | cavekit-content-blocking.md | R5, R2 | T-486 | yes | M | DONE 2026-10-19 (forced update, network off, no repeat within 4 days or a day after a failure, on the Enyo card; two bugs found and fixed: EasyPrivacy refused as truncated on every update, the early look skipped and the next six hours away) |
 | T-488 | Settings → Privacy → "Content blocking": two switches, the read-only versions/counter line, the first button row ("Update the lists now") in `preferences.js`; effect-verified live in the three variants | cavekit-content-blocking.md | R4 | T-486, T-487 | yes | M | DONE on all three (v5.1 packages; Mochi's rows shot, Enyo and Mojo to shoot in the second run) |
 | T-489 | Device verification and numbers: MDN without adverts (before/after screenshots, the README's image), VmRSS delta, `load done` with the sheet on/off and with 200 blocked images, the exception and third-party cases, network-off update; the README's feature sentence with the credit | cavekit-content-blocking.md | R1, R3, R6, R7 | T-488 | yes | M | DONE as far as the device allows (MDN without adverts in the README; the http test page; memory: idle 34.8 MB with the lists on demand, +22 MB with a card (the 12 MB goal not met, accepted for this version); the GC criterion met (1,073 vs 1,101 ms); the 200-image timing not measured (the stalls are the pages' own scripts); the updater's forced update and network-off tests still pending) |
 | T-490 | Review of Tier 86 (the inspector; Codex when its credits return) and fixes; the licensing audit extended to the lists | cavekit-content-blocking.md | R7 | T-485…T-489 | no | M | DONE 2026-10-19 (round 26 in impl-review-findings-6.md, every finding dispositioned; F-6 fixed with the recomposed card view; F-21 fixed: `count-blocklist.py` has a character-level tokenizer sharing nothing with the compiler, and `--cross` finds no rule the two classify differently in the bundled lists or today's 202610191813; the component's fixes run on the bench in all three interfaces) |

render/goanna/components/jihadContentBlocker.js

@@ -117,6 +117,8 @@ function readFileText(file, max) {
   try { return NetUtil.readInputStreamToString(fis, max ? Math.min(max, fis.available()) : fis.available(), {charset: "UTF-8"}); }
   finally { fis.close(); }
 }
+// Lines in a text, without the array a regex match of every newline would make.
+function countLines(text) { let n = 0; for (let i = text.indexOf("\n"); i >= 0; i = text.indexOf("\n", i + 1)) n++; return n; }
 // The list's "! Version:" from its first 2 KB — never the whole file for that (inspector F-4).
 function versionOfFile(file) { try { return versionOf(readFileText(file, 2048)); } catch (e) { return ""; } }
 function versionOf(text) {
@@ -395,6 +397,7 @@ function JihadContentBlocker() {
   this._updating = false;
   this._lastError = "";
   this._lastDocAt = 0;
+  this._soonTimer = null;
   this._noLists = false;
   try { Services.prefs.addObserver("jihad.blocking.", this, false); } catch (e) {}
   try { Services.obs.addObserver(this, "document-element-inserted", false); } catch (e) {}
@@ -557,6 +560,9 @@ JihadContentBlocker.prototype = {
     if (pv && pv >= bv) return p;
     return b.exists() ? b : (p.exists() ? p : b);
   },
+  _lineCount: function (name) {
+    try { return countLines(readFileText(this._rawFile(name))); } catch (e) { return 0; }
+  },
   _load: function () {
     try {
       let files = {}, versions = {};
@@ -621,6 +627,7 @@ JihadContentBlocker.prototype = {
   },
   _install: function (r, versions, how) {
     this._hosts = r.hosts; this._tp = r.tp; this._excHosts = r.excHosts; this._typed = r.typed; this._paths = r.paths;
+    if (how !== "update") this._checkSoon();   // T-487: lists loaded after the timer's early look
     this._excRules = r.excRules; this._site = r.site; this._counts = r.counts;
     this._versions = versions;
     try { Services.prefs.setCharPref(PREF_VERSIONS, (versions.easylist || "-") + "/" + (versions.easyprivacy || "-")); } catch (e) {}
@@ -697,7 +704,10 @@ JihadContentBlocker.prototype = {
     if (!force) {
       if (t - last < UPDATE_EVERY_S) return;
       if (t - attempt < RETRY_AFTER_S) return;                             // a failed try: a day's rest
-      if (Date.now() - this._lastDocAt < QUIET_AFTER_DOC_MS) return;        // not while a page starts (R5)
+      if (Date.now() - this._lastDocAt < QUIET_AFTER_DOC_MS) {             // not while a page starts (R5)
+        this._checkSoon();   // T-487: due, so again when it is quiet, not at the next 6-hour look
+        return;
+      }
     }
     this._updating = true; this._lastError = "";
     setIntPref(PREF_ATTEMPT, t);
@@ -735,8 +745,10 @@ JihadContentBlocker.prototype = {
         xhr.onload = () => {
           let txt = xhr.responseText || "";
           let ok = xhr.status === 200 && /^\[Adblock/.test(txt.slice(0, 20)) && versionOf(txt);
-          // a truncated download has far fewer rules than what it would replace
-          if (ok) { let n = (txt.match(/\n/g) || []).length, cur = self._counts ? self._counts.rules : 0; if (cur && n < cur * 0.5) { ok = false; self._lastError = l.name + ": truncated (" + n + " lines)"; } }
+          // a truncated download has far fewer lines than the copy of the same list it would replace
+          // (T-487: it was held against both lists' rules together, which EasyPrivacy alone never
+          // reaches, so EasyPrivacy was refused as truncated on every update)
+          if (ok) { let n = countLines(txt), cur = self._lineCount(l.name); if (cur && n < cur * 0.5) { ok = false; self._lastError = l.name + ": truncated (" + n + " lines against " + cur + ")"; } }
           if (ok) got[l.name] = txt;
           else { if (!self._lastError) self._lastError = l.name + ": HTTP " + xhr.status; log("update: " + self._lastError); }
           if (--pending === 0) done();
@@ -749,6 +761,14 @@ JihadContentBlocker.prototype = {
 
   // ── for the settings page (wrappedJSObject) ──
   update: function () { this._maybeUpdate(true); },
+  // T-487: one look at the clock in a minute or so (never two pending). The device run of 2026-10-19
+  // found the timer's one early look skipped because a document had started within 20 s (any
+  // document counts, the chrome's own too), and the next look six hours away; lists loaded after
+  // the early look waited as long.
+  _checkSoon: function () {
+    if (this._soonTimer) return;
+    this._soonTimer = setTimeout(() => { this._soonTimer = null; this._maybeUpdate(false); }, (60 + Math.floor(Math.random() * 30)) * 1000);
+  },
   state: function () {
     return {enabled: this._enabled, cosmetic: this._cosmetic, loaded: !!this._hosts, noLists: this._noLists, versions: this._versions,
             lastUpdate: intPref(PREF_LAST, 0), count: this._count, counts: this._counts, updating: this._updating, error: this._lastError};

render/goanna/test/jihad-blocking-update.html

@@ -0,0 +1,25 @@
+<!DOCTYPE html>
+<!-- Jihad Browser test page (cavekit-content-blocking.md R5): after a forced update of the lists the
+     next page is blocked by the new set. ||beacon.cdnpure.com^ is in EasyPrivacy 202610191813 and not
+     in the bundled 202610151429 (which has only rum.cdnpure.com and paths under cdnpure.com), so with
+     the bundled lists the image below is fetched and after the update it is refused: with
+     jihad.blocking.log on, the daemon logs "blocked image by ||beacon.cdnpure.com^". The second image
+     is a host rule of both lists (the control). Serve it over http from the host PC (python3 -m
+     http.server 8077 in render/goanna/test/). Copyright 2026 NotAlexNoyle. Apache-2.0. -->
+<html><head><meta charset="utf-8"><title>jihad-blocking-update</title>
+<style>li{font:15px sans-serif;margin:6px} .ok{color:#060} .bad{color:#900}</style></head>
+<body><h1>jihad-blocking-update</h1><ul id="r"></ul>
+<script>
+var tests = [
+  ["https://beacon.cdnpure.com/jihad-test.gif", "EasyPrivacy 202610191813 ||beacon.cdnpure.com^ (new in the update)"],
+  ["https://ad.doubleclick.net/ddm/ad/pixel.gif", "EasyList ||ad.doubleclick.net^ (in both: the control)"]
+];
+function row(t, cls) { var li = document.createElement("li"); li.className = cls; li.textContent = t; document.getElementById("r").appendChild(li); }
+tests.forEach(function (t) {
+  var el = document.createElement("img"), done = false;
+  el.onload = function () { if (!done) { done = true; row("loaded   " + t[0] + "  — " + t[1], "ok"); } };
+  el.onerror = function () { if (!done) { done = true; row("refused/failed  " + t[0] + "  — " + t[1], "bad"); } };
+  el.src = t[0];
+  document.body.appendChild(el);
+});
+</script></body></html>

tools/device/set-prefs.sh

@@ -16,7 +16,19 @@ esac
 RE="${2:?name regex}"; NEW="${3:-}"
 P=/media/cryptofs/apps/usr/palm/applications/$id/profile/prefs.js
 [ -f "$P" ] || { echo "no $P"; exit 1; }
-stop $job >/dev/null; sleep 3
+stop $job >/dev/null
+# The daemon's clean shutdown saves its prefs when they changed: an edit made before it has exited
+# can be written over, so wait for the process to go (the fixed 3 s did not always cover it). NB a
+# cleared jihad.blocking.lastUpdate comes back by design: the blocker sets it from EasyList's
+# version when it is 0; force an update with the value 1.
+i=0
+while [ $i -lt 30 ]; do
+	alive=0
+	for q in /proc/[0-9]*; do case "$(tr '\0' ' ' 2>/dev/null < $q/cmdline)" in *"applications/$id/"*jihad-browserserver*) alive=1;; esac; done
+	[ $alive = 0 ] && break
+	sleep 1; i=$((i+1))
+done
+[ $alive = 0 ] || { echo "the $job daemon is still running after 30 s: prefs not changed"; start $job >/dev/null; exit 1; }
 # BusyBox 1.17 sed knows -r (not -E); the name is matched inside user_pref("…").
 sed -i -r "/user_pref\(\"($RE)/d" "$P"
 [ -n "$NEW" ] && [ -f "$NEW" ] && { cat "$NEW" >> "$P"; rm -f "$NEW"; }