Commit Graph
1 Commits
Author SHA1 Message Date
Lauri Ojansivu e876b6029e Security report script: flatten the paginated pages, so the dumps are not silently empty.
`./pull-security-reports.sh` reported OK for every endpoint and then wrote empty
split files and blank counts:

  Counts (open / closed):
    CodeQuality/StandardFindings:  / -
    Dependabot/Vulnerabilities:  / -

`gh api --paginate --slurp` returns ONE ARRAY PER PAGE, so a list endpoint arrives
as [[alert,...],[alert,...]]. Nothing flattened that, so `.[]` yielded PAGES, not
alerts, and every `select(.state == "open")` was applied to an array. jq failed,
its error went to /dev/null, and the `>` redirect left a 0-byte file - which is
not valid JSON, so every later step on it failed too and printed nothing. A
security report that quietly claims there is nothing to see is worse than one
that fails, so:

- fetch() now concatenates the pages into a single list, but only when EVERY
  element is itself an array; endpoints returning one object (/repos/{o}/{r},
  the SBOM, default-setup) are slurped to [ {...} ] and are left alone, because
  their renders index .[0].
- split() writes through a temp file, keeps a valid empty ARRAY and the jq error
  on failure, and warns in the summary instead of truncating the destination.
- the secret-scanning redaction validates the redacted copy before deleting the
  file that still holds live secrets - it used to write nothing and delete the
  source anyway - and now removes that file on the failure path as well.
- count() always prints a number, so the summary can no longer show a blank
  column, and the closed column starts at "-" rather than an empty string.

The same shape also broke the newest-analysis lookup (`sort_by | last | .id`
over pages) and the codeql-databases dump; both come right with the flatten.

Verified offline with the fake-gh harness the README documents, replaying the
real fixtures from the failed run: code scanning 4 pages -> 392 alerts,
Dependabot 1 page -> 86, analyses 152 pages -> 15118, advisories -> 22, and the
splits land as 2 open code scanning / 386 closed and 2 open Dependabot / 83
closed, with no zero-byte file anywhere in the run.

Tests: tests/securityReportScript.test.cjs guards the flatten, the non-truncating
split, the redaction ordering and the count helper, and - when jq is installed -
runs the actual filters, including a check that select() on an array-of-pages
still fails, so the guard cannot quietly stop being what protects the split.

Thanks to xet7 !
2026-07-25 02:28:58 +03:00