The decision that made DAST cheap was made three months earlier, and it had nothing to do with security

Okay, confession time.

One of my side projects has about eight hundred lines of bash that run as root on a VPS. They disable password login, open firewall ports and write the deploy that swaps the running container. And until very recently, nothing looked at them. CI checked that TypeScript compiled, that the tests passed and that the image built, and never once looked at those scripts.

By the time I added shellcheck, one bug had already shipped and been found by luck: a helper function called head was shadowing coreutils’ head inside a command substitution. It only showed up on the failure path, which is, of course, the one you never actually run.

That’s the real problem, and it’s what this post is about. My CI was checking that the software worked, not that what it built and shipped was defensible. So I set out to close that gap end to end (code, secrets, dependencies, infrastructure, image and the running app) at zero cost, on a private repo on GitHub Free. This is the first of four posts: the build, and what the first run and the first merge found. The second is the three weeks I then lived with it, which is the part almost nobody writes up. The third is about how little the scanner was actually reaching, and how I found out. The fourth is what all of it cost to run, which turned out not to be money. So here goes.

The project, in case you’re curious: a Telegram bot that keeps track of the food in my kitchen and its expiry dates, and asks an LLM for recipes with whatever’s about to go off. TypeScript, Fastify, ports and adapters, Drizzle and Postgres, deployed with Docker Compose behind Caddy. Hold on to that last detail; it comes back in a following post.

The part I thought would be hard

The thing I’d been putting off was DAST: attacking the app while it runs instead of reading its source. It sounds expensive. Before you can attack something you have to stand it up, and on paper standing this up means a Postgres database, a paid API key for the LLM and a Telegram bot token.

That arithmetic is what pushes people towards the easy way out: pointing the scanner at production, which is a terrible idea. An active scan doesn’t just read responses, it sends real payloads to everything it finds, and against an authenticated API that means POST, PATCH and DELETE with a valid token. On production, that’s a scanner emptying and corrupting the very pantry it’s supposed to protect, on a two-vCPU box that’s also running the database.

It turned out I needed none of that:

- name: Boot an ephemeral instance
  env:
    GENERATOR: fake
    REPOSITORY: memory
    AUTH_MODE: token
  run: |
    TOKEN="$(openssl rand -hex 24)"
    echo "::add-mask::$TOKEN"
    AUTH_TOKENS="$TOKEN:zap" node dist/main.js > app.log 2>&1 &

No database, no API key, no bot token. It boots inside the runner, ZAP gets thrown at it, and it’s thrown away, together with whatever the scanner did to its in-memory pantry. The token is minted by the job itself, and AUTH_MODE=token is the mode production runs, not the dev mode that trusts a user-id header.

GENERATOR=fake and REPOSITORY=memory don’t exist for security reasons. They exist because I wanted pnpm test to work on a freshly cloned laptop, with no secrets and no docker compose up. The recipe generator and the food repository are ports, and each has a fake adapter behind it. It was a convenience decision about writing tests, nothing more.

And “this app boots in full with no external dependencies” turns out to be exactly what DAST needs. Without those adapters, the choice would have been between running Postgres in the workflow, writing a mock LLM provider just so a scanner could run, or pointing at production. All three are worse, and the third is where a lot of teams end up.

The lesson isn’t “use hexagonal architecture”. It’s that testability and scannability share one prerequisite: an app you can run without production infrastructure. If your app can’t boot without secrets, it isn’t just awkward to test. You won’t be able to scan it anywhere except production, and you’ll find that out the day you want to, not the day you decided it.

Two details from that job worth knowing before you start. Without an auth header, ZAP bounces off 401s and cheerfully reports that your site is one page long. And a page that renders its data client-side needs a crawler that runs the page, which is what -j does. What -j means changed under me, though. Since July 2026 it starts ZAP’s newer Client Spider rather than the AJAX Spider it used to, the stable image had already picked that up, and I only found out by running the scan with -d and reading which job the plan names. A flag whose meaning moves under a tag called stable is worth knowing about.

The rest of the pipeline, in short

The tools weren’t a comparison. They were what a private repo on a free plan leaves you. No CodeQL, no native secret scanning, no code scanning alerts to upload results into (all three need the paid add-ons; the Security tab is there, but without them the only alerts it shows are Dependabot’s), and no protected branches, so a red check doesn’t stop a merge.

SonarQube Cloud would have fitted (its free tier does cover a private project this size), but I wanted nothing that needs an account on a third-party service. What’s left: Semgrep Community Edition, gitleaks, pnpm audit and OSV-Scanner, hadolint and Trivy, and ZAP. And with nowhere to upload results to, the exit code is the only control I have, and the artifacts are the report.

The scanners themselves are dependencies, and they get treated as such: every action pinned to a commit SHA, with the repo setting that refuses unpinned actions turned on; every job on contents: read unless it can name the write it needs; gitleaks pinned by version and sha256; Semgrep by image digest, with --metrics=off.

Two decisions where the ordering is the control, not the tool. pnpm audit runs before pnpm install, so a known vulnerable dependency is rejected before any install script gets to run. That matters here because this repo sets dangerouslyAllowAllBuilds: true (the targeted onlyBuiltDependencies list didn’t approve esbuild on pnpm 11, so for now it’s wholesale), so those scripts do run. The catch is in known: it only sees published advisories, and the window between a package being compromised and the advisory existing is a different fence, one a following post is about.

And the image is built, then scanned by Trivy, and only then pushed. Scan after the push and the vulnerable image is already pullable by a deploy; your gate has quietly become a notification. One seam: the push rebuilds from cache rather than pushing the exact bytes Trivy saw, so both builds must get the same arguments, and nothing compares the two digests. Building once and pushing by digest is the proper fix, and it’s on the list.

Last, what blocks. A gate that cries wolf ends up switched off, and nobody switches it back on. So three things block: a committed secret, a Semgrep ERROR, a HIGH or CRITICAL misconfiguration. Everything else (ZAP alerts, OSV’s extra advisories) only reports, until I know what volume it produces. Keep that ERROR threshold in mind. It’s going to do exactly what I told it to, word for word, and in a following post that turns out to be the problem.

What happened the first time I ran it

It failed.

The SAST job failed, and it found a real vulnerability in the pipeline itself.

The deploy workflow put ${{ github.event.release.tag_name }} straight into a run: block. GitHub substitutes that before the shell ever sees the line, so a tag name with shell metacharacters would run on a runner that, in the very next step, gets handed the SSH deploy key, and could poison $GITHUB_PATH and $GITHUB_OUTPUT on the way. The scanner I’d just added found the hole in the mechanism that was deploying it.

A stranger couldn’t exploit it (creating a release needs write access to the repo). What makes it a finding rather than noise is something else: the step right above it, in the same file, already did it correctly, passing the value through env:. An inconsistency you don’t see rereading your own file and a machine does. That, for me, is the whole argument for running SAST on a one-person side project.

The same run also exposed a bug in the gate I’d written myself. Semgrep puts partial-parse notices, at level warn, in the same errors[] array as real failures, and ${{ }} inside a run: block produces one every time it’s there. My script treated anything in that array as “the scanner is broken” and exited 2. So the first run failed for the right reason and the wrong reason at the same time. One line fixed it:

errors = [e for e in report.get("errors", []) if e.get("level") != "warn"]

The lesson is one of those that sounds obvious until you pay for it: your gate is code, and code you haven’t run against real data is untested code.

DAST found nothing serious that day, no High and no Medium. But remember, a scan that bounces off 401s also reports a clean site, one page long, and that looks a lot like a pass. So the number I cared about wasn’t zero findings; it was whether ZAP had reached /pantry and /pantry/items rather than just the landing page. It had: six URLs. The page (counted twice, with and without the trailing slash), the two API routes, and the robots.txt and sitemap.xml probes.

Six isn’t a lot, and I assumed I knew why. I was wrong, and finding out took another eleven days. That’s the third post.

The first merge to main, and eleven findings I could never have seen

Trivy killed it. Eleven HIGH or CRITICAL: node-tar at CRITICAL, plus sigstore, pacote, picomatch and ip-address among others. No push, no SBOM, no attestation. Nothing vulnerable reached the registry.

None of it was my app. It was the dependencies npm bundles inside itself, shipped in node:22-slim, invisible to pnpm-lock.yaml and therefore invisible to pnpm audit forever. That’s why scanning your lockfile and scanning the image are different jobs: the lockfile only knows what you asked for, and the image is what you actually ship. Those eleven had been there all along; nothing had been looking.

The fix wasn’t a suppression. Nothing in the runtime image runs npm (the container starts node dist/main.js against a node_modules built in an earlier stage), so npm came out:

RUN rm -rf /usr/local/lib/node_modules/npm \
           /usr/local/lib/node_modules/corepack \
           /usr/local/bin/npm /usr/local/bin/npx /usr/local/bin/corepack

The fix for a dependency you don’t run is not to have it, and the scan went clean.

A second gap sat right next to that one. The image job only ran on main, so the Dockerfile change that removed npm couldn’t be verified before merging. Neither could any Dockerfile change, ever. It now builds and scans on pull requests and pushes only from main. Nothing reaches the server without review, because the push is still the gated step; only the build moved.

If you only keep one thing from this, keep the first one: an app that boots without secrets is an app you can scan anywhere, and mine got there through a decision about tests, not about security. The other two are smaller. Scan the image before you push it, not after. And treat the gate script like any other code: until real data has gone through it, it hasn’t been tested.

That’s the build. What three weeks of living with it did, including the next time it blocked me and the second bug it found in my own gate, is the next post.

Update, 29 September 2026

Two things above stopped being true this week. On the 28th of the month GitHub stopped starting jobs: two ZAP scans on every PR, and the same two again on every push to main, had used up the 2,000 free Actions minutes a private repo gets a month (GitHub’s meter put the month’s usage at $12.09, all covered by the allowance). A release sat ten minutes waiting for an image job that never started. The pipeline now runs on a self-hosted runner, a 2-core VPS separate from the bot’s. And the image job no longer pushes a sha-<commit> image on every merge for the release to re-tag: on a PR it builds and scans without pushing, and the deploy builds, scans and pushes the release image itself. The story is in the fourth post.