DAST coverage is counted at the server, not in the scanner's report

This is the third of four posts. The first built a DevSecOps pipeline for free on a private repo, and its cheapest part was DAST: the app boots with fake adapters behind its ports, so ZAP attacks a throwaway instance inside the runner, with the auth header added at ZAP’s proxy. The second was three weeks of living with the gates.

Where the first post left the scan: six URLs. I’d guessed the per-item routes were missing because an empty pantry has no ids to offer, so I seeded two items before the scan and ran it again. Still six. The reason was one line in the page, if (!token()) return gate(false): it checked sessionStorage for a token before fetching anything, found none, and showed the login gate, so ZAP’s browser never rendered a row. The two API URLs ZAP had came from the script’s source text, not from the page. The scanner was authenticated; the page it was driving wasn’t.

Both posts left something out, and this one is about that: what the scan doesn’t cover, and the two days it took to find out how little it was reaching.

What the scan doesn’t cover

The DAST scans an app with its riskiest parts removed, and that follows directly from the thing I spent the first post praising. REPOSITORY=memory means there is no SQL, so ZAP can’t find an injection in the Drizzle adapter even if one exists. GENERATOR=fake means the LLM adapter is never on the path, so the scan never touches my prompt injection surface. And there is one, because food names are user input and they end up in a prompt. What the DAST covers is the HTTP surface and the domain logic.

Of that HTTP surface, the spider reaches less than it appears to. Six URLs, and the per-item routes and the suggestions endpoint never exercised, because the spider couldn’t log into the page. An OpenAPI definition and ZAP’s API scan would cover them, and the next section is about doing exactly that.

The Telegram webhook isn’t in the ephemeral instance at all, and that’s where the thesis ends. It only registers when a bot token is present, and there isn’t one. So the property the first post opened with, that the app boots without secrets and can therefore be scanned, holds for two of the three adapters. The third needs a secret to exist, and it happens to be the most exposed thing the app has: a public endpoint whose only authentication is a secret header Telegram sends, with no user or session behind it. It has unit tests; it has no scanner. A fake adapter for the Telegram transport would bring it inside the scan the same way the other two came in. Until that’s done, the argument covers the surface I built, not the one that faces the internet.

The SBOM was a record that expired. It was uploaded as a workflow artifact, which GitHub keeps for ninety days, so an advisory landing in month four would have found nothing to compare against. It now attaches to the GitHub release it describes, generated from the image that release shipped. The first releases to carry one are in the next section, because they didn’t go the way I expected.

Semgrep’s rules aren’t pinned, and ZAP’s are worse than I’d first written: the scan scripts refresh the rule packs from ZAP’s marketplace on every run unless you pass -silent, which I don’t, so even a digest-pinned image would fetch fresh rules each time. Both are fixable and I’ve done neither, which is a choice I should own.

And none of it checks business logic. Not one of the scanners in the pipeline will notice that one user can read another user’s pantry. That’s still tests, and it’s still the part you write by hand.

Put together: the adapters are exactly what the scan can’t see, and they’re where I’d look first if I were attacking this.

Two days later

In the first post I said the fix for the per-item routes was an API-definition scan and that I hadn’t made it. Then I made it, and a couple of other things happened on the way. Every step below is measured from the app’s own request log, which the DAST job now dumps at the end, not from ZAP’s report. The rows from before that log existed are ZAP’s own counts, and I say which is which, because the two don’t count the same thing.

The web session moved from sessionStorage to an HttpOnly cookie. The reason was security, not DAST: a token in sessionStorage is a token any script on the page can read. Now POST /auth/login answers with a __Host- cookie, HttpOnly, Secure, SameSite=Strict, thirty days, stored server-side as a SHA-256 digest. The page no longer touches sessionStorage and sends no auth header at all; it starts by asking GET /auth/me. And a test greps the page script and asserts it contains none of sessionStorage, localStorage, Authorization or the dev-mode user header, so none of it can creep back.

The side effect on DAST was immediate, and this time I measured it. The page now fetches before deciding whether to show the login, that fetch goes through ZAP’s proxy, and the proxy adds the Bearer. So the spider’s browser got a 200 from /auth/me, loaded /pantry and rendered the rows, which had never happened before. Six URLs became eight in the app’s log. It even submitted the add form, a POST /pantry/items that Zod rejected with 422, which is proof the request travelled authenticated.

But it still didn’t press the buttons on the rows. Not one PATCH, not one DELETE to /pantry/items/<id>. I gave it three minutes instead of one (-m 3): the exact same ten requests, taking 21 seconds longer. Reverted, with the reason in the commit message. At that point it’s no longer auth and no longer time.

Two crawlers, same ceiling. The spider behind -j turned out to be ZAP’s current Client Spider, not the AJAX Spider I thought I was running (the first post has that story). It reached eight URLs; the older AJAX Spider, asked for explicitly with --ajax-spider, reached eleven, and the two agree on exactly which five carry alerts. Neither touched a per-item route. That’s not a crawler being lazy; it’s a crawler being a crawler. A route that only exists behind a button’s click handler, with no link or form pointing at it, isn’t something a crawl finds. It’s something you have to tell the scanner about.

So, the API-definition scan. A hand-written openapi.json with the nine routes, and zaproxy/action-api-scan hitting each one through the same header replacer. Two details that matter more than the file:

• A test keeps the OpenAPI in sync with the routes Fastify actually registers, read from printRoutes(), in both directions: a new route without an entry breaks CI, and an entry without a route breaks it too. The map can’t go stale in silence.
• The seed keeps the ids of the two rows it creates and injects them into the spec before the scan, because a per-item route scanned against a made-up id only ever exercises the 404.

From 10 requests to 3,010. On the item URLs alone: 418 PATCH on the expiry route, 50 DELETE, and another 62 GETs and POSTs probing them, with real payloads: PHP injection via auto_prepend_file, Spring’s classLoader, query-string fuzzing. The per-item routes, exercised for the first time. The DAST job went from about two minutes to over four, and the new scan started report-only, with fail_action: false and a date next to it, the same way the baseline had. Eleven days later it had gated nothing and it was four of a PR’s eight minutes, so it moved to the weekly run. On the box the pipeline runs on now (the fourth post), the API scan alone was 241 of the DAST job’s 468 seconds; without it, a PR’s DAST is 111.

And it found a bug on its first run. Server Error response code, 58 times. The app was answering 500 to requests Fastify had already classified as the client’s fault: a DELETE with Content-Type: application/json and no body (Fastify throws FST_ERR_CTP_EMPTY_JSON_BODY with statusCode: 400), an unsupported media type (415), malformed JSON (400). My error handler sent everything it didn’t recognise to a generic 500 without looking at the statusCode already on the error. It wasn’t exploitable, but every client mistake was being logged as a server failure, and the error handler is the one place that decides what a client gets to see, so it was deciding wrong. The fix is one branch: an error carrying a 4xx statusCode answers with that code and { error: <code> }, no message. Everything else stays a generic 500, and a test throws db password is hunter2 from a route and asserts the body says internal and nothing else.

The whole arc, with a number at each step:

Step	Requests to the app	Per-item routes
Spider, no header	—	bounces off 401
Spider with header (where the first post left it)	6 URLs, ZAP’s count	0, the page shows the login gate
Pantry seeded	6 URLs, ZAP’s count	0, same cause
HttpOnly cookie	8 URLs, 10 requests	0, renders, clicks once, stops
Three minutes of spider	10 requests	0, identical
API-definition scan	3,010 requests	530

What this adds to the two posts before it: DAST coverage is measured by counting requests at the server, not by reading the scanner’s report. Four rounds looked right from ZAP’s side (green, 63 rules passed) and none of them had touched the routes that mattered. And when something finally did, what it found wasn’t a vulnerability but an error handler confusing “you got it wrong” with “I got it wrong”, the kind of thing that only shows up when something sends three thousand malformed requests on purpose.

And then the SBOM blocked a deploy. The record I’d moved to the release so it would outlive the ninety-day artifact failed on the first two releases that tried to carry it. On v0.23.0 it failed on a documented permission the job didn’t have, and because I’d put the step before the deploy, in the same job, the release was tagged and never shipped: a record of what shipped had stopped the shipping. I moved the step after the deploy, and on v0.23.1 it failed again on something else entirely (gh looking for a git remote in a job with no checkout; one --repo flag fixes it), but this time the release was already live and the run went red so the missing record would show. Same step, one bug each, and the difference between them isn’t the bug but the order: a record of what ships goes after the shipping, and when it fails, it fails in red. I then went to backfill the two missing records by redeploying those tags, and stopped, because the server had moved on to v0.23.1 and redeploying v0.23.0 would have rolled production back. Neither record gets added, it’s written down that the two releases before v0.23.2 don’t have one, and v0.23.2 attached its SBOM, generated from the same image the deploy had just put on the server. Three releases to get one step working: a permission, an order and a flag.

A note on what this cost. The second post measured the pipeline in CI seconds: 45 seconds to 2m10s of wall-clock. Those were GitHub-hosted numbers, and they didn’t survive the month. On the 28th GitHub stopped starting jobs and a release sat waiting for an image that was never coming: two ZAP scans on every PR, and the same two again on every push to main, had used up the 2,000 free Actions minutes a private repo gets a month. GitHub’s own meter put September’s usage for this repo at $12.09, all of it covered by the free allowance, and none of it available on the 28th. DAST was cheap in engineering and expensive in minutes, and that’s the fourth post. Even then, minutes weren’t the real cost. Nothing the SBOM took, two broken releases and a rollback avoided by luck, shows up in a table of timings. If you’re going to build something like this, the budget that matters isn’t runner minutes. It’s evenings.