Opus 4.7’s Blind Spot: Tests Exist, CI Does Not

Grade every Opus 4.7 repo through an LLM and the same warnings surface over and over:

Seven of the top ten flagged risks are some variant of “tests aren’t there” or “CI isn’t there.”

But tests are there

Let’s cross-check with the file-level data. Across 9,281 Opus 4.7 repos:

• 64,556 test files exist (role = test)
• 394 repos pull in Vitest as a framework
• 344 repos have pytest
• 95 repos have Jest

That’s actual test code. 9% of all analyzed files are in the test role — not bad.

So where’s the gap?

The LLM’s complaint isn’t “no test files.” It’s “no CI that runs them.”

And that checks out. Look at signals for CI infrastructure:

• .github/workflows/ directories: rare in our file index — under 5% of repos have them
• Dockerfile: 867 repos (9%)
• .env.example: 1,076 repos (12%)
• docker-compose.yml: 495 repos (5%)

Most Opus 4.7 repos set up the test runner (installing Vitest, adding a test script) but skip the GitHub Actions workflow that runs it automatically.

Why the gap exists

Three hypotheses:

1. One-shot prompts favor “write the feature”

When a user asks “build me a chat app,” the model builds the app. CI is boring plumbing that doesn’t show up in demos. It gets left for “later.”

2. Agent loops don’t reach CI

Even with multi-turn agents, the iteration loop usually terminates when the feature works locally. The agent sees “test passes locally” and moves on.

3. CI requires repo-level context

Writing .github/workflows/ci.yml needs knowing the test command, the build command, the deploy target. Many of these are implicit to the human maintainer but not written down.

The fix is simple

Opus 4.7 already writes the tests. The post-generation step we recommend in any “build me X” product:

# .github/workflows/ci.yml
name: CI
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm test

Add this file automatically. Done. You’ve closed the single largest blind spot.

Dependency hygiene

The other frequent complaint — “potentially outdated dependencies” (26 repos, 7% of sample) — is harder. Opus 4.7 is trained on a point-in-time snapshot of npm/PyPI; it doesn’t know your dependency versions are stale. This is a continuous problem, not a one-shot one.

A Dependabot config (10 lines of YAML) covers 80% of this:

version: 2
updates:
  - package-ecosystem: npm
    directory: /
    schedule:
      interval: weekly

Security: auth is the pain point

“Potential security vulnerabilities in authentication” was flagged in 14 repos. The specific complaints usually center on:
- JWT without proper expiry/refresh handling
- Passwords hashed with weak or missing salt
- Session cookies without HttpOnly + SameSite
- Direct use of Math.random() for tokens

This is textbook LLM-written-auth territory and worth a dedicated post.

Summary: Opus 4.7 writes tests. Opus 4.7 writes docs. Opus 4.7 often does not wire either into automation. A tiny amount of automated post-generation scaffolding closes 70% of the gap.