Public scan — anyone with this URL can view this analysis. Sign up to track your own repos privately, run scheduled re-scans, and get AI fix prompts via your dashboard.

UKGovernmentBEIS/inspect_ai

https://github.com/UKGovernmentBEIS/inspect_ai · scanned 2026-06-16 03:49 UTC (2 months ago)

994 raw signals (0 security + 994 graph)

File as issue Image

UNIFIED Repobility · multi-layer engine · AI coders

Complete repo analysis

Last scanned 2 months ago · v5 · last Δ +7.0 (diff) · 420 actionable findings from 1 signal source. 58 repeated signals grouped for readability. Security checks, system graph analysis, and verified AI-agent feedback are merged into one review queue.

JSON

88.9% cov · 420 findings

Severity distribution — click a segment to filter

Active filters: excluding tests × Reset all

Severity: Critical 1 High 7 Medium 43 Low 180 9-Layer: Software Security Quality Integrity Frontend Hardware Data Network Cicd Source: Security checks 0 System graph 420 Crowd 0 Layer: Software 23 Network 1 Hardware 22 Security 12 Cicd 3 Quality 359

Exclude dismissed / FP Exclude test files

Bug-class explainers. Each card groups findings of the same shape — these are the patterns most likely to ship to prod and reappear in future scans unless you systematically fix the cause, not just the instance.

Duplicates & near-duplicates 4 findings

What it is: Same function copy-pasted into multiple modules with minor variations.

Why it matters: Each copy drifts independently — bug fixes apply to one, miss the others.

How AI causes it: AI completes the same pattern in each file rather than refactoring to a shared helper.

Fix approach: Extract the duplicated logic into the most general module both call sites already import. Add tests at the helper level.

4 matching findings on this repo

low Near-duplicate function bodies in 2 places repo-level
low Near-duplicate function bodies in 4 places
low Near-duplicate function bodies in 3 places repo-level
low Near-duplicate function bodies in 5 places

View all duplicates & near-duplicates findings →

Legacy markers 6 findings

What it is: TODO, FIXME, XXX, HACK comments. Often indicate a known-broken path the author meant to fix.

Why it matters: Each marker is an unfinished thought. Production code shouldn't ship with debt that's documented but not tracked.

How AI causes it: AI mirrors the style of the codebase, so existing TODOs propagate into new code.

Fix approach: Convert each into a ticket. Delete the comment when the ticket lands. Use a pre-commit hook to block new TODOs without an issue link.

6 matching findings on this repo

low Old/deprecated-named symbol `test_api_key_override_falls_back_to_legacy` in tes…
low Old/deprecated-named symbol `override_api_key_legacy` in src/inspect_ai/hooks/_…
low Old/deprecated-named symbol `send_telemetry_legacy` in src/inspect_ai/hooks/_le…
info Commented-code block (7 lines) in src/inspect_ai/hooks/_legacy.py:26
low Old/deprecated-named symbol `send_telemetry_legacy` in src/inspect_ai/model/_mo…
low Old/deprecated-named symbol `send_telemetry_legacy` in src/inspect_ai/_eval/tas…

View all legacy markers findings →

Commented-out code 188 findings

What it is: Lines of source that were intentionally disabled but never deleted.

Why it matters: Git already remembers history — commented code rots, becomes wrong, and adds noise to diffs.

How AI causes it: AI sometimes comments out broken code instead of fixing it. Reviewers approve out of inertia.

Fix approach: Delete. Trust `git log`. If you really need to remember, save it in a notes file under `docs/`.

12 matching findings on this repo

info Commented-code block (5 lines) in tests/test_fail_on_error.py:130
info Commented-code block (5 lines) in tests/test_score_on_error.py:188
info Commented-code block (6 lines) in tests/test_eval_set.py:497
info Commented-code block (5 lines) in tests/test_eval_set_scanner.py:206
info Commented-code block (5 lines) in tests/display/test_sample_toolbar.py:66
info Commented-code block (12 lines) in tests/util/test_notify.py:307
info Commented-code block (5 lines) in tests/util/test_limit.py:135
info Commented-code block (13 lines) in tests/util/test_text.py:18
info Commented-code block (5 lines) in tests/util/test_limit_working.py:166
info Commented-code block (5 lines) in tests/util/test_limit_token.py:154
info Commented-code block (5 lines) in tests/agent/test_bridge_tool_search.py:139
info Commented-code block (5 lines) in tests/agent/test_agent_compaction_native.py:34

View all commented-out code findings →

Config drift 1 finding

What it is: Settings duplicated across env files, Docker compose, K8s, and code defaults, all with slightly different values.

Why it matters: Production behaviour depends on whichever copy your loader reads first. Subtle bugs in staging that don't reproduce in dev.

How AI causes it: AI writes new config from memory rather than reading the existing source.

Fix approach: Pick one source of truth (env vars + a settings module). Have every other place import from there. Lint for duplicates in CI.

1 matching finding on this repo

info Commented-code block (6 lines) in src/inspect_ai/agent/_acp/_config.py:13

View all config drift findings →

For AI agents: Voting guide (TP/FP) MCP manifest Stdio wrapper SARIF Integrate Findings queue Vote TP/FP on findings to calibrate the engine.

For AI agents + API integrations

Email me when this repo regresses

Free. We re-scan periodically; new criticals → your inbox. No signup required for the scan itself.

API access

This page is publicly accessible at: https://repobility.com/scan/51bf3298-1f85-4d62-a4d4-61afab25c50e/

To check status programmatically (no auth required):

curl -s https://repobility.com/api/v1/public/scan/51bf3298-1f85-4d62-a4d4-61afab25c50e/

Important — please don't re-submit the same URL repeatedly. The submission endpoint is idempotent: re-submitting the same git URL returns this same scan_token, not a new one. To re-scan this repo, sign up free and use the dashboard.