scrapy/scrapy

Public scan — anyone with this URL can view this analysis. Sign up to track your own repos privately, run scheduled re-scans, and get AI fix prompts via your dashboard.

145 of your 223 findings came from Repobility's proprietary detections. ✓ Repobility tags below mark them.

Scan timing: clone 2.94s · analysis 51.7s · 4.6 MB · GitHub API rate-limit (preflight)

https://github.com/scrapy/scrapy · scanned 2026-06-05 09:27 UTC (5 days, 18 hours ago) · 10 languages

438 raw signals (208 security + 230 graph) 63rd percentile · Python · medium (20-100K LoC) System graph score 74 (lower by 2)

File as issue Image

UNIFIED Repobility · multi-layer engine · AI coders

Complete repo analysis

Last scanned 5 days, 18 hours ago · v2 · 182 actionable findings from 2 signal sources. 140 repeated signals grouped for readability. Security checks, system graph analysis, and verified AI-agent feedback are merged into one review queue.

JSON

88.9% cov · 87 findings

Score breakdown â 2026-05-18-v5

Component	Sub-score	Weight	Contribution
`structure_score`	60.0	0.15	9.00
`security_score`	46.0	0.25	11.50
`testing_score`	100.0	0.20	20.00
`documentation_score`	94.0	0.15	14.10
`practices_score`	70.0	0.15	10.50
`code_quality`	69.3	0.10	6.93
Overall		1.00	72.0

Severity distribution — click a segment to filter

Active filters: excluding tests × Reset all

Severity: Critical 5 High 16 Medium 11 Low 89 9-Layer: Software Security Quality Integrity Frontend Hardware Data Network Cicd Source: Security checks 95 System graph 87 Crowd 0 Layer: Software 70 Security 12 Quality 95 Cicd 3 Api 1 Frontend 1

Exclude dismissed / FP Exclude test files

Bug-class explainers. Each card groups findings of the same shape — these are the patterns most likely to ship to prod and reappear in future scans unless you systematically fix the cause, not just the instance.

Duplicates & near-duplicates 4 findings

What it is: Same function copy-pasted into multiple modules with minor variations.

Why it matters: Each copy drifts independently — bug fixes apply to one, miss the others.

How AI causes it: AI completes the same pattern in each file rather than refactoring to a shared helper.

Fix approach: Extract the duplicated logic into the most general module both call sites already import. Add tests at the helper level.

4 matching findings on this repo

low Near-duplicate function bodies in 2 places repo-level
low Near-duplicate function bodies in 9 places
low Near-duplicate function bodies in 3 places repo-level
low Near-duplicate function bodies in 4 places repo-level

View all duplicates & near-duplicates findings →

Legacy markers 1 finding

What it is: TODO, FIXME, XXX, HACK comments. Often indicate a known-broken path the author meant to fix.

Why it matters: Each marker is an unfinished thought. Production code shouldn't ship with debt that's documented but not tracked.

How AI causes it: AI mirrors the style of the codebase, so existing TODOs propagate into new code.

Fix approach: Convert each into a ticket. Delete the comment when the ticket lands. Use a pre-commit hook to block new TODOs without an issue link.

1 matching finding on this repo

low Old/deprecated-named symbol `TestHttpAuthMiddlewareLegacy` in tests/test_downlo…

View all legacy markers findings →

Commented-out code 10 findings

What it is: Lines of source that were intentionally disabled but never deleted.

Why it matters: Git already remembers history — commented code rots, becomes wrong, and adds noise to diffs.

How AI causes it: AI sometimes comments out broken code instead of fixing it. Reviewers approve out of inertia.

Fix approach: Delete. Trust `git log`. If you really need to remember, save it in a notes file under `docs/`.

10 matching findings on this repo

info Commented-code block (6 lines) in tests/test_downloader_handlers_http_base.py:66
info Commented-code block (5 lines) in tests/test_downloader_handler_twisted_http2.p…
info Commented-code block (5 lines) in tests/test_command_runspider.py:99
info Commented-code block (7 lines) in scrapy/crawler.py:899
info Commented-code block (6 lines) in scrapy/shell.py:45
info Commented-code block (5 lines) in scrapy/resolver.py:57
info Commented-code block (6 lines) in scrapy/pipelines/media.py:239
info Commented-code block (9 lines) in scrapy/spiders/sitemap.py:139
info Commented-code block (6 lines) in scrapy/core/downloader/handlers/__init__.py:33
info Commented-code block (6 lines) in scrapy/core/http2/stream.py:216

View all commented-out code findings →

For AI agents: Voting guide (TP/FP) MCP manifest Stdio wrapper SARIF Integrate Findings queue Vote TP/FP on findings to calibrate the engine.

For AI agents + API integrations

Email me when this repo regresses

Free. We re-scan periodically; new criticals → your inbox. No signup required for the scan itself.

API access

This page is publicly accessible at: https://repobility.com/scan/9db5f11e-57f3-477a-9bb0-dfae01c72bf5/

To check status programmatically (no auth required):

curl -s https://repobility.com/api/v1/public/scan/9db5f11e-57f3-477a-9bb0-dfae01c72bf5/

Important — please don't re-submit the same URL repeatedly. The submission endpoint is idempotent: re-submitting the same git URL returns this same scan_token, not a new one. To re-scan this repo, sign up free and use the dashboard.