← Back to scan
File as GitHub Issue repo: scrapy/scrapy

Push this scan report to scrapy/scrapy

Click the green button below to open GitHub’s new-issue form, pre-filled with the report title, summary table, top findings, and an embedded score-card image. No authentication needed — you review on GitHub before submitting. Repobility is credited as the scanner.

Embedded score card image

This image will render at the top of the issue body. Hosted on Repobility, refreshes automatically after re-scans.

Repobility score card

Issue title

Missing import: `queue` used but not imported

Curate findings to include

Pick exactly which findings appear in the issue body. By default the top 5 are included. Uncheck noise, check what matters.

Top 5 (default)
Severity Rule Title File:line
CRIT MINED018 [MINED018] Unsafe Deserialization Pickle: pickle.loads / yaml.load (without Loader=SafeLo… scrapy/squeues.py:152
CRIT MINED030 [MINED030] Python Pickle Loads: pickle.loads() can execute arbitrary code via __reduce__. scrapy/squeues.py:152
CRIT MINED030 [MINED030] Python Pickle Loads: pickle.loads() can execute arbitrary code via __reduce__. scrapy/extensions/spiderstate.py:44
CRIT SEC081 [SEC081] Python: pickle.loads / marshal.loads on untrusted data: pickle.load(s) and marsh… scrapy/extensions/spiderstate.py:44
CRIT MINED107 Missing import: `queue` used but not imported scrapy/utils/asyncio.py:114
CRIT MINED107 Missing import: `copy` used but not imported tests/test_settings/__init__.py:371
CRIT MINED107 Missing import: `queue` used but not imported scrapy/pqueues.py:404
CRIT private-key Identified a Private Key, which may compromise cryptographic security and sensitive data … tests/keys/mitmproxy-ca.pem:1
CRIT private-key Identified a Private Key, which may compromise cryptographic security and sensitive data … tests/keys/localhost.ip.key:1
CRIT private-key Identified a Private Key, which may compromise cryptographic security and sensitive data … tests/keys/example-com.key.pem:1
HIGH MINED006 [MINED006] Overcatch Baseexception: except BaseException: ... — prevents Ctrl+C and Syste… scrapy/utils/console.py:137
HIGH MINED004 [MINED004] Weak Crypto: MD5/SHA1/DES/RC4 used for security context (not just checksums). scrapy/utils/request.py:94
HIGH MINED004 [MINED004] Weak Crypto: MD5/SHA1/DES/RC4 used for security context (not just checksums). scrapy/utils/misc.py:139
HIGH MINED004 [MINED004] Weak Crypto: MD5/SHA1/DES/RC4 used for security context (not just checksums). scrapy/pipelines/images.py:248
HIGH SEC128 [SEC128] Async function without await — fire-and-forget Promise (AI mistake): Async call … scrapy/spiders/__init__.py:50
HIGH SEC128 [SEC128] Async function without await — fire-and-forget Promise (AI mistake): Async call … scrapy/pipelines/images.py:220
HIGH SEC128 [SEC128] Async function without await — fire-and-forget Promise (AI mistake): Async call … scrapy/loader/__init__.py:105
HIGH SEC035 [SEC035] Unbounded Resource Allocation — DoS risk: Allocating resources (buffers, recursi… scrapy/utils/request.py:134
HIGH SEC035 [SEC035] Unbounded Resource Allocation — DoS risk: Allocating resources (buffers, recursi… scrapy/pipelines/images.py:248
HIGH SEC035 [SEC035] Unbounded Resource Allocation — DoS risk: Allocating resources (buffers, recursi… scrapy/core/http2/agent.py:156
HIGH SEC029 [SEC029] Server-Side Request Forgery (SSRF) — outbound HTTP from user input: Outbound HTT… scrapy/http/request/form.py:69
HIGH SEC029 [SEC029] Server-Side Request Forgery (SSRF) — outbound HTTP from user input: Outbound HTT… scrapy/http/cookies.py:147
HIGH SEC029 [SEC029] Server-Side Request Forgery (SSRF) — outbound HTTP from user input: Outbound HTT… scrapy/commands/fetch.py:70
HIGH MINED036 [MINED036] Python Os System Call: os.system() invokes shell with no escaping. scrapy/commands/genspider.py:123
HIGH MINED036 [MINED036] Python Os System Call: os.system() invokes shell with no escaping. scrapy/commands/edit.py:48
HIGH MINED001 [MINED001] Bare Except Pass: except: pass or except Exception: pass — silently swallows e… scrapy/downloadermiddlewares/httpcompre…:39
HIGH MINED001 [MINED001] Bare Except Pass: except: pass or except Exception: pass — silently swallows e… scrapy/commands/genspider.py:205
HIGH MINED001 [MINED001] Bare Except Pass: except: pass or except Exception: pass — silently swallows e… scrapy/cmdline.py:178
HIGH MINED108 `self._beautify_newline` used but never assigned in __init__ scrapy/exporters.py:157
HIGH MINED108 `self._beautify_newline` used but never assigned in __init__ scrapy/exporters.py:154
HIGH MINED108 `self._beautify_newline` used but never assigned in __init__ scrapy/exporters.py:150
HIGH MINED108 `self.indent` used but never assigned in __init__ scrapy/exporters.py:142
HIGH MINED108 `self.encoding` used but never assigned in __init__ scrapy/exporters.py:123
HIGH MINED108 `self._get_serialized_fields` used but never assigned in __init__ scrapy/exporters.py:121
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:92
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:97
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:89
HIGH MINED108 `self.serialize_field` used but never assigned in __init__ scrapy/exporters.py:106
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:95
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:87
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:85
HIGH MINED108 `self.export_empty_fields` used but never assigned in __init__ scrapy/exporters.py:83
HIGH MINED108 `self.indent` used but never assigned in __init__ scrapy/exporters.py:54
HIGH MINED108 `self.export_empty_fields` used but never assigned in __init__ scrapy/exporters.py:53
HIGH MINED108 `self.fields_to_export` used but never assigned in __init__ scrapy/exporters.py:50
HIGH MINED108 `self.encoding` used but never assigned in __init__ scrapy/exporters.py:49
HIGH MINED108 `self._crawler` used but never assigned in __init__ scrapy/statscollectors.py:110
HIGH MINED108 `self._crawler` used but never assigned in __init__ scrapy/statscollectors.py:109
HIGH MINED108 `self._persist_stats` used but never assigned in __init__ scrapy/statscollectors.py:97
HIGH MINED108 `self._create_sender_factory` used but never assigned in __init__ scrapy/mail.py:210
HIGH MINED108 `self._sent_failed` used but never assigned in __init__ scrapy/mail.py:163
HIGH MINED108 `self._sent_ok` used but never assigned in __init__ scrapy/mail.py:162
HIGH MINED108 `self._sendmail` used but never assigned in __init__ scrapy/mail.py:159
HIGH MINED115 Action `pypa/gh-action-pypi-publish` pinned to mutable ref `@release/v1` .github/workflows/publish.yml:29
HIGH MINED115 Action `actions/setup-python` pinned to mutable ref `@v6` .github/workflows/publish.yml:22
HIGH MINED115 Action `actions/checkout` pinned to mutable ref `@v6` .github/workflows/publish.yml:21
HIGH MINED115 Action `codecov/codecov-action` pinned to mutable ref `@v5` .github/workflows/tests-ubuntu.yml:111
HIGH MINED115 Action `codecov/codecov-action` pinned to mutable ref `@v5` .github/workflows/tests-ubuntu.yml:107
HIGH MINED115 Action `actions/setup-python` pinned to mutable ref `@v6` .github/workflows/tests-ubuntu.yml:90
HIGH MINED115 Action `actions/checkout` pinned to mutable ref `@v6` .github/workflows/tests-ubuntu.yml:87
HIGH MINED115 Action `codecov/codecov-action` pinned to mutable ref `@v5` .github/workflows/tests-macos.yml:48
HIGH MINED115 Action `codecov/codecov-action` pinned to mutable ref `@v5` .github/workflows/tests-macos.yml:44
HIGH MINED115 Action `actions/setup-python` pinned to mutable ref `@v6` .github/workflows/tests-macos.yml:33
HIGH MINED115 Action `actions/checkout` pinned to mutable ref `@v6` .github/workflows/tests-macos.yml:30
HIGH MINED115 Action `codecov/codecov-action` pinned to mutable ref `@v5` .github/workflows/tests-windows.yml:75
HIGH MINED115 Action `codecov/codecov-action` pinned to mutable ref `@v5` .github/workflows/tests-windows.yml:71
HIGH MINED115 Action `actions/setup-python` pinned to mutable ref `@v6` .github/workflows/tests-windows.yml:60
HIGH MINED115 Action `actions/checkout` pinned to mutable ref `@v6` .github/workflows/tests-windows.yml:57
HIGH MINED115 Action `pre-commit/action` pinned to mutable ref `@v3.0.1` .github/workflows/checks.yml:58
HIGH MINED115 Action `actions/checkout` pinned to mutable ref `@v6` .github/workflows/checks.yml:57
HIGH MINED115 Action `actions/setup-python` pinned to mutable ref `@v6` .github/workflows/checks.yml:44
HIGH MINED115 Action `actions/checkout` pinned to mutable ref `@v6` .github/workflows/checks.yml:41
HIGH MINED115 Action `actions/github-script` pinned to mutable ref `@v6` .github/workflows/auto-close-llm-pr.yml:14
HIGH MINED121 requirements.txt installs from `sphinx-scrapy @ git+https://github.com/scrapy/sphi...` (g… docs/requirements.txt:156
HIGH MINED121 requirements.txt installs from `sphinx-markdown-builder @ git+https://github.com/z...` (g… docs/requirements.txt:146
HIGH MINED121 requirements.txt installs from `sphinx-llms-txt @ git+https://github.com/zytedata/...` (g… docs/requirements.txt:144
HIGH MINED131 pre-commit hook `https://github.com/scrapy/sphinx-scrapy` pinned to mutable rev `0.8.6` .pre-commit-config.yaml:29
HIGH MINED131 pre-commit hook `https://github.com/sphinx-contrib/sphinx-lint` pinned to mutable rev `v1… .pre-commit-config.yaml:25
HIGH MINED131 pre-commit hook `https://github.com/pre-commit/pre-commit-hooks` pinned to mutable rev `v… .pre-commit-config.yaml:20
HIGH MINED131 pre-commit hook `https://github.com/adamchainz/blacken-docs` pinned to mutable rev `1.20.… .pre-commit-config.yaml:14
HIGH MINED131 pre-commit hook `https://github.com/astral-sh/ruff-pre-commit` pinned to mutable rev `v0.… .pre-commit-config.yaml:8
HIGH PYSEC-2026-142 urllib3: PYSEC-2026-142 docs/requirements.txt
HIGH PYSEC-2026-141 urllib3: PYSEC-2026-141 docs/requirements.txt
HIGH PYSEC-2026-160 twisted: PYSEC-2026-160 docs/requirements.txt
HIGH PYSEC-2017-83 scrapy: PYSEC-2017-83 docs/requirements.txt
HIGH PYSEC-2026-87 lxml: PYSEC-2026-87 docs/requirements.txt
HIGH PYSEC-2026-36 cryptography: PYSEC-2026-36 docs/requirements.txt
HIGH SEC020 [SEC020] Secret Printed to Logs: Debug or diagnostic code appears to print a credential-b… scrapy/extensions/telnet.py:64
HIGH SEC005 [SEC005] Command Injection Risk: Unsafe shell execution or eval of user input. scrapy/commands/edit.py:48
HIGH MINED112 FastAPI PATCH scrapy.settings.default_settings has no auth tests/test_settings/__init__.py:595
HIGH MINED112 FastAPI PATCH scrapy.settings.default_settings has no auth tests/test_settings/__init__.py:586
HIGH MINED112 FastAPI PATCH scrapy.settings.default_settings has no auth tests/test_settings/__init__.py:575
HIGH MINED112 FastAPI PATCH scrapy.settings.default_settings has no auth tests/test_settings/__init__.py:563
MED SEC107 [SEC107] Weak TLS version requested (TLSv1.0, TLSv1.1, SSLv3, SSLv2): TLS 1.0 and 1.1 wer… scrapy/utils/ssl.py:26
MED SEC014 [SEC014] SSL Verification Disabled: SSL certificate verification is disabled, allowing ma… scrapy/utils/ssl.py:51
MED SEC034 [SEC034] Log Injection / Log Forging — unsanitized user input in log: User input is logge… scrapy/utils/_download_handlers.py:86
MED SEC007 [SEC007] Unsafe Deserialization: Unsafe deserialization can execute arbitrary code. scrapy/extensions/spiderstate.py:44
MED SEC127 [SEC127] AI agent stub — TODO: implement / pass placeholder body: Function body left as T… scrapy/resolver.py:78
MED SEC127 [SEC127] AI agent stub — TODO: implement / pass placeholder body: Function body left as T… scrapy/middleware.py:84
MED SEC127 [SEC127] AI agent stub — TODO: implement / pass placeholder body: Function body left as T… scrapy/core/downloader/handlers/base.py:28
MED MINED111 Bare except continues silently scrapy/core/downloader/middleware.py:83
MED MINED111 Bare except continues silently scrapy/core/downloader/__init__.py:259
MED MINED111 Bare except continues silently scrapy/pipelines/media.py:216
MED MINED111 Bare except continues silently scrapy/pipelines/files.py:416
MED MINED111 Bare except continues silently scrapy/utils/defer.py:379
MED MINED111 Bare except continues silently scrapy/utils/defer.py:360
MED MINED111 Bare except continues silently scrapy/utils/defer.py:440
MED MINED111 Bare except continues silently scrapy/utils/defer.py:154
MED MINED111 Bare except continues silently scrapy/utils/engine.py:36
MED MINED111 Bare except continues silently scrapy/utils/deprecate.py:128
MED MINED111 Bare except continues silently scrapy/extensions/httpcache.py:418
MED MINED111 Bare except continues silently scrapy/core/spidermw.py:96
MED MINED111 Bare except continues silently scrapy/core/spidermw.py:226
MED MINED111 Bare except continues silently scrapy/core/spidermw.py:110
MED MINED111 Bare except continues silently scrapy/core/scraper.py:258
MED MINED111 Bare except continues silently scrapy/core/scraper.py:290
MED MINED111 Bare except continues silently scrapy/contracts/__init__.py:78
MED MINED111 Bare except continues silently scrapy/contracts/__init__.py:48
MED MINED111 Bare except continues silently scrapy/contracts/__init__.py:187
MED MINED111 Bare except continues silently scrapy/contracts/__init__.py:131
MED COMP001 [COMP001] High cognitive complexity: Function `main` has cognitive complexity 18 (SonarSo… docs/utils/linkfix.py:20
MED DEPCUR-PY Python package `twisted` is 1 major version(s) behind (25.5.0 -> 26.4.0) docs/requirements.txt:178
MED DEPCUR-PY Python package `service-identity` is 2 major version(s) behind (24.2.0 -> 26.1.0) docs/requirements.txt:125
MED GHSA-65pc-fj4g-8rjx idna: GHSA-65pc-fj4g-8rjx docs/requirements.txt
MED SEC005 [SEC005] Command Injection Risk: Unsafe shell execution or eval of user input. scrapy/commands/genspider.py:123
LOW COMP001 [COMP001] High cognitive complexity: Function `execute` has cognitive complexity 10 (Sona… scrapy/cmdline.py:169
LOW COMP001 [COMP001] High cognitive complexity: Function `load_settings` has cognitive complexity 8 … scrapy/addons.py:25
LOW DEPCUR-PY Python package `zope-interface` is minor version(s) behind (8.2 -> 8.5) docs/requirements.txt:194
LOW DEPCUR-PY Python package `urllib3` is minor version(s) behind (2.6.3 -> 2.7.0) docs/requirements.txt:188
LOW DEPCUR-PY Python package `snowballstemmer` is minor version(s) behind (3.0.1 -> 3.1.1) docs/requirements.txt:127
LOW DEPCUR-PY Python package `scrapy` is minor version(s) behind (2.14.2 -> 2.16.0) docs/requirements.txt:121
LOW DEPCUR-PY Python package `requests` is minor version(s) behind (2.33.0 -> 2.34.2) docs/requirements.txt:112
LOW DEPCUR-PY Python package `pyopenssl` is minor version(s) behind (26.0.0 -> 26.2.0) docs/requirements.txt:108
LOW DEPCUR-PY Python package `pygments` is minor version(s) behind (2.19.2 -> 2.20.0) docs/requirements.txt:106
LOW DEPCUR-PY Python package `pydantic` is minor version(s) behind (2.12.5 -> 2.13.4) docs/requirements.txt:98
LOW DEPCUR-PY Python package `packaging` is minor version(s) behind (26.0 -> 26.2) docs/requirements.txt:76
LOW DEPCUR-PY Python package `idna` is minor version(s) behind (3.11 -> 3.18) docs/requirements.txt:49
LOW DEPCUR-PY Python package `filelock` is minor version(s) behind (3.25.2 -> 3.29.1) docs/requirements.txt:39
LOW DEPCUR-PY Python package `docutils` is minor version(s) behind (0.22.4 -> 0.23) docs/requirements.txt:34
LOW DEPCUR-PY Python package `certifi` is minor version(s) behind (2026.2.25 -> 2026.5.20) docs/requirements.txt:15
LOW GHSA-5239-wwwm-4pmq pygments: GHSA-5239-wwwm-4pmq docs/requirements.txt
LOW AIC003 Duplicated implementation block across source files tests/CrawlerRunner/custom_loop_same.py:1
LOW AIC003 Duplicated implementation block across source files tests/CrawlerProcess/reactor_select_sub…:10
LOW AIC003 Duplicated implementation block across source files tests/CrawlerProcess/caching_hostname_r…:4
LOW AIC003 Duplicated implementation block across source files tests/CrawlerProcess/asyncio_enabled_re…:12
LOW AIC003 Duplicated implementation block across source files tests/CrawlerProcess/asyncio_enabled_re…:38
LOW AIC003 Duplicated implementation block across source files tests/CrawlerProcess/asyncio_enabled_re…:4
LOW AIC003 Duplicated implementation block across source files tests/CrawlerProcess/asyncio_deferred_s…:6
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerRunner/multi_seq.py:2
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerRunner/custom_loop_sa…:2
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerProcess/asyncio_enabl…:11
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerProcess/asyncio_enabl…:10
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerProcess/asyncio_enabl…:9
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerProcess/asyncio_enabl…:25
LOW AIC003 Duplicated implementation block across source files tests/AsyncCrawlerProcess/asyncio_custo…:1
LOW AIC003 Duplicated implementation block across source files scrapy/http/response/text.py:166
INFO MINED062 [MINED062] Python Dataclass No Fields: @dataclass over an empty class — unfinished model. scrapy/core/downloader/__init__.py:44
INFO MINED072 [MINED072] Python Pass Only Class: class Foo: pass — stub waiting to be filled in. docs/_ext/scrapydocs.py:19
INFO MINED050 [MINED050] Stub Only Function: Function declared but body is just pass, return None, rais… scrapy/cmdline.py:179
INFO MINED050 [MINED050] Stub Only Function: Function declared but body is just pass, return None, rais… extras/qpsclient.py:55
INFO MINED050 [MINED050] Stub Only Function: Function declared but body is just pass, return None, rais… docs/_ext/scrapydocs.py:20
INFO MINED043 [MINED043] Http Not Https: Hardcoded http:// (not localhost) for endpoints that handle cr… scrapy/contracts/default.py:19
INFO MINED043 [MINED043] Http Not Https: Hardcoded http:// (not localhost) for endpoints that handle cr… docs/conf.py:95
INFO MINED043 [MINED043] Http Not Https: Hardcoded http:// (not localhost) for endpoints that handle cr… docs/_ext/scrapydocs.py:159
INFO DEPCUR-PY Python package `charset-normalizer` is patch version(s) behind (3.4.6 -> 3.4.7) docs/requirements.txt:19
Reset to top 5 165 findings available (after auto-suppression of test files + won't-fix)

Issue body (markdown)

## Code-quality scan: `scrapy/scrapy`

**Score: 67/100 (B)**  ·  208 findings  ·  scanned 2026-06-05 09:27 UTC  ·  76,750 LOC

| Severity | Count |
|---|---|
| CRITICAL | 10 |
| HIGH | 83 |
| MEDIUM | 32 |
| LOW | 31 |

📊 [Full filterable report](https://repobility.com/scan/9db5f11e-57f3-477a-9bb0-dfae01c72bf5/)  ·  ![scorecard](https://repobility.com/scan/9db5f11e-57f3-477a-9bb0-dfae01c72bf5/report.png?v=1780651653-s2)

### Top findings

1. **CRITICAL** `MINED018` — Unsafe Deserialization Pickle
   `scrapy/squeues.py:152` · CWE-502 · ✓ Repobility
2. **CRITICAL** `MINED030` — Python Pickle Loads
   `scrapy/squeues.py:152` · CWE-502 · ✓ Repobility
3. **CRITICAL** `MINED030` — Python Pickle Loads
   `scrapy/extensions/spiderstate.py:44` · CWE-502 · ✓ Repobility
4. **CRITICAL** `SEC081` — Python: pickle.loads / marshal.loads on untrusted data
   `scrapy/extensions/spiderstate.py:44` · A05:2021 Security Misconfiguration
5. **CRITICAL** `MINED107` — Missing import: `queue` used but not imported
   `scrapy/utils/asyncio.py:114` · ✓ Repobility

---

**Security note**: this issue is public. If any flagged finding is a real, exploitable vulnerability, please redirect to your `SECURITY.md` policy or open a [private security advisory](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability) instead. We're happy to close this and re-submit privately.

---

_Filed automatically. Close this issue if not useful — we won't refile. Full report: https://repobility.com/scan/9db5f11e-57f3-477a-9bb0-dfae01c72bf5/_
Already filed
This repo publishes a SECURITY.md policy and the scan contains 12 Critical/High security finding(s). Public issue filing would violate coordinated disclosure. Submit privately via the project's security reporting channel.
Megaproject â high spam risk
Could not determine 'scrapy/scrapy' star count (GitHub API rate-limited or unreachable). When in doubt about repo size, prefer opening a focused PR or a discussion rather than an issue.
Already filed
117/217 findings (54%) on this scan are already flagged as test-file, won't-fix, or suppressed. The scan is too noisy to file as a single issue. Curate down to specific actionable findings, or address the FP source first.

The button opens GitHubâs new-issue page in a new tab. You will see the title + body pre-filled â review, edit if you want, then click GitHubâs "Submit new issue" button. Repobility never posts anything on your behalf.

For real security findings on big repos: use the project's SECURITY.md or private advisory flow instead of a public issue.