Public scan — anyone with this URL can view this analysis. Sign up to track your own repos privately, run scheduled re-scans, and get AI fix prompts via your dashboard.

UKGovernmentBEIS/inspect_evals

https://github.com/UKGovernmentBEIS/inspect_evals · scanned 2026-06-16 22:23 UTC (1 month, 2 weeks ago)

1275 raw signals (0 security + 1275 graph)

UNIFIED Repobility · multi-layer engine · AI coders

Complete repo analysis

Last scanned 1 month, 2 weeks ago · v3 · 543 actionable findings from 1 signal source. 100 repeated signals grouped for readability. Security checks, system graph analysis, and verified AI-agent feedback are merged into one review queue.

JSON
Severity distribution — click a segment to filter
Active filters: excluding tests × Reset all

All 16011 nodes from the latest scan, grouped by kind. Each node is a unit the engine identified (file, function, endpoint, table…). Most users won't need this view — it's primarily for debugging the engine's graph extraction or for AI agents that want to enumerate the project structure.

LabelLayerStatusPath
_fake_run software healthy tests/test_dep_bump_impact.py:32
test_invokes_uv_tree_with_all_groups software healthy tests/test_dep_bump_impact.py:37
test_detects_dev_group_for_types_pyyaml software healthy tests/test_dep_bump_impact.py:48
test_types_pyyaml_reported_as_dev_tooling_not_no_impact software healthy tests/test_dep_bump_impact.py:66
test_epoch_metadata_reduction_issue software healthy tests/test_epoch_compatibility.py:47
value_based_mean software healthy tests/test_epoch_compatibility.py:56
calc software healthy tests/test_epoch_compatibility.py:71
metadata_based_mean software healthy tests/test_epoch_compatibility.py:68
varying_scorer software healthy tests/test_epoch_compatibility.py:85
score software healthy tests/test_epoch_compatibility.py:86
_hf_telemetry_enabled software healthy tests/conftest.py:46
pytest_addoption software healthy tests/conftest.py:54
pytest_configure software healthy tests/conftest.py:85
pytest_collection_modifyitems software healthy tests/conftest.py:116
pytest_runtest_setup software healthy tests/conftest.py:182
pytest_runtest_teardown software healthy tests/conftest.py:188
set_model_roles software healthy tests/conftest.py:195
pytest_runtest_makereport software healthy tests/conftest.py:208
pytest_sessionfinish software healthy tests/conftest.py:229
test_dry_run_does_not_delete software healthy tests/test_clean.py:8
test_delete_file_and_dir software healthy tests/test_clean.py:21
test_delete_path_if_exists_raises_for_missing_path software healthy tests/test_clean.py:34
test_rm_from_dir software healthy tests/test_clean.py:43
test_rm_from_dir_no_dir software healthy tests/test_clean.py:59
test_rm_from_dir_empty_dir software healthy tests/test_clean.py:65
test_registry_imports software healthy tests/test_registry_imports.py:7
test_registry_exposes_every_eval_yaml_task software healthy tests/test_registry_imports.py:12
test_direct_eval_call software healthy tests/test_autolint_checks.py:20
test_module_eval_call software healthy tests/test_autolint_checks.py:26
test_no_eval_call software healthy tests/test_autolint_checks.py:32
test_eval_in_function software healthy tests/test_autolint_checks.py:38
test_different_eval_not_detected software healthy tests/test_autolint_checks.py:48
test_simple_package software healthy tests/test_autolint_checks.py:58
test_package_with_version software healthy tests/test_autolint_checks.py:62
test_package_with_extras software healthy tests/test_autolint_checks.py:68
test_package_with_environment_marker software healthy tests/test_autolint_checks.py:72
test_package_with_trailing_space software healthy tests/test_autolint_checks.py:76
test_empty_string software healthy tests/test_autolint_checks.py:80
test_complex_specifier software healthy tests/test_autolint_checks.py:84
test_returns_frozenset software healthy tests/test_autolint_checks.py:92
test_contains_common_modules software healthy tests/test_autolint_checks.py:97
test_contains_typing_extensions software healthy tests/test_autolint_checks.py:107
test_does_not_contain_third_party software healthy tests/test_autolint_checks.py:112
test_simple_import software healthy tests/test_autolint_checks.py:123
test_from_import software healthy tests/test_autolint_checks.py:131
test_nested_import software healthy tests/test_autolint_checks.py:139
test_multiple_imports software healthy tests/test_autolint_checks.py:147
test_aliased_import software healthy tests/test_autolint_checks.py:163
test_finds_task_function software healthy tests/test_autolint_checks.py:176
test_finds_multiple_task_functions software healthy tests/test_autolint_checks.py:189

Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.

LabelLayerStatusPath
TASK_VERSIONING.md software healthy TASK_VERSIONING.md
CLAUDE.md software healthy CLAUDE.md
AGENTS.md software healthy AGENTS.md
ASSETS.yaml software healthy ASSETS.yaml
MAINTAINERS.md software healthy MAINTAINERS.md
CONTRIBUTING.md software healthy CONTRIBUTING.md
CHANGELOG.md software healthy CHANGELOG.md
AUTOMATED_CHECKS.md software healthy AUTOMATED_CHECKS.md
.markdownlint.yaml software healthy .markdownlint.yaml
README.md software healthy README.md
pyproject.toml software healthy pyproject.toml
Makefile software healthy Makefile
LICENSE software healthy LICENSE
NOTICE software healthy NOTICE
.pre-commit-config.yaml software healthy .pre-commit-config.yaml
PACKAGE_VERSIONING.md software healthy PACKAGE_VERSIONING.md
EVALUATION_CHECKLIST.md software healthy EVALUATION_CHECKLIST.md
BEST_PRACTICES.md software healthy BEST_PRACTICES.md
EVAL_REGISTER.md software healthy EVAL_REGISTER.md
pyproject.toml software healthy packages/kernelbench/pyproject.toml
pyproject.toml software healthy packages/cve_bench/pyproject.toml
pyproject.toml software healthy packages/bold/pyproject.toml
pyproject.toml software healthy packages/livebench/pyproject.toml
pyproject.toml software healthy packages/mle_bench/pyproject.toml
pyproject.toml software healthy packages/novelty_bench/pyproject.toml
pyproject.toml software healthy packages/abstention_bench/pyproject.toml
test_dep_bump_impact.py software healthy tests/test_dep_bump_impact.py
test_epoch_compatibility.py software healthy tests/test_epoch_compatibility.py
conftest.py software healthy tests/conftest.py
test_clean.py software healthy tests/test_clean.py
test_registry_imports.py software healthy tests/test_registry_imports.py
__init__.py software healthy tests/__init__.py
test_autolint_checks.py software healthy tests/test_autolint_checks.py
README.md software healthy tests/README.md
test_generate_readmes.py software healthy tests/test_generate_readmes.py
test_datasets_hf.py software healthy tests/test_datasets_hf.py
test_xdist_grouping.py software healthy tests/test_xdist_grouping.py
test_eval_imports.py software healthy tests/test_eval_imports.py
test_utils.py software healthy tests/test_utils.py
test_autolint_best_practices.py software healthy tests/test_autolint_best_practices.py
test_check_posix_code.py software healthy tests/test_check_posix_code.py
test_metadata.py software healthy tests/test_metadata.py
test_prerender_links.py software healthy tests/test_prerender_links.py
test_asset_tools.py software healthy tests/test_asset_tools.py
docker_build_trace.py software healthy tests/docker_build_trace.py
test_add_readme_section.py software healthy tests/test_add_readme_section.py
test_uccb.py software healthy tests/test_uccb/test_uccb.py
__init__.py software healthy tests/test_uccb/__init__.py
__init__.py software healthy tests/sevenllm/__init__.py
test_sevenllm.py software healthy tests/sevenllm/test_sevenllm.py

Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.

LabelLayerStatusPath
TestFindInstallTreeConsumers software healthy tests/test_dep_bump_impact.py:36
TestRenderHuman software healthy tests/test_dep_bump_impact.py:65
TestHasEvalCall software healthy tests/test_autolint_checks.py:17
TestExtractPackageName software healthy tests/test_autolint_checks.py:55
TestGetStdlibModules software healthy tests/test_autolint_checks.py:89
TestGetImportsFromFile software healthy tests/test_autolint_checks.py:120
TestFindTaskFunctions software healthy tests/test_autolint_checks.py:173
TestGetExportedNames software healthy tests/test_autolint_checks.py:247
TestBuildParametersSection software healthy tests/test_generate_readmes.py:67
TestReadmeExists software healthy tests/test_generate_readmes.py:216
TestGenerateBasicReadme software healthy tests/test_generate_readmes.py:233
TestFormatParameter software healthy tests/test_generate_readmes.py:286
TestParseDocstringParameters software healthy tests/test_generate_readmes.py:442
TestFormatTypeAnnotation software healthy tests/test_generate_readmes.py:650
CustomClass software healthy tests/test_generate_readmes.py:696
TestCleanDefaultValue software healthy tests/test_generate_readmes.py:739
TestBuildEvaluationReportSection software healthy tests/test_generate_readmes.py:858
HfDatasetCall software healthy tests/test_datasets_hf.py:72
TestXdistApplyCollectionGroups software healthy tests/test_xdist_grouping.py:27
TestCreateStableId software healthy tests/test_utils.py:8
TestFilterDuplicateIds software healthy tests/test_utils.py:134
TestGetDecoratorName software healthy tests/test_autolint_best_practices.py:15
TestGetCallName software healthy tests/test_autolint_best_practices.py:61
TestGetModelVisitor software healthy tests/test_autolint_best_practices.py:107
TestSampleIdVisitor software healthy tests/test_autolint_best_practices.py:201
TestTaskParameterVisitor software healthy tests/test_autolint_best_practices.py:275
TestTaskDefaultsVisitor software healthy tests/test_autolint_best_practices.py:340
TestPosixCodeChecker software healthy tests/test_check_posix_code.py:9
TestCheckFile software healthy tests/test_check_posix_code.py:139
TestMain software healthy tests/test_check_posix_code.py:175
TestNormalizeUrl software healthy tests/test_asset_tools.py:16
TestIsIgnorableUrl software healthy tests/test_asset_tools.py:35
TestUrlMatchesDeclared software healthy tests/test_asset_tools.py:73
TestGenerateAssetManifest software healthy tests/test_asset_tools.py:140
PopenWrapper software healthy tests/docker_build_trace.py:199
TestNormalizeKey software healthy tests/test_add_readme_section.py:16
TestHasSectionTags software healthy tests/test_add_readme_section.py:41
TestFindInsertionPoint software healthy tests/test_add_readme_section.py:83
TestAddSectionTags software healthy tests/test_add_readme_section.py:228
TestIntegrationScenarios software healthy tests/test_add_readme_section.py:364
MockOutput software healthy tests/sevenllm/test_sevenllm.py:21
DummyMemoryDataset software healthy tests/instrumentaleval/test_process_data.py:12
SampleModel software healthy tests/agentdojo/test_utils.py:10
TravelEnvironmentFactory software healthy tests/agentdojo/test_deepdiff_wrappers.py:21
WorkspaceEnvironmentFactory software healthy tests/agentdojo/test_deepdiff_wrappers.py:25
TestTokenFuncs software healthy tests/niah/test_niah.py:45
TestGenerateContextLengths software healthy tests/niah/test_niah.py:76
TestSampleGenerationFunctions software healthy tests/niah/test_niah.py:134
TestSampleNeedles software healthy tests/niah/test_niah.py:272
TestStretchOrSquashNeedles software healthy tests/niah/test_niah.py:349

Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.

LabelLayerStatusPath
packages software healthy packages
kernelbench software healthy packages/kernelbench
cve_bench software healthy packages/cve_bench
bold software healthy packages/bold
livebench software healthy packages/livebench
mle_bench software healthy packages/mle_bench
novelty_bench software healthy packages/novelty_bench
abstention_bench software healthy packages/abstention_bench
tests software healthy tests
test_uccb software healthy tests/test_uccb
sevenllm software healthy tests/sevenllm
docvqa software healthy tests/docvqa
instrumentaleval software healthy tests/instrumentaleval
agentdojo software healthy tests/agentdojo
niah software healthy tests/niah
aime2025 software healthy tests/aime2025
vimgolf_challenges software healthy tests/vimgolf_challenges
boolq software healthy tests/boolq
mmiu software healthy tests/mmiu
b3 software healthy tests/b3
mask software healthy tests/mask
mind2web software healthy tests/mind2web
cyberseceval_4 software healthy tests/cyberseceval_4
vstar_bench software healthy tests/vstar_bench
hle software healthy tests/hle
gdpval software healthy tests/gdpval
register_tooling software healthy tests/register_tooling
mmlu software healthy tests/mmlu
mmmu software healthy tests/mmmu
sosbench software healthy tests/sosbench
class_eval software healthy tests/class_eval
worldsense software healthy tests/worldsense
mind2web_sc software healthy tests/mind2web_sc
swe_lancer software healthy tests/swe_lancer
gpqa software healthy tests/gpqa
sciknoweval software healthy tests/sciknoweval
utils software healthy tests/utils
test_artifacts software healthy tests/utils/test_artifacts
humaneval software healthy tests/humaneval
fortress software healthy tests/fortress
gaia software healthy tests/gaia
arc software healthy tests/arc
frontierscience software healthy tests/frontierscience
gdm_self_reasoning software healthy tests/gdm_self_reasoning
kernelbench software healthy tests/kernelbench
air_bench software healthy tests/air_bench
simpleqa software healthy tests/simpleqa
assistant_bench software healthy tests/assistant_bench
bfcl software healthy tests/bfcl
fixtures software healthy tests/bfcl/fixtures

Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.

LabelLayerStatusPath
image::tests/utils/test_artifacts/Dockerfile hardware healthy tests/utils/test_artifacts/Dockerfile
image::src/inspect_evals/vimgolf_challenges/Dockerfile hardware healthy src/inspect_evals/vimgolf_challenges/Dockerfile
image::src/inspect_evals/gdpval/Dockerfile hardware healthy src/inspect_evals/gdpval/Dockerfile
image::src/inspect_evals/class_eval/Dockerfile hardware healthy src/inspect_evals/class_eval/Dockerfile
image::src/inspect_evals/kernelbench/Dockerfile hardware healthy src/inspect_evals/kernelbench/Dockerfile
image::src/inspect_evals/scbench/Dockerfile hardware healthy src/inspect_evals/scbench/Dockerfile
image::src/inspect_evals/cybench/agent_sandbox/Dockerfile hardware healthy src/inspect_evals/cybench/agent_sandbox/Dockerfile
image::src/inspect_evals/cybench/challenges/locktalk/images… hardware healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
image::src/inspect_evals/cybench/challenges/glacier_exchang… hardware healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
image::src/inspect_evals/cybench/challenges/frog_waf/images… hardware healthy src/inspect_evals/cybench/challenges/frog_waf/images/victim…
image::src/inspect_evals/cybench/challenges/skilift/images/… hardware healthy src/inspect_evals/cybench/challenges/skilift/images/victim/…
image::src/inspect_evals/cybench/challenges/were_pickle_phr… hardware healthy src/inspect_evals/cybench/challenges/were_pickle_phreaks_re…
image::src/inspect_evals/cybench/challenges/randsubware/ima… hardware healthy src/inspect_evals/cybench/challenges/randsubware/images/vic…
image::src/inspect_evals/cybench/challenges/noisier_crc/ima… hardware healthy src/inspect_evals/cybench/challenges/noisier_crc/images/vic…
image::src/inspect_evals/cybench/challenges/failproof/image… hardware healthy src/inspect_evals/cybench/challenges/failproof/images/victi…
image::src/inspect_evals/cybench/challenges/robust_cbc/imag… hardware healthy src/inspect_evals/cybench/challenges/robust_cbc/images/vict…
image::src/inspect_evals/cybench/challenges/flag_command/im… hardware healthy src/inspect_evals/cybench/challenges/flag_command/images/vi…
image::src/inspect_evals/cybench/challenges/back_to_the_pas… hardware healthy src/inspect_evals/cybench/challenges/back_to_the_past/image…
image::src/inspect_evals/cybench/challenges/diffecient/imag… hardware healthy src/inspect_evals/cybench/challenges/diffecient/images/vict…
image::src/inspect_evals/cybench/challenges/shuffled_aes/im… hardware healthy src/inspect_evals/cybench/challenges/shuffled_aes/images/vi…
image::src/inspect_evals/cybench/challenges/delulu/images/v… hardware healthy src/inspect_evals/cybench/challenges/delulu/images/victim/D…
image::src/inspect_evals/cybench/challenges/just_another_pi… hardware healthy src/inspect_evals/cybench/challenges/just_another_pickle_ja…
image::src/inspect_evals/cybench/challenges/network_tools/i… hardware healthy src/inspect_evals/cybench/challenges/network_tools/images/v…
image::src/inspect_evals/cybench/challenges/walking_to_the_… hardware healthy src/inspect_evals/cybench/challenges/walking_to_the_sea_sid…
image::src/inspect_evals/cybench/challenges/labyrinth_lingu… hardware healthy src/inspect_evals/cybench/challenges/labyrinth_linguist/ima…
image::src/inspect_evals/cybench/challenges/avatar/images/v… hardware healthy src/inspect_evals/cybench/challenges/avatar/images/victim/D…
image::src/inspect_evals/cybench/challenges/chunky/images/n… hardware healthy src/inspect_evals/cybench/challenges/chunky/images/nginx/Do…
image::src/inspect_evals/cybench/challenges/chunky/images/r… hardware healthy src/inspect_evals/cybench/challenges/chunky/images/rcache/D…
image::src/inspect_evals/cybench/challenges/chunky/images/b… hardware healthy src/inspect_evals/cybench/challenges/chunky/images/blog/Doc…
image::src/inspect_evals/cybench/challenges/noisy_crc/image… hardware healthy src/inspect_evals/cybench/challenges/noisy_crc/images/victi…
image::src/inspect_evals/cybench/challenges/ezmaze/images/v… hardware healthy src/inspect_evals/cybench/challenges/ezmaze/images/victim/D…
image::src/inspect_evals/cybench/challenges/unbreakable/ima… hardware healthy src/inspect_evals/cybench/challenges/unbreakable/images/vic…
image::src/inspect_evals/threecb/task_configs/evasion_debug… hardware healthy src/inspect_evals/threecb/task_configs/evasion_debugger/Doc…
image::src/inspect_evals/threecb/task_configs/web_navigatio… hardware healthy src/inspect_evals/threecb/task_configs/web_navigation/Docke…
image::src/inspect_evals/threecb/task_configs/resourcedevel… hardware healthy src/inspect_evals/threecb/task_configs/resourcedevelopment_…
image::src/inspect_evals/threecb/task_configs/python_repl_f… hardware healthy src/inspect_evals/threecb/task_configs/python_repl_factoria…
image::src/inspect_evals/threecb/task_configs/escalation_se… hardware healthy src/inspect_evals/threecb/task_configs/escalation_setuid/Do…
image::src/inspect_evals/threecb/task_configs/find_all_user… hardware healthy src/inspect_evals/threecb/task_configs/find_all_users/Docke…
image::src/inspect_evals/threecb/task_configs/binpwn_gdb_re… hardware healthy src/inspect_evals/threecb/task_configs/binpwn_gdb_repl/Dock…
image::src/inspect_evals/threecb/task_configs/impact_defaci… hardware healthy src/inspect_evals/threecb/task_configs/impact_defacing/Dock…
image::src/inspect_evals/threecb/task_configs/nmap/Dockerfi… hardware healthy src/inspect_evals/threecb/task_configs/nmap/Dockerfile
image::src/inspect_evals/threecb/task_configs/initialaccess… hardware healthy src/inspect_evals/threecb/task_configs/initialaccess_mail_l…
image::src/inspect_evals/threecb/task_configs/null_writing/… hardware healthy src/inspect_evals/threecb/task_configs/null_writing/Dockerf…
image::src/inspect_evals/bigcodebench/Dockerfile hardware healthy src/inspect_evals/bigcodebench/Dockerfile
image::src/inspect_evals/gdm_stealth/Dockerfile hardware healthy src/inspect_evals/gdm_stealth/Dockerfile
image::src/inspect_evals/ifevalcode/Dockerfile hardware healthy src/inspect_evals/ifevalcode/Dockerfile
image::src/inspect_evals/cyberseceval_2/vulnerability_explo… hardware healthy src/inspect_evals/cyberseceval_2/vulnerability_exploit/Dock…
image::src/inspect_evals/mlrc_bench/Dockerfile hardware healthy src/inspect_evals/mlrc_bench/Dockerfile
image::src/inspect_evals/paperbench/images/Dockerfile hardware healthy src/inspect_evals/paperbench/images/Dockerfile
image::src/inspect_evals/compute_eval/Dockerfile hardware healthy src/inspect_evals/compute_eval/Dockerfile

Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.

LabelLayerStatusPath
prepare cicd healthy .github/workflows/weekly-release.yml
review cicd healthy .github/workflows/claude-review.yaml
check-pr-template cicd healthy .github/workflows/pr-template-check.yml
validate cicd healthy .github/workflows/register-submission.yaml
programmatic-checks cicd healthy .github/workflows/register-submission.yaml
eval-validity cicd healthy .github/workflows/register-submission.yaml
claude-review cicd healthy .github/workflows/register-submission.yaml
await-manual-review cicd healthy .github/workflows/register-submission.yaml
lint-all-evals cicd healthy .github/workflows/lint-new-evals.yml
deploy cicd healthy .github/workflows/docs-preview.yml
render cicd healthy .github/workflows/docs.yml
publish cicd healthy .github/workflows/docs.yml
submit cicd healthy .github/workflows/register-issue-submission.yaml
lint cicd healthy .github/workflows/workflow-lint.yml
run-readme-check cicd healthy .github/workflows/checks.yml
posix-code-check cicd healthy .github/workflows/checks.yml
ruff cicd healthy .github/workflows/checks.yml
mypy cicd healthy .github/workflows/checks.yml
package cicd healthy .github/workflows/checks.yml
workflow-scripts-tests cicd healthy .github/workflows/checks.yml
prepare cicd healthy .github/workflows/docker-image-rebuild.yml
docker-build cicd healthy .github/workflows/docker-image-rebuild.yml
publish-manifests cicd healthy .github/workflows/docker-image-rebuild.yml
markdownlint cicd healthy .github/workflows/markdown-lint.yml
publish cicd healthy .github/workflows/publish.yaml
build-repo-context cicd healthy .github/workflows/build-repo-context.yaml
trigger-tests cicd healthy .github/workflows/build.yml
impact cicd healthy .github/workflows/dep-bump-impact.yml
solve-issue cicd healthy .github/workflows/claude-issue-solver.yaml
fix-pr cicd healthy .github/workflows/claude-issue-solver.yaml
eval-validity cicd healthy .github/workflows/register-submission-eval-validity.yaml
release cicd healthy .github/workflows/release-on-merge.yml
notify cicd healthy .github/workflows/release-on-merge.yml
scorecard cicd healthy .github/workflows/register-submission-programmatic-checks.y…
fix-from-ci cicd healthy .github/workflows/claude-fix-tests.yaml
fix-from-review cicd healthy .github/workflows/claude-fix-tests.yaml
parse-smoke-failures cicd healthy .github/workflows/claude-fix-tests.yaml
fix-smoke-failure cicd healthy .github/workflows/claude-fix-tests.yaml

LabelLayerStatusPath
ANY / api healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
ANY /get_ticket api healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
ANY /chat/<int:chat_id> api healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
ANY /flag api healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
ANY /assets/<path:path> api healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
ANY /api/data/fetch/<path:coin> api healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
ANY /api/wallet/transaction api healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
ANY /api/wallet/join_glacier_club api healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
ANY /api/wallet/balances api healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
ANY /api/fetch_coins api healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
ANY /api/options api healthy src/inspect_evals/cybench/challenges/flag_command/images/vi…
ANY /api/monitor api healthy src/inspect_evals/cybench/challenges/flag_command/images/vi…
ANY /signup api healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
ANY /login api healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
ANY /logout api healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
ANY /create_post api healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
ANY /post/<user_id>/<post_id> api healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
ANY /<user_id>/.well-known/jwks.json api healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
POST /submit-vulnerable api healthy src/inspect_evals/cybergym/task_template/controller/server.…
POST /submit-fixed api healthy src/inspect_evals/cybergym/task_template/controller/server.…
GET /health api healthy src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.…
GET /search/{keyword} api healthy src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.…
GET /tactic/{tactic} api healthy src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.…
ANY /models api healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py
ANY /sample api healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py
ANY /users api healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py
ANY /users/<login> api healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py
ANY /home api healthy src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py
ANY /accounts api healthy src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py
ANY /creds api healthy src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py
ANY /creds/<account_id> api healthy src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py
ANY /network api healthy src/inspect_evals/gdm_in_house_ctf/challenges/cmd_injection…
ANY /about api healthy src/inspect_evals/gdm_in_house_ctf/challenges/cmd_injection…

LabelLayerStatusPath
10.0.0.1 network healthy tests/cyberseceval_4/test_e2e.py
10.0.0.2 network healthy tests/cyberseceval_4/test_e2e.py
192.168.1.100 network healthy tests/paperbench/score/test_utils.py
127.0.0.1 network healthy tests/livecodebench_pro/test_judge.py
8.8.8.8 network healthy src/inspect_evals/cybench/challenges/it_has_begun/resources…
128.90.59.19 network healthy src/inspect_evals/cybench/challenges/it_has_begun/resources…
4.13.0.0 network healthy src/inspect_evals/cybench/challenges/randsubware/solution/r…
35.196.65.151 network healthy src/inspect_evals/cybench/challenges/eval_me/resources/extr…
192.168.1.1 network healthy src/inspect_evals/tau2/telecom/TelecomUserTools.py
12.4.5.8 network healthy src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met…
9.1.0.70 network healthy src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met…
11.2.1.3 network healthy src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met…
10.3.5.147 network healthy src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met…
11.6.1.9 network healthy src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met…
12.3.1.170 network healthy src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met…
24.04.9.2 network healthy src/inspect_evals/theagentcompany/tasks/admin_watch_video/c…
17.172.224.47 network healthy src/inspect_evals/agentharm/benchmark/harmful_grading_funct…
192.168.0.12 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
192.168.0.10 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
192.168.1.15 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
192.168.1.255 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
255.255.255.0 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
192.168.1.0 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
203.0.113.50 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
17.58.100.1 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/get_webp…
192.0.2.255 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/get_webp…
31.13.79.246 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/get_webp…
13.77.161.179 network healthy src/inspect_evals/agentharm/benchmark/benign_tools/get_webp…
2.6.2.1 network healthy src/inspect_evals/agentharm/benchmark/harmful_tools/termina…

LabelLayerStatusPath
auth::src/inspect_evals/cybench/challenges/locktalk/images/… security healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
auth::.github/workflows/register-submission.yaml security healthy .github/workflows/register-submission.yaml
auth::src/inspect_evals/agentharm/benchmark/benign_tools/qu… security healthy src/inspect_evals/agentharm/benchmark/benign_tools/query_bi…
auth::src/inspect_evals/cybench/challenges/locktalk/images/… security healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
auth::src/inspect_evals/agentharm/benchmark/benign_tools/qu… security healthy src/inspect_evals/agentharm/benchmark/benign_tools/query_on…
auth::src/inspect_evals/cybench/challenges/chunky/solution/… security healthy src/inspect_evals/cybench/challenges/chunky/solution/solve.…
auth::.github/workflows/publish.yaml security healthy .github/workflows/publish.yaml
auth::src/inspect_evals/agentharm/benchmark/harmful_tools/g… security healthy src/inspect_evals/agentharm/benchmark/harmful_tools/get_web…
auth::.github/workflows/register-issue-submission.yaml security healthy .github/workflows/register-issue-submission.yaml
auth::src/inspect_evals/theagentcompany/tasks/admin_mass_fo… security healthy src/inspect_evals/theagentcompany/tasks/admin_mass_forms_fi…
auth::src/inspect_evals/cybench/challenges/chunky/images/bl… security healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…
auth::src/inspect_evals/cybench/challenges/locktalk/images/… security healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
auth::.github/workflows/claude-issue-solver.yaml security healthy .github/workflows/claude-issue-solver.yaml
auth::.github/workflows/claude-review.yaml security healthy .github/workflows/claude-review.yaml
auth::src/register_tooling/deriver.py security healthy src/register_tooling/deriver.py
auth::src/inspect_evals/agentharm/benchmark/benign_tools/ge… security healthy src/inspect_evals/agentharm/benchmark/benign_tools/get_webp…
auth::.github/workflows/register-submission-eval-validity.y… security healthy .github/workflows/register-submission-eval-validity.yaml
auth::.github/actions/claude-setup/action.yaml security healthy .github/actions/claude-setup/action.yaml
auth::src/inspect_evals/agentharm/benchmark/harmful_tools/q… security healthy src/inspect_evals/agentharm/benchmark/harmful_tools/query_o…
auth::src/inspect_evals/cybench/challenges/locktalk/images/… security healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
auth::.github/workflows/claude-fix-tests.yaml security healthy .github/workflows/claude-fix-tests.yaml
auth::src/inspect_evals/agentdojo/tools/travel_booking_clie… security healthy src/inspect_evals/agentdojo/tools/travel_booking_client.py
auth::src/inspect_evals/agentdojo/task_suites/travel/inject… security healthy src/inspect_evals/agentdojo/task_suites/travel/injection_ta…
auth::.github/workflows/build-repo-context.yaml security healthy .github/workflows/build-repo-context.yaml
auth::src/inspect_evals/agentharm/benchmark/harmful_grading… security healthy src/inspect_evals/agentharm/benchmark/harmful_grading_funct…

LabelLayerStatusPath
port:8888 network healthy src/inspect_evals/gdpval/Dockerfile
port:256 network healthy src/inspect_evals/cybench/agent_sandbox/Dockerfile
port:99 network healthy src/inspect_evals/cybench/agent_sandbox/Dockerfile
port:1337 network healthy src/inspect_evals/cybench/challenges/locktalk/images/victim…
port:8080 network healthy src/inspect_evals/cybench/challenges/glacier_exchange/image…
port:9999 network healthy src/inspect_evals/cybench/challenges/randsubware/images/vic…
port:151 network healthy src/inspect_evals/cybench/challenges/eval_me/resources/extr…
port:30899 network healthy src/inspect_evals/cybench/challenges/eval_me/resources/extr…
port:2000 network healthy src/inspect_evals/threecb/task_configs/find_all_users/setup…
port:5000 network healthy src/inspect_evals/threecb/task_configs/nmap/app.py
port:1000 network healthy src/inspect_evals/mlrc_bench/Dockerfile
port:9090 network healthy src/inspect_evals/livecodebench_pro/compose.yaml
port:17 network healthy src/inspect_evals/theagentcompany/tasks/hr_check_attendance…
port:30 network healthy src/inspect_evals/theagentcompany/tasks/hr_check_attendance…
port:18 network healthy src/inspect_evals/theagentcompany/tasks/hr_check_attendance…
port:65 network healthy src/inspect_evals/cti_realm/docker/compose.yaml
port:1001 network healthy src/inspect_evals/cti_realm/docker/compose.yaml
port:8081 network healthy src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.…
port:80 network healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py
port:22 network healthy src/inspect_evals/gdm_in_house_ctf/challenges/ssh/Dockerfile
port:8082 network healthy src/inspect_evals/gdm_in_house_ctf/challenges/spray/app.py
port:3000 network healthy src/inspect_evals/gdm_in_house_ctf/challenges/grafana/Docke…

LabelLayerStatusPath
gha::weekly-release cicd healthy .github/workflows/weekly-release.yml
gha::claude-review cicd healthy .github/workflows/claude-review.yaml
gha::pr-template-check cicd healthy .github/workflows/pr-template-check.yml
gha::register-submission cicd healthy .github/workflows/register-submission.yaml
gha::lint-new-evals cicd healthy .github/workflows/lint-new-evals.yml
gha::docs-preview cicd healthy .github/workflows/docs-preview.yml
gha::docs cicd healthy .github/workflows/docs.yml
gha::register-issue-submission cicd healthy .github/workflows/register-issue-submission.yaml
gha::workflow-lint cicd healthy .github/workflows/workflow-lint.yml
gha::checks cicd healthy .github/workflows/checks.yml
gha::docker-image-rebuild cicd healthy .github/workflows/docker-image-rebuild.yml
gha::markdown-lint cicd healthy .github/workflows/markdown-lint.yml
gha::publish cicd healthy .github/workflows/publish.yaml
gha::build-repo-context cicd healthy .github/workflows/build-repo-context.yaml
gha::build cicd healthy .github/workflows/build.yml
gha::dep-bump-impact cicd healthy .github/workflows/dep-bump-impact.yml
gha::claude-issue-solver cicd healthy .github/workflows/claude-issue-solver.yaml
gha::register-submission-eval-validity cicd healthy .github/workflows/register-submission-eval-validity.yaml
gha::release-on-merge cicd healthy .github/workflows/release-on-merge.yml
gha::register-submission-programmatic-checks cicd healthy .github/workflows/register-submission-programmatic-checks.y…
gha::claude-fix-tests cicd healthy .github/workflows/claude-fix-tests.yaml

LabelLayerStatusPath
password_literal::src/inspect_evals/agentdojo/task_suites/b… security healthy src/inspect_evals/agentdojo/task_suites/banking/injection_t…
password_literal::src/inspect_evals/agentdojo/task_suites/b… security healthy src/inspect_evals/agentdojo/task_suites/banking/user_tasks.…
password_literal::src/inspect_evals/cyberseceval_4/insecure… security healthy src/inspect_evals/cyberseceval_4/insecure_code_detector/rul…
password_literal::src/inspect_evals/gdm_stealth/solver/stra… security healthy src/inspect_evals/gdm_stealth/solver/strategic_rule_breakin…
password_literal::src/inspect_evals/gdm_stealth/solver/stra… security healthy src/inspect_evals/gdm_stealth/solver/strategic_rule_breakin…
password_literal::src/inspect_evals/theagentcompany/tasks/h… security healthy src/inspect_evals/theagentcompany/tasks/hr_populate_salary_…
password_literal::src/inspect_evals/theagentcompany/tasks/h… security healthy src/inspect_evals/theagentcompany/tasks/hr_populate_salary_…
password_literal::src/inspect_evals/theagentcompany/tasks/h… security healthy src/inspect_evals/theagentcompany/tasks/hr_populate_salary_…
password_literal::src/inspect_evals/theagentcompany/tasks/h… security healthy src/inspect_evals/theagentcompany/tasks/hr_salary_analysis/…
generic_api_key::src/inspect_evals/cybergym/scorers.py security healthy src/inspect_evals/cybergym/scorers.py
generic_api_key::src/inspect_evals/cybergym/task_template/c… security healthy src/inspect_evals/cybergym/task_template/controller/server.…
github_token::src/inspect_evals/agentharm/benchmark/benign_… security healthy src/inspect_evals/agentharm/benchmark/benign_grading_functi…
github_token::src/inspect_evals/agentharm/benchmark/harmful… security healthy src/inspect_evals/agentharm/benchmark/harmful_grading_funct…
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… security healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… security healthy src/inspect_evals/agentharm/benchmark/benign_tools/terminal…
github_token::src/inspect_evals/agentharm/benchmark/benign_… security healthy src/inspect_evals/agentharm/benchmark/benign_tools/content/…
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… security healthy src/inspect_evals/agentharm/benchmark/harmful_tools/termina…
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… security healthy src/inspect_evals/agentharm/benchmark/harmful_tools/termina…
github_token::src/inspect_evals/agentharm/benchmark/harmful… security healthy src/inspect_evals/agentharm/benchmark/harmful_tools/content…

LabelLayerStatusPath
SLACK_WEBHOOK_URL cicd healthy
INSPECT_COMMUNITY_SLACK_WEBHOOK_URL cicd healthy
ARCADIA_GHCR_TOKEN cicd healthy
SEMANTIC_SCHOLAR_API_KEY cicd healthy
INSPECT_EVALS_DISPATCH_TOKEN cicd healthy
PR_NUMBER cicd healthy
ARCADIA_GHCR_USERNAME cicd healthy
ANTHROPIC_ROLE_ARN cicd healthy
GITHUB_TOKEN cicd healthy
REGISTER_SLACK_WEBHOOK_URL cicd healthy
APP_PRIVATE_KEY cicd healthy
APP_ID cicd healthy
ANTHROPIC_API_KEY cicd healthy

LabelLayerStatusPath
products data healthy src/inspect_evals/theagentcompany/tasks/ds_sql_exercise/ini…
models data healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.sql
users data healthy src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.sql
members data healthy src/inspect_evals/gdm_in_house_ctf/challenges/pw_reuse/db.s…
posts data healthy src/inspect_evals/gdm_in_house_ctf/challenges/pw_reuse/db.s…
transactions data healthy src/inspect_evals/gdm_self_reasoning/common_tools.py
blog_posts data healthy src/inspect_evals/cybench/challenges/chunky/images/blog/src…

LabelLayerStatusPath
postgres data healthy tests/utils/test_docker_utils.py
postgresql data healthy tests/bfcl/test_solver.py
sqlite data healthy tools/run_evals.py
mysql data healthy src/inspect_evals/cyberseceval_4/insecure_code_detector/rul…
cassandra data healthy src/inspect_evals/cyberseceval_4/insecure_code_detector/rul…
mongodb data healthy src/inspect_evals/cyberseceval_4/insecure_code_detector/rul…

LabelLayerStatusPath
vps::azure hardware healthy README.md
vps::aws hardware healthy README.md
vps::gcp hardware healthy tests/gdm_self_proliferation/test_dataset_utils.py
vps::ec2 hardware healthy src/inspect_evals/gdpval/README.md

LabelLayerStatusPath
haproxy network healthy ASSETS.yaml
nginx network healthy .pre-commit-config.yaml

LabelLayerStatusPath
UKGovernmentBEIS__inspect_evals software healthy /data/fable5_failed_archive/UKGovernmentBEIS__inspect_evals

LabelLayerStatusPath
celery data healthy tests/sosbench/test_utils.py

LabelLayerStatusPath
gpu (detected) hardware healthy ASSETS.yaml
For AI agents: Voting guide (TP/FP) MCP manifest Stdio wrapper SARIF Integrate Findings queue Vote TP/FP on findings to calibrate the engine.
For AI agents + API integrations
Email me when this repo regresses
Free. We re-scan periodically; new criticals → your inbox. No signup required for the scan itself.
API access

This page is publicly accessible at: https://repobility.com/scan/13d59e23-a6db-41ca-bf82-51b9ab36f8d1/

To check status programmatically (no auth required):

curl -s https://repobility.com/api/v1/public/scan/13d59e23-a6db-41ca-bf82-51b9ab36f8d1/

Important — please don't re-submit the same URL repeatedly. The submission endpoint is idempotent: re-submitting the same git URL returns this same scan_token, not a new one. To re-scan this repo, sign up free and use the dashboard.