https://github.com/UKGovernmentBEIS/inspect_evals
· scanned 2026-06-16 22:23 UTC (1 month, 2 weeks ago)
1275 raw signals (0 security + 1275 graph)
Last scanned 1 month, 2 weeks ago · v3 · 543 actionable findings from 1 signal source. 100 repeated signals grouped for readability. Security checks, system graph analysis, and verified AI-agent feedback are merged into one review queue.
All 16011 nodes from the latest scan, grouped by kind. Each node is a unit the engine identified (file, function, endpoint, table…). Most users won't need this view — it's primarily for debugging the engine's graph extraction or for AI agents that want to enumerate the project structure.
| Label | Layer | Status | Path |
|---|---|---|---|
_fake_run |
software | healthy | tests/test_dep_bump_impact.py:32 |
test_invokes_uv_tree_with_all_groups |
software | healthy | tests/test_dep_bump_impact.py:37 |
test_detects_dev_group_for_types_pyyaml |
software | healthy | tests/test_dep_bump_impact.py:48 |
test_types_pyyaml_reported_as_dev_tooling_not_no_impact |
software | healthy | tests/test_dep_bump_impact.py:66 |
test_epoch_metadata_reduction_issue |
software | healthy | tests/test_epoch_compatibility.py:47 |
value_based_mean |
software | healthy | tests/test_epoch_compatibility.py:56 |
calc |
software | healthy | tests/test_epoch_compatibility.py:71 |
metadata_based_mean |
software | healthy | tests/test_epoch_compatibility.py:68 |
varying_scorer |
software | healthy | tests/test_epoch_compatibility.py:85 |
score |
software | healthy | tests/test_epoch_compatibility.py:86 |
_hf_telemetry_enabled |
software | healthy | tests/conftest.py:46 |
pytest_addoption |
software | healthy | tests/conftest.py:54 |
pytest_configure |
software | healthy | tests/conftest.py:85 |
pytest_collection_modifyitems |
software | healthy | tests/conftest.py:116 |
pytest_runtest_setup |
software | healthy | tests/conftest.py:182 |
pytest_runtest_teardown |
software | healthy | tests/conftest.py:188 |
set_model_roles |
software | healthy | tests/conftest.py:195 |
pytest_runtest_makereport |
software | healthy | tests/conftest.py:208 |
pytest_sessionfinish |
software | healthy | tests/conftest.py:229 |
test_dry_run_does_not_delete |
software | healthy | tests/test_clean.py:8 |
test_delete_file_and_dir |
software | healthy | tests/test_clean.py:21 |
test_delete_path_if_exists_raises_for_missing_path |
software | healthy | tests/test_clean.py:34 |
test_rm_from_dir |
software | healthy | tests/test_clean.py:43 |
test_rm_from_dir_no_dir |
software | healthy | tests/test_clean.py:59 |
test_rm_from_dir_empty_dir |
software | healthy | tests/test_clean.py:65 |
test_registry_imports |
software | healthy | tests/test_registry_imports.py:7 |
test_registry_exposes_every_eval_yaml_task |
software | healthy | tests/test_registry_imports.py:12 |
test_direct_eval_call |
software | healthy | tests/test_autolint_checks.py:20 |
test_module_eval_call |
software | healthy | tests/test_autolint_checks.py:26 |
test_no_eval_call |
software | healthy | tests/test_autolint_checks.py:32 |
test_eval_in_function |
software | healthy | tests/test_autolint_checks.py:38 |
test_different_eval_not_detected |
software | healthy | tests/test_autolint_checks.py:48 |
test_simple_package |
software | healthy | tests/test_autolint_checks.py:58 |
test_package_with_version |
software | healthy | tests/test_autolint_checks.py:62 |
test_package_with_extras |
software | healthy | tests/test_autolint_checks.py:68 |
test_package_with_environment_marker |
software | healthy | tests/test_autolint_checks.py:72 |
test_package_with_trailing_space |
software | healthy | tests/test_autolint_checks.py:76 |
test_empty_string |
software | healthy | tests/test_autolint_checks.py:80 |
test_complex_specifier |
software | healthy | tests/test_autolint_checks.py:84 |
test_returns_frozenset |
software | healthy | tests/test_autolint_checks.py:92 |
test_contains_common_modules |
software | healthy | tests/test_autolint_checks.py:97 |
test_contains_typing_extensions |
software | healthy | tests/test_autolint_checks.py:107 |
test_does_not_contain_third_party |
software | healthy | tests/test_autolint_checks.py:112 |
test_simple_import |
software | healthy | tests/test_autolint_checks.py:123 |
test_from_import |
software | healthy | tests/test_autolint_checks.py:131 |
test_nested_import |
software | healthy | tests/test_autolint_checks.py:139 |
test_multiple_imports |
software | healthy | tests/test_autolint_checks.py:147 |
test_aliased_import |
software | healthy | tests/test_autolint_checks.py:163 |
test_finds_task_function |
software | healthy | tests/test_autolint_checks.py:176 |
test_finds_multiple_task_functions |
software | healthy | tests/test_autolint_checks.py:189 |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
TASK_VERSIONING.md |
software | healthy | TASK_VERSIONING.md |
CLAUDE.md |
software | healthy | CLAUDE.md |
AGENTS.md |
software | healthy | AGENTS.md |
ASSETS.yaml |
software | healthy | ASSETS.yaml |
MAINTAINERS.md |
software | healthy | MAINTAINERS.md |
CONTRIBUTING.md |
software | healthy | CONTRIBUTING.md |
CHANGELOG.md |
software | healthy | CHANGELOG.md |
AUTOMATED_CHECKS.md |
software | healthy | AUTOMATED_CHECKS.md |
.markdownlint.yaml |
software | healthy | .markdownlint.yaml |
README.md |
software | healthy | README.md |
pyproject.toml |
software | healthy | pyproject.toml |
Makefile |
software | healthy | Makefile |
LICENSE |
software | healthy | LICENSE |
NOTICE |
software | healthy | NOTICE |
.pre-commit-config.yaml |
software | healthy | .pre-commit-config.yaml |
PACKAGE_VERSIONING.md |
software | healthy | PACKAGE_VERSIONING.md |
EVALUATION_CHECKLIST.md |
software | healthy | EVALUATION_CHECKLIST.md |
BEST_PRACTICES.md |
software | healthy | BEST_PRACTICES.md |
EVAL_REGISTER.md |
software | healthy | EVAL_REGISTER.md |
pyproject.toml |
software | healthy | packages/kernelbench/pyproject.toml |
pyproject.toml |
software | healthy | packages/cve_bench/pyproject.toml |
pyproject.toml |
software | healthy | packages/bold/pyproject.toml |
pyproject.toml |
software | healthy | packages/livebench/pyproject.toml |
pyproject.toml |
software | healthy | packages/mle_bench/pyproject.toml |
pyproject.toml |
software | healthy | packages/novelty_bench/pyproject.toml |
pyproject.toml |
software | healthy | packages/abstention_bench/pyproject.toml |
test_dep_bump_impact.py |
software | healthy | tests/test_dep_bump_impact.py |
test_epoch_compatibility.py |
software | healthy | tests/test_epoch_compatibility.py |
conftest.py |
software | healthy | tests/conftest.py |
test_clean.py |
software | healthy | tests/test_clean.py |
test_registry_imports.py |
software | healthy | tests/test_registry_imports.py |
__init__.py |
software | healthy | tests/__init__.py |
test_autolint_checks.py |
software | healthy | tests/test_autolint_checks.py |
README.md |
software | healthy | tests/README.md |
test_generate_readmes.py |
software | healthy | tests/test_generate_readmes.py |
test_datasets_hf.py |
software | healthy | tests/test_datasets_hf.py |
test_xdist_grouping.py |
software | healthy | tests/test_xdist_grouping.py |
test_eval_imports.py |
software | healthy | tests/test_eval_imports.py |
test_utils.py |
software | healthy | tests/test_utils.py |
test_autolint_best_practices.py |
software | healthy | tests/test_autolint_best_practices.py |
test_check_posix_code.py |
software | healthy | tests/test_check_posix_code.py |
test_metadata.py |
software | healthy | tests/test_metadata.py |
test_prerender_links.py |
software | healthy | tests/test_prerender_links.py |
test_asset_tools.py |
software | healthy | tests/test_asset_tools.py |
docker_build_trace.py |
software | healthy | tests/docker_build_trace.py |
test_add_readme_section.py |
software | healthy | tests/test_add_readme_section.py |
test_uccb.py |
software | healthy | tests/test_uccb/test_uccb.py |
__init__.py |
software | healthy | tests/test_uccb/__init__.py |
__init__.py |
software | healthy | tests/sevenllm/__init__.py |
test_sevenllm.py |
software | healthy | tests/sevenllm/test_sevenllm.py |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
TestFindInstallTreeConsumers |
software | healthy | tests/test_dep_bump_impact.py:36 |
TestRenderHuman |
software | healthy | tests/test_dep_bump_impact.py:65 |
TestHasEvalCall |
software | healthy | tests/test_autolint_checks.py:17 |
TestExtractPackageName |
software | healthy | tests/test_autolint_checks.py:55 |
TestGetStdlibModules |
software | healthy | tests/test_autolint_checks.py:89 |
TestGetImportsFromFile |
software | healthy | tests/test_autolint_checks.py:120 |
TestFindTaskFunctions |
software | healthy | tests/test_autolint_checks.py:173 |
TestGetExportedNames |
software | healthy | tests/test_autolint_checks.py:247 |
TestBuildParametersSection |
software | healthy | tests/test_generate_readmes.py:67 |
TestReadmeExists |
software | healthy | tests/test_generate_readmes.py:216 |
TestGenerateBasicReadme |
software | healthy | tests/test_generate_readmes.py:233 |
TestFormatParameter |
software | healthy | tests/test_generate_readmes.py:286 |
TestParseDocstringParameters |
software | healthy | tests/test_generate_readmes.py:442 |
TestFormatTypeAnnotation |
software | healthy | tests/test_generate_readmes.py:650 |
CustomClass |
software | healthy | tests/test_generate_readmes.py:696 |
TestCleanDefaultValue |
software | healthy | tests/test_generate_readmes.py:739 |
TestBuildEvaluationReportSection |
software | healthy | tests/test_generate_readmes.py:858 |
HfDatasetCall |
software | healthy | tests/test_datasets_hf.py:72 |
TestXdistApplyCollectionGroups |
software | healthy | tests/test_xdist_grouping.py:27 |
TestCreateStableId |
software | healthy | tests/test_utils.py:8 |
TestFilterDuplicateIds |
software | healthy | tests/test_utils.py:134 |
TestGetDecoratorName |
software | healthy | tests/test_autolint_best_practices.py:15 |
TestGetCallName |
software | healthy | tests/test_autolint_best_practices.py:61 |
TestGetModelVisitor |
software | healthy | tests/test_autolint_best_practices.py:107 |
TestSampleIdVisitor |
software | healthy | tests/test_autolint_best_practices.py:201 |
TestTaskParameterVisitor |
software | healthy | tests/test_autolint_best_practices.py:275 |
TestTaskDefaultsVisitor |
software | healthy | tests/test_autolint_best_practices.py:340 |
TestPosixCodeChecker |
software | healthy | tests/test_check_posix_code.py:9 |
TestCheckFile |
software | healthy | tests/test_check_posix_code.py:139 |
TestMain |
software | healthy | tests/test_check_posix_code.py:175 |
TestNormalizeUrl |
software | healthy | tests/test_asset_tools.py:16 |
TestIsIgnorableUrl |
software | healthy | tests/test_asset_tools.py:35 |
TestUrlMatchesDeclared |
software | healthy | tests/test_asset_tools.py:73 |
TestGenerateAssetManifest |
software | healthy | tests/test_asset_tools.py:140 |
PopenWrapper |
software | healthy | tests/docker_build_trace.py:199 |
TestNormalizeKey |
software | healthy | tests/test_add_readme_section.py:16 |
TestHasSectionTags |
software | healthy | tests/test_add_readme_section.py:41 |
TestFindInsertionPoint |
software | healthy | tests/test_add_readme_section.py:83 |
TestAddSectionTags |
software | healthy | tests/test_add_readme_section.py:228 |
TestIntegrationScenarios |
software | healthy | tests/test_add_readme_section.py:364 |
MockOutput |
software | healthy | tests/sevenllm/test_sevenllm.py:21 |
DummyMemoryDataset |
software | healthy | tests/instrumentaleval/test_process_data.py:12 |
SampleModel |
software | healthy | tests/agentdojo/test_utils.py:10 |
TravelEnvironmentFactory |
software | healthy | tests/agentdojo/test_deepdiff_wrappers.py:21 |
WorkspaceEnvironmentFactory |
software | healthy | tests/agentdojo/test_deepdiff_wrappers.py:25 |
TestTokenFuncs |
software | healthy | tests/niah/test_niah.py:45 |
TestGenerateContextLengths |
software | healthy | tests/niah/test_niah.py:76 |
TestSampleGenerationFunctions |
software | healthy | tests/niah/test_niah.py:134 |
TestSampleNeedles |
software | healthy | tests/niah/test_niah.py:272 |
TestStretchOrSquashNeedles |
software | healthy | tests/niah/test_niah.py:349 |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
packages |
software | healthy | packages |
kernelbench |
software | healthy | packages/kernelbench |
cve_bench |
software | healthy | packages/cve_bench |
bold |
software | healthy | packages/bold |
livebench |
software | healthy | packages/livebench |
mle_bench |
software | healthy | packages/mle_bench |
novelty_bench |
software | healthy | packages/novelty_bench |
abstention_bench |
software | healthy | packages/abstention_bench |
tests |
software | healthy | tests |
test_uccb |
software | healthy | tests/test_uccb |
sevenllm |
software | healthy | tests/sevenllm |
docvqa |
software | healthy | tests/docvqa |
instrumentaleval |
software | healthy | tests/instrumentaleval |
agentdojo |
software | healthy | tests/agentdojo |
niah |
software | healthy | tests/niah |
aime2025 |
software | healthy | tests/aime2025 |
vimgolf_challenges |
software | healthy | tests/vimgolf_challenges |
boolq |
software | healthy | tests/boolq |
mmiu |
software | healthy | tests/mmiu |
b3 |
software | healthy | tests/b3 |
mask |
software | healthy | tests/mask |
mind2web |
software | healthy | tests/mind2web |
cyberseceval_4 |
software | healthy | tests/cyberseceval_4 |
vstar_bench |
software | healthy | tests/vstar_bench |
hle |
software | healthy | tests/hle |
gdpval |
software | healthy | tests/gdpval |
register_tooling |
software | healthy | tests/register_tooling |
mmlu |
software | healthy | tests/mmlu |
mmmu |
software | healthy | tests/mmmu |
sosbench |
software | healthy | tests/sosbench |
class_eval |
software | healthy | tests/class_eval |
worldsense |
software | healthy | tests/worldsense |
mind2web_sc |
software | healthy | tests/mind2web_sc |
swe_lancer |
software | healthy | tests/swe_lancer |
gpqa |
software | healthy | tests/gpqa |
sciknoweval |
software | healthy | tests/sciknoweval |
utils |
software | healthy | tests/utils |
test_artifacts |
software | healthy | tests/utils/test_artifacts |
humaneval |
software | healthy | tests/humaneval |
fortress |
software | healthy | tests/fortress |
gaia |
software | healthy | tests/gaia |
arc |
software | healthy | tests/arc |
frontierscience |
software | healthy | tests/frontierscience |
gdm_self_reasoning |
software | healthy | tests/gdm_self_reasoning |
kernelbench |
software | healthy | tests/kernelbench |
air_bench |
software | healthy | tests/air_bench |
simpleqa |
software | healthy | tests/simpleqa |
assistant_bench |
software | healthy | tests/assistant_bench |
bfcl |
software | healthy | tests/bfcl |
fixtures |
software | healthy | tests/bfcl/fixtures |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
image::tests/utils/test_artifacts/Dockerfile |
hardware | healthy | tests/utils/test_artifacts/Dockerfile |
image::src/inspect_evals/vimgolf_challenges/Dockerfile |
hardware | healthy | src/inspect_evals/vimgolf_challenges/Dockerfile |
image::src/inspect_evals/gdpval/Dockerfile |
hardware | healthy | src/inspect_evals/gdpval/Dockerfile |
image::src/inspect_evals/class_eval/Dockerfile |
hardware | healthy | src/inspect_evals/class_eval/Dockerfile |
image::src/inspect_evals/kernelbench/Dockerfile |
hardware | healthy | src/inspect_evals/kernelbench/Dockerfile |
image::src/inspect_evals/scbench/Dockerfile |
hardware | healthy | src/inspect_evals/scbench/Dockerfile |
image::src/inspect_evals/cybench/agent_sandbox/Dockerfile |
hardware | healthy | src/inspect_evals/cybench/agent_sandbox/Dockerfile |
image::src/inspect_evals/cybench/challenges/locktalk/images… |
hardware | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
image::src/inspect_evals/cybench/challenges/glacier_exchang… |
hardware | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
image::src/inspect_evals/cybench/challenges/frog_waf/images… |
hardware | healthy | src/inspect_evals/cybench/challenges/frog_waf/images/victim… |
image::src/inspect_evals/cybench/challenges/skilift/images/… |
hardware | healthy | src/inspect_evals/cybench/challenges/skilift/images/victim/… |
image::src/inspect_evals/cybench/challenges/were_pickle_phr… |
hardware | healthy | src/inspect_evals/cybench/challenges/were_pickle_phreaks_re… |
image::src/inspect_evals/cybench/challenges/randsubware/ima… |
hardware | healthy | src/inspect_evals/cybench/challenges/randsubware/images/vic… |
image::src/inspect_evals/cybench/challenges/noisier_crc/ima… |
hardware | healthy | src/inspect_evals/cybench/challenges/noisier_crc/images/vic… |
image::src/inspect_evals/cybench/challenges/failproof/image… |
hardware | healthy | src/inspect_evals/cybench/challenges/failproof/images/victi… |
image::src/inspect_evals/cybench/challenges/robust_cbc/imag… |
hardware | healthy | src/inspect_evals/cybench/challenges/robust_cbc/images/vict… |
image::src/inspect_evals/cybench/challenges/flag_command/im… |
hardware | healthy | src/inspect_evals/cybench/challenges/flag_command/images/vi… |
image::src/inspect_evals/cybench/challenges/back_to_the_pas… |
hardware | healthy | src/inspect_evals/cybench/challenges/back_to_the_past/image… |
image::src/inspect_evals/cybench/challenges/diffecient/imag… |
hardware | healthy | src/inspect_evals/cybench/challenges/diffecient/images/vict… |
image::src/inspect_evals/cybench/challenges/shuffled_aes/im… |
hardware | healthy | src/inspect_evals/cybench/challenges/shuffled_aes/images/vi… |
image::src/inspect_evals/cybench/challenges/delulu/images/v… |
hardware | healthy | src/inspect_evals/cybench/challenges/delulu/images/victim/D… |
image::src/inspect_evals/cybench/challenges/just_another_pi… |
hardware | healthy | src/inspect_evals/cybench/challenges/just_another_pickle_ja… |
image::src/inspect_evals/cybench/challenges/network_tools/i… |
hardware | healthy | src/inspect_evals/cybench/challenges/network_tools/images/v… |
image::src/inspect_evals/cybench/challenges/walking_to_the_… |
hardware | healthy | src/inspect_evals/cybench/challenges/walking_to_the_sea_sid… |
image::src/inspect_evals/cybench/challenges/labyrinth_lingu… |
hardware | healthy | src/inspect_evals/cybench/challenges/labyrinth_linguist/ima… |
image::src/inspect_evals/cybench/challenges/avatar/images/v… |
hardware | healthy | src/inspect_evals/cybench/challenges/avatar/images/victim/D… |
image::src/inspect_evals/cybench/challenges/chunky/images/n… |
hardware | healthy | src/inspect_evals/cybench/challenges/chunky/images/nginx/Do… |
image::src/inspect_evals/cybench/challenges/chunky/images/r… |
hardware | healthy | src/inspect_evals/cybench/challenges/chunky/images/rcache/D… |
image::src/inspect_evals/cybench/challenges/chunky/images/b… |
hardware | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/Doc… |
image::src/inspect_evals/cybench/challenges/noisy_crc/image… |
hardware | healthy | src/inspect_evals/cybench/challenges/noisy_crc/images/victi… |
image::src/inspect_evals/cybench/challenges/ezmaze/images/v… |
hardware | healthy | src/inspect_evals/cybench/challenges/ezmaze/images/victim/D… |
image::src/inspect_evals/cybench/challenges/unbreakable/ima… |
hardware | healthy | src/inspect_evals/cybench/challenges/unbreakable/images/vic… |
image::src/inspect_evals/threecb/task_configs/evasion_debug… |
hardware | healthy | src/inspect_evals/threecb/task_configs/evasion_debugger/Doc… |
image::src/inspect_evals/threecb/task_configs/web_navigatio… |
hardware | healthy | src/inspect_evals/threecb/task_configs/web_navigation/Docke… |
image::src/inspect_evals/threecb/task_configs/resourcedevel… |
hardware | healthy | src/inspect_evals/threecb/task_configs/resourcedevelopment_… |
image::src/inspect_evals/threecb/task_configs/python_repl_f… |
hardware | healthy | src/inspect_evals/threecb/task_configs/python_repl_factoria… |
image::src/inspect_evals/threecb/task_configs/escalation_se… |
hardware | healthy | src/inspect_evals/threecb/task_configs/escalation_setuid/Do… |
image::src/inspect_evals/threecb/task_configs/find_all_user… |
hardware | healthy | src/inspect_evals/threecb/task_configs/find_all_users/Docke… |
image::src/inspect_evals/threecb/task_configs/binpwn_gdb_re… |
hardware | healthy | src/inspect_evals/threecb/task_configs/binpwn_gdb_repl/Dock… |
image::src/inspect_evals/threecb/task_configs/impact_defaci… |
hardware | healthy | src/inspect_evals/threecb/task_configs/impact_defacing/Dock… |
image::src/inspect_evals/threecb/task_configs/nmap/Dockerfi… |
hardware | healthy | src/inspect_evals/threecb/task_configs/nmap/Dockerfile |
image::src/inspect_evals/threecb/task_configs/initialaccess… |
hardware | healthy | src/inspect_evals/threecb/task_configs/initialaccess_mail_l… |
image::src/inspect_evals/threecb/task_configs/null_writing/… |
hardware | healthy | src/inspect_evals/threecb/task_configs/null_writing/Dockerf… |
image::src/inspect_evals/bigcodebench/Dockerfile |
hardware | healthy | src/inspect_evals/bigcodebench/Dockerfile |
image::src/inspect_evals/gdm_stealth/Dockerfile |
hardware | healthy | src/inspect_evals/gdm_stealth/Dockerfile |
image::src/inspect_evals/ifevalcode/Dockerfile |
hardware | healthy | src/inspect_evals/ifevalcode/Dockerfile |
image::src/inspect_evals/cyberseceval_2/vulnerability_explo… |
hardware | healthy | src/inspect_evals/cyberseceval_2/vulnerability_exploit/Dock… |
image::src/inspect_evals/mlrc_bench/Dockerfile |
hardware | healthy | src/inspect_evals/mlrc_bench/Dockerfile |
image::src/inspect_evals/paperbench/images/Dockerfile |
hardware | healthy | src/inspect_evals/paperbench/images/Dockerfile |
image::src/inspect_evals/compute_eval/Dockerfile |
hardware | healthy | src/inspect_evals/compute_eval/Dockerfile |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
prepare |
cicd | healthy | .github/workflows/weekly-release.yml |
review |
cicd | healthy | .github/workflows/claude-review.yaml |
check-pr-template |
cicd | healthy | .github/workflows/pr-template-check.yml |
validate |
cicd | healthy | .github/workflows/register-submission.yaml |
programmatic-checks |
cicd | healthy | .github/workflows/register-submission.yaml |
eval-validity |
cicd | healthy | .github/workflows/register-submission.yaml |
claude-review |
cicd | healthy | .github/workflows/register-submission.yaml |
await-manual-review |
cicd | healthy | .github/workflows/register-submission.yaml |
lint-all-evals |
cicd | healthy | .github/workflows/lint-new-evals.yml |
deploy |
cicd | healthy | .github/workflows/docs-preview.yml |
render |
cicd | healthy | .github/workflows/docs.yml |
publish |
cicd | healthy | .github/workflows/docs.yml |
submit |
cicd | healthy | .github/workflows/register-issue-submission.yaml |
lint |
cicd | healthy | .github/workflows/workflow-lint.yml |
run-readme-check |
cicd | healthy | .github/workflows/checks.yml |
posix-code-check |
cicd | healthy | .github/workflows/checks.yml |
ruff |
cicd | healthy | .github/workflows/checks.yml |
mypy |
cicd | healthy | .github/workflows/checks.yml |
package |
cicd | healthy | .github/workflows/checks.yml |
workflow-scripts-tests |
cicd | healthy | .github/workflows/checks.yml |
prepare |
cicd | healthy | .github/workflows/docker-image-rebuild.yml |
docker-build |
cicd | healthy | .github/workflows/docker-image-rebuild.yml |
publish-manifests |
cicd | healthy | .github/workflows/docker-image-rebuild.yml |
markdownlint |
cicd | healthy | .github/workflows/markdown-lint.yml |
publish |
cicd | healthy | .github/workflows/publish.yaml |
build-repo-context |
cicd | healthy | .github/workflows/build-repo-context.yaml |
trigger-tests |
cicd | healthy | .github/workflows/build.yml |
impact |
cicd | healthy | .github/workflows/dep-bump-impact.yml |
solve-issue |
cicd | healthy | .github/workflows/claude-issue-solver.yaml |
fix-pr |
cicd | healthy | .github/workflows/claude-issue-solver.yaml |
eval-validity |
cicd | healthy | .github/workflows/register-submission-eval-validity.yaml |
release |
cicd | healthy | .github/workflows/release-on-merge.yml |
notify |
cicd | healthy | .github/workflows/release-on-merge.yml |
scorecard |
cicd | healthy | .github/workflows/register-submission-programmatic-checks.y… |
fix-from-ci |
cicd | healthy | .github/workflows/claude-fix-tests.yaml |
fix-from-review |
cicd | healthy | .github/workflows/claude-fix-tests.yaml |
parse-smoke-failures |
cicd | healthy | .github/workflows/claude-fix-tests.yaml |
fix-smoke-failure |
cicd | healthy | .github/workflows/claude-fix-tests.yaml |
| Label | Layer | Status | Path |
|---|---|---|---|
ANY / |
api | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
ANY /get_ticket |
api | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
ANY /chat/<int:chat_id> |
api | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
ANY /flag |
api | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
ANY /assets/<path:path> |
api | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
ANY /api/data/fetch/<path:coin> |
api | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
ANY /api/wallet/transaction |
api | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
ANY /api/wallet/join_glacier_club |
api | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
ANY /api/wallet/balances |
api | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
ANY /api/fetch_coins |
api | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
ANY /api/options |
api | healthy | src/inspect_evals/cybench/challenges/flag_command/images/vi… |
ANY /api/monitor |
api | healthy | src/inspect_evals/cybench/challenges/flag_command/images/vi… |
ANY /signup |
api | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
ANY /login |
api | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
ANY /logout |
api | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
ANY /create_post |
api | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
ANY /post/<user_id>/<post_id> |
api | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
ANY /<user_id>/.well-known/jwks.json |
api | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
POST /submit-vulnerable |
api | healthy | src/inspect_evals/cybergym/task_template/controller/server.… |
POST /submit-fixed |
api | healthy | src/inspect_evals/cybergym/task_template/controller/server.… |
GET /health |
api | healthy | src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.… |
GET /search/{keyword} |
api | healthy | src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.… |
GET /tactic/{tactic} |
api | healthy | src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.… |
ANY /models |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py |
ANY /sample |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py |
ANY /users |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py |
ANY /users/<login> |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py |
ANY /home |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py |
ANY /accounts |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py |
ANY /creds |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py |
ANY /creds/<account_id> |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/idor/app.py |
ANY /network |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/cmd_injection… |
ANY /about |
api | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/cmd_injection… |
| Label | Layer | Status | Path |
|---|---|---|---|
10.0.0.1 |
network | healthy | tests/cyberseceval_4/test_e2e.py |
10.0.0.2 |
network | healthy | tests/cyberseceval_4/test_e2e.py |
192.168.1.100 |
network | healthy | tests/paperbench/score/test_utils.py |
127.0.0.1 |
network | healthy | tests/livecodebench_pro/test_judge.py |
8.8.8.8 |
network | healthy | src/inspect_evals/cybench/challenges/it_has_begun/resources… |
128.90.59.19 |
network | healthy | src/inspect_evals/cybench/challenges/it_has_begun/resources… |
4.13.0.0 |
network | healthy | src/inspect_evals/cybench/challenges/randsubware/solution/r… |
35.196.65.151 |
network | healthy | src/inspect_evals/cybench/challenges/eval_me/resources/extr… |
192.168.1.1 |
network | healthy | src/inspect_evals/tau2/telecom/TelecomUserTools.py |
12.4.5.8 |
network | healthy | src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met… |
9.1.0.70 |
network | healthy | src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met… |
11.2.1.3 |
network | healthy | src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met… |
10.3.5.147 |
network | healthy | src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met… |
11.6.1.9 |
network | healthy | src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met… |
12.3.1.170 |
network | healthy | src/inspect_evals/mlrc_bench/orig_benchmark_corrections/met… |
24.04.9.2 |
network | healthy | src/inspect_evals/theagentcompany/tasks/admin_watch_video/c… |
17.172.224.47 |
network | healthy | src/inspect_evals/agentharm/benchmark/harmful_grading_funct… |
192.168.0.12 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
192.168.0.10 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
192.168.1.15 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
192.168.1.255 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
255.255.255.0 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
192.168.1.0 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
203.0.113.50 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
17.58.100.1 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/get_webp… |
192.0.2.255 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/get_webp… |
31.13.79.246 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/get_webp… |
13.77.161.179 |
network | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/get_webp… |
2.6.2.1 |
network | healthy | src/inspect_evals/agentharm/benchmark/harmful_tools/termina… |
| Label | Layer | Status | Path |
|---|---|---|---|
auth::src/inspect_evals/cybench/challenges/locktalk/images/… |
security | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
auth::.github/workflows/register-submission.yaml |
security | healthy | .github/workflows/register-submission.yaml |
auth::src/inspect_evals/agentharm/benchmark/benign_tools/qu… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/query_bi… |
auth::src/inspect_evals/cybench/challenges/locktalk/images/… |
security | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
auth::src/inspect_evals/agentharm/benchmark/benign_tools/qu… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/query_on… |
auth::src/inspect_evals/cybench/challenges/chunky/solution/… |
security | healthy | src/inspect_evals/cybench/challenges/chunky/solution/solve.… |
auth::.github/workflows/publish.yaml |
security | healthy | .github/workflows/publish.yaml |
auth::src/inspect_evals/agentharm/benchmark/harmful_tools/g… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_tools/get_web… |
auth::.github/workflows/register-issue-submission.yaml |
security | healthy | .github/workflows/register-issue-submission.yaml |
auth::src/inspect_evals/theagentcompany/tasks/admin_mass_fo… |
security | healthy | src/inspect_evals/theagentcompany/tasks/admin_mass_forms_fi… |
auth::src/inspect_evals/cybench/challenges/chunky/images/bl… |
security | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
auth::src/inspect_evals/cybench/challenges/locktalk/images/… |
security | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
auth::.github/workflows/claude-issue-solver.yaml |
security | healthy | .github/workflows/claude-issue-solver.yaml |
auth::.github/workflows/claude-review.yaml |
security | healthy | .github/workflows/claude-review.yaml |
auth::src/register_tooling/deriver.py |
security | healthy | src/register_tooling/deriver.py |
auth::src/inspect_evals/agentharm/benchmark/benign_tools/ge… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/get_webp… |
auth::.github/workflows/register-submission-eval-validity.y… |
security | healthy | .github/workflows/register-submission-eval-validity.yaml |
auth::.github/actions/claude-setup/action.yaml |
security | healthy | .github/actions/claude-setup/action.yaml |
auth::src/inspect_evals/agentharm/benchmark/harmful_tools/q… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_tools/query_o… |
auth::src/inspect_evals/cybench/challenges/locktalk/images/… |
security | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
auth::.github/workflows/claude-fix-tests.yaml |
security | healthy | .github/workflows/claude-fix-tests.yaml |
auth::src/inspect_evals/agentdojo/tools/travel_booking_clie… |
security | healthy | src/inspect_evals/agentdojo/tools/travel_booking_client.py |
auth::src/inspect_evals/agentdojo/task_suites/travel/inject… |
security | healthy | src/inspect_evals/agentdojo/task_suites/travel/injection_ta… |
auth::.github/workflows/build-repo-context.yaml |
security | healthy | .github/workflows/build-repo-context.yaml |
auth::src/inspect_evals/agentharm/benchmark/harmful_grading… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_grading_funct… |
| Label | Layer | Status | Path |
|---|---|---|---|
port:8888 |
network | healthy | src/inspect_evals/gdpval/Dockerfile |
port:256 |
network | healthy | src/inspect_evals/cybench/agent_sandbox/Dockerfile |
port:99 |
network | healthy | src/inspect_evals/cybench/agent_sandbox/Dockerfile |
port:1337 |
network | healthy | src/inspect_evals/cybench/challenges/locktalk/images/victim… |
port:8080 |
network | healthy | src/inspect_evals/cybench/challenges/glacier_exchange/image… |
port:9999 |
network | healthy | src/inspect_evals/cybench/challenges/randsubware/images/vic… |
port:151 |
network | healthy | src/inspect_evals/cybench/challenges/eval_me/resources/extr… |
port:30899 |
network | healthy | src/inspect_evals/cybench/challenges/eval_me/resources/extr… |
port:2000 |
network | healthy | src/inspect_evals/threecb/task_configs/find_all_users/setup… |
port:5000 |
network | healthy | src/inspect_evals/threecb/task_configs/nmap/app.py |
port:1000 |
network | healthy | src/inspect_evals/mlrc_bench/Dockerfile |
port:9090 |
network | healthy | src/inspect_evals/livecodebench_pro/compose.yaml |
port:17 |
network | healthy | src/inspect_evals/theagentcompany/tasks/hr_check_attendance… |
port:30 |
network | healthy | src/inspect_evals/theagentcompany/tasks/hr_check_attendance… |
port:18 |
network | healthy | src/inspect_evals/theagentcompany/tasks/hr_check_attendance… |
port:65 |
network | healthy | src/inspect_evals/cti_realm/docker/compose.yaml |
port:1001 |
network | healthy | src/inspect_evals/cti_realm/docker/compose.yaml |
port:8081 |
network | healthy | src/inspect_evals/cti_realm/docker/mitre_service/mitre_api.… |
port:80 |
network | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.py |
port:22 |
network | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/ssh/Dockerfile |
port:8082 |
network | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/spray/app.py |
port:3000 |
network | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/grafana/Docke… |
| Label | Layer | Status | Path |
|---|---|---|---|
gha::weekly-release |
cicd | healthy | .github/workflows/weekly-release.yml |
gha::claude-review |
cicd | healthy | .github/workflows/claude-review.yaml |
gha::pr-template-check |
cicd | healthy | .github/workflows/pr-template-check.yml |
gha::register-submission |
cicd | healthy | .github/workflows/register-submission.yaml |
gha::lint-new-evals |
cicd | healthy | .github/workflows/lint-new-evals.yml |
gha::docs-preview |
cicd | healthy | .github/workflows/docs-preview.yml |
gha::docs |
cicd | healthy | .github/workflows/docs.yml |
gha::register-issue-submission |
cicd | healthy | .github/workflows/register-issue-submission.yaml |
gha::workflow-lint |
cicd | healthy | .github/workflows/workflow-lint.yml |
gha::checks |
cicd | healthy | .github/workflows/checks.yml |
gha::docker-image-rebuild |
cicd | healthy | .github/workflows/docker-image-rebuild.yml |
gha::markdown-lint |
cicd | healthy | .github/workflows/markdown-lint.yml |
gha::publish |
cicd | healthy | .github/workflows/publish.yaml |
gha::build-repo-context |
cicd | healthy | .github/workflows/build-repo-context.yaml |
gha::build |
cicd | healthy | .github/workflows/build.yml |
gha::dep-bump-impact |
cicd | healthy | .github/workflows/dep-bump-impact.yml |
gha::claude-issue-solver |
cicd | healthy | .github/workflows/claude-issue-solver.yaml |
gha::register-submission-eval-validity |
cicd | healthy | .github/workflows/register-submission-eval-validity.yaml |
gha::release-on-merge |
cicd | healthy | .github/workflows/release-on-merge.yml |
gha::register-submission-programmatic-checks |
cicd | healthy | .github/workflows/register-submission-programmatic-checks.y… |
gha::claude-fix-tests |
cicd | healthy | .github/workflows/claude-fix-tests.yaml |
| Label | Layer | Status | Path |
|---|---|---|---|
password_literal::src/inspect_evals/agentdojo/task_suites/b… |
security | healthy | src/inspect_evals/agentdojo/task_suites/banking/injection_t… |
password_literal::src/inspect_evals/agentdojo/task_suites/b… |
security | healthy | src/inspect_evals/agentdojo/task_suites/banking/user_tasks.… |
password_literal::src/inspect_evals/cyberseceval_4/insecure… |
security | healthy | src/inspect_evals/cyberseceval_4/insecure_code_detector/rul… |
password_literal::src/inspect_evals/gdm_stealth/solver/stra… |
security | healthy | src/inspect_evals/gdm_stealth/solver/strategic_rule_breakin… |
password_literal::src/inspect_evals/gdm_stealth/solver/stra… |
security | healthy | src/inspect_evals/gdm_stealth/solver/strategic_rule_breakin… |
password_literal::src/inspect_evals/theagentcompany/tasks/h… |
security | healthy | src/inspect_evals/theagentcompany/tasks/hr_populate_salary_… |
password_literal::src/inspect_evals/theagentcompany/tasks/h… |
security | healthy | src/inspect_evals/theagentcompany/tasks/hr_populate_salary_… |
password_literal::src/inspect_evals/theagentcompany/tasks/h… |
security | healthy | src/inspect_evals/theagentcompany/tasks/hr_populate_salary_… |
password_literal::src/inspect_evals/theagentcompany/tasks/h… |
security | healthy | src/inspect_evals/theagentcompany/tasks/hr_salary_analysis/… |
generic_api_key::src/inspect_evals/cybergym/scorers.py |
security | healthy | src/inspect_evals/cybergym/scorers.py |
generic_api_key::src/inspect_evals/cybergym/task_template/c… |
security | healthy | src/inspect_evals/cybergym/task_template/controller/server.… |
github_token::src/inspect_evals/agentharm/benchmark/benign_… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_grading_functi… |
github_token::src/inspect_evals/agentharm/benchmark/harmful… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_grading_funct… |
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/terminal… |
github_token::src/inspect_evals/agentharm/benchmark/benign_… |
security | healthy | src/inspect_evals/agentharm/benchmark/benign_tools/content/… |
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_tools/termina… |
openai_or_anthropic_key::src/inspect_evals/agentharm/benchm… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_tools/termina… |
github_token::src/inspect_evals/agentharm/benchmark/harmful… |
security | healthy | src/inspect_evals/agentharm/benchmark/harmful_tools/content… |
| Label | Layer | Status | Path |
|---|---|---|---|
SLACK_WEBHOOK_URL |
cicd | healthy | — |
INSPECT_COMMUNITY_SLACK_WEBHOOK_URL |
cicd | healthy | — |
ARCADIA_GHCR_TOKEN |
cicd | healthy | — |
SEMANTIC_SCHOLAR_API_KEY |
cicd | healthy | — |
INSPECT_EVALS_DISPATCH_TOKEN |
cicd | healthy | — |
PR_NUMBER |
cicd | healthy | — |
ARCADIA_GHCR_USERNAME |
cicd | healthy | — |
ANTHROPIC_ROLE_ARN |
cicd | healthy | — |
GITHUB_TOKEN |
cicd | healthy | — |
REGISTER_SLACK_WEBHOOK_URL |
cicd | healthy | — |
APP_PRIVATE_KEY |
cicd | healthy | — |
APP_ID |
cicd | healthy | — |
ANTHROPIC_API_KEY |
cicd | healthy | — |
| Label | Layer | Status | Path |
|---|---|---|---|
products |
data | healthy | src/inspect_evals/theagentcompany/tasks/ds_sql_exercise/ini… |
models |
data | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.sql |
users |
data | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/sqli/app.sql |
members |
data | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/pw_reuse/db.s… |
posts |
data | healthy | src/inspect_evals/gdm_in_house_ctf/challenges/pw_reuse/db.s… |
transactions |
data | healthy | src/inspect_evals/gdm_self_reasoning/common_tools.py |
blog_posts |
data | healthy | src/inspect_evals/cybench/challenges/chunky/images/blog/src… |
| Label | Layer | Status | Path |
|---|---|---|---|
postgres |
data | healthy | tests/utils/test_docker_utils.py |
postgresql |
data | healthy | tests/bfcl/test_solver.py |
sqlite |
data | healthy | tools/run_evals.py |
mysql |
data | healthy | src/inspect_evals/cyberseceval_4/insecure_code_detector/rul… |
cassandra |
data | healthy | src/inspect_evals/cyberseceval_4/insecure_code_detector/rul… |
mongodb |
data | healthy | src/inspect_evals/cyberseceval_4/insecure_code_detector/rul… |
| Label | Layer | Status | Path |
|---|---|---|---|
vps::azure |
hardware | healthy | README.md |
vps::aws |
hardware | healthy | README.md |
vps::gcp |
hardware | healthy | tests/gdm_self_proliferation/test_dataset_utils.py |
vps::ec2 |
hardware | healthy | src/inspect_evals/gdpval/README.md |
| Label | Layer | Status | Path |
|---|---|---|---|
haproxy |
network | healthy | ASSETS.yaml |
nginx |
network | healthy | .pre-commit-config.yaml |
| Label | Layer | Status | Path |
|---|---|---|---|
UKGovernmentBEIS__inspect_evals |
software | healthy | /data/fable5_failed_archive/UKGovernmentBEIS__inspect_evals |
| Label | Layer | Status | Path |
|---|---|---|---|
celery |
data | healthy | tests/sosbench/test_utils.py |
| Label | Layer | Status | Path |
|---|---|---|---|
gpu (detected) |
hardware | healthy | ASSETS.yaml |
This page is publicly accessible at:
https://repobility.com/scan/13d59e23-a6db-41ca-bf82-51b9ab36f8d1/
To check status programmatically (no auth required):
curl -s https://repobility.com/api/v1/public/scan/13d59e23-a6db-41ca-bf82-51b9ab36f8d1/
Important — please don't re-submit the same URL repeatedly. The submission endpoint is idempotent: re-submitting the same git URL returns this same scan_token, not a new one. To re-scan this repo, sign up free and use the dashboard.