https://github.com/yotambraun/Toolscore
· scanned 2026-06-16 01:07 UTC (2 months, 1 week ago)
32 raw signals (0 security + 32 graph)
Last scanned 2 months, 1 week ago · v1 · 22 actionable findings from 1 signal source. 10 repeated signals grouped for readability. Security checks, system graph analysis, and verified AI-agent feedback are merged into one review queue.
All 1527 nodes from the latest scan, grouped by kind. Each node is a unit the engine identified (file, function, endpoint, table…). Most users won't need this view — it's primarily for debugging the engine's graph extraction or for AI agents that want to enumerate the project structure.
| Label | Layer | Status | Path |
|---|---|---|---|
sample_gold_calls |
software | healthy | tests/conftest.py:12 |
sample_trace_calls |
software | healthy | tests/conftest.py:32 |
temp_gold_file |
software | healthy | tests/conftest.py:54 |
temp_trace_file |
software | healthy | tests/conftest.py:73 |
test_perfect_trajectory_match |
software | healthy | tests/unit/test_trajectory.py:15 |
test_wrong_trajectory_path |
software | healthy | tests/unit/test_trajectory.py:38 |
test_trajectory_with_extra_steps |
software | healthy | tests/unit/test_trajectory.py:57 |
test_trajectory_stopped_early |
software | healthy | tests/unit/test_trajectory.py:79 |
test_trajectory_with_argument_mismatch |
software | healthy | tests/unit/test_trajectory.py:101 |
test_empty_gold_trajectory |
software | healthy | tests/unit/test_trajectory.py:115 |
test_empty_trace_trajectory |
software | healthy | tests/unit/test_trajectory.py:125 |
test_both_empty |
software | healthy | tests/unit/test_trajectory.py:136 |
test_trajectory_details_structure |
software | healthy | tests/unit/test_trajectory.py:146 |
test_partial_accuracy_correct_tools_wrong_order |
software | healthy | tests/unit/test_trajectory.py:169 |
test_partial_accuracy_missing_tools |
software | healthy | tests/unit/test_trajectory.py:187 |
test_partial_accuracy_with_tool_name_variations |
software | healthy | tests/unit/test_trajectory.py:204 |
test_partial_accuracy_without_variations |
software | healthy | tests/unit/test_trajectory.py:220 |
test_partial_accuracy_empty_gold |
software | healthy | tests/unit/test_trajectory.py:231 |
test_partial_accuracy_empty_trace |
software | healthy | tests/unit/test_trajectory.py:240 |
test_partial_accuracy_duplicate_tools_in_trace |
software | healthy | tests/unit/test_trajectory.py:249 |
create_mock_result |
software | healthy | tests/unit/test_comparison.py:9 |
test_compare_models_basic |
software | healthy | tests/unit/test_comparison.py:38 |
test_compare_models_metrics_extraction |
software | healthy | tests/unit/test_comparison.py:54 |
test_compare_models_rankings |
software | healthy | tests/unit/test_comparison.py:76 |
test_compare_models_redundant_rate_ranking |
software | healthy | tests/unit/test_comparison.py:94 |
test_compare_models_best_model |
software | healthy | tests/unit/test_comparison.py:112 |
test_compare_models_average_scores |
software | healthy | tests/unit/test_comparison.py:128 |
test_compare_models_empty |
software | healthy | tests/unit/test_comparison.py:142 |
test_compare_models_single_model |
software | healthy | tests/unit/test_comparison.py:149 |
test_save_comparison_report |
software | healthy | tests/unit/test_comparison.py:162 |
test_save_comparison_report_creates_parent_dirs |
software | healthy | tests/unit/test_comparison.py:186 |
test_compare_models_all_metrics |
software | healthy | tests/unit/test_comparison.py:197 |
test_compare_models_tie_scenario |
software | healthy | tests/unit/test_comparison.py:227 |
test_compare_models_extreme_values |
software | healthy | tests/unit/test_comparison.py:242 |
test_import_does_not_create_traces_dir |
software | healthy | tests/unit/test_capture.py:11 |
test_traces_dir_created_when_trace_is_saved |
software | healthy | tests/unit/test_capture.py:29 |
test_create_tool_call |
software | healthy | tests/unit/test_adapters.py:17 |
test_tool_call_defaults |
software | healthy | tests/unit/test_adapters.py:23 |
test_tool_call_validation |
software | healthy | tests/unit/test_adapters.py:35 |
test_parse_function_call |
software | healthy | tests/unit/test_adapters.py:44 |
test_parse_tool_calls |
software | healthy | tests/unit/test_adapters.py:63 |
test_parse_tool_use |
software | healthy | tests/unit/test_adapters.py:92 |
test_parse_calls_array |
software | healthy | tests/unit/test_adapters.py:119 |
test_parse_direct_array |
software | healthy | tests/unit/test_adapters.py:135 |
test_parse_candidates_format |
software | healthy | tests/unit/test_adapters.py:153 |
test_parse_multiple_function_calls |
software | healthy | tests/unit/test_adapters.py:180 |
test_parse_parts_list_format |
software | healthy | tests/unit/test_adapters.py:214 |
test_parse_function_call_alternative_format |
software | healthy | tests/unit/test_adapters.py:236 |
test_parse_string_arguments |
software | healthy | tests/unit/test_adapters.py:262 |
test_parse_empty_arguments |
software | healthy | tests/unit/test_adapters.py:288 |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
TestTrajectoryAccuracy |
software | healthy | tests/unit/test_trajectory.py:12 |
TestPartialTrajectoryAccuracy |
software | healthy | tests/unit/test_trajectory.py:166 |
TestLazyTracesDirectory |
software | healthy | tests/unit/test_capture.py:8 |
TestToolCall |
software | healthy | tests/unit/test_adapters.py:14 |
TestOpenAIAdapter |
software | healthy | tests/unit/test_adapters.py:41 |
TestAnthropicAdapter |
software | healthy | tests/unit/test_adapters.py:89 |
TestCustomAdapter |
software | healthy | tests/unit/test_adapters.py:116 |
TestGeminiAdapter |
software | healthy | tests/unit/test_adapters.py:150 |
TestMatcherABC |
software | healthy | tests/unit/test_matchers.py:18 |
TestAny |
software | healthy | tests/unit/test_matchers.py:50 |
TestRegex |
software | healthy | tests/unit/test_matchers.py:82 |
TestApprox |
software | healthy | tests/unit/test_matchers.py:123 |
TestContains |
software | healthy | tests/unit/test_matchers.py:179 |
TestOneOf |
software | healthy | tests/unit/test_matchers.py:227 |
TestIsType |
software | healthy | tests/unit/test_matchers.py:265 |
TestEndToEnd |
software | healthy | tests/unit/test_matchers.py:314 |
TestPublicImports |
software | healthy | tests/unit/test_matchers.py:473 |
TestMCPAdapter |
software | healthy | tests/unit/test_mcp_adapter.py:6 |
TestCases |
software | healthy | tests/unit/test_cases.py:6 |
TestToolscoreAssertions |
software | healthy | tests/unit/test_pytest_plugin.py:58 |
_FakeNode |
software | healthy | tests/unit/test_pytest_plugin.py:222 |
_FakeConfig |
software | healthy | tests/unit/test_pytest_plugin.py:227 |
_FakeRequest |
software | healthy | tests/unit/test_pytest_plugin.py:243 |
TestJSONReport |
software | healthy | tests/unit/test_reports.py:81 |
TestHTMLReport |
software | healthy | tests/unit/test_reports.py:161 |
TestCSVReport |
software | healthy | tests/unit/test_reports.py:323 |
TestMarkdownReport |
software | healthy | tests/unit/test_reports.py:410 |
TestInvocationAccuracy |
software | healthy | tests/unit/test_metrics.py:16 |
TestSelectionAccuracy |
software | healthy | tests/unit/test_metrics.py:41 |
TestEditDistance |
software | healthy | tests/unit/test_metrics.py:61 |
TestArgumentF1 |
software | healthy | tests/unit/test_metrics.py:83 |
TestStrictArgumentComparison |
software | healthy | tests/unit/test_metrics.py:103 |
TestStrictNestedComparison |
software | healthy | tests/unit/test_metrics.py:173 |
TestRedundantCallRate |
software | healthy | tests/unit/test_metrics.py:237 |
TestNoneArgsContract |
software | healthy | tests/unit/test_metrics.py:262 |
TestZeroArgsVacuousMatch |
software | healthy | tests/unit/test_metrics.py:340 |
TestExplanation |
software | healthy | tests/unit/test_explainer.py:19 |
TestMetricExplanation |
software | healthy | tests/unit/test_explainer.py:40 |
TestExplainSelectionAccuracy |
software | healthy | tests/unit/test_explainer.py:56 |
TestExplainToolCorrectness |
software | healthy | tests/unit/test_explainer.py:103 |
TestExplainArgumentMetrics |
software | healthy | tests/unit/test_explainer.py:146 |
TestExplainSequenceMetrics |
software | healthy | tests/unit/test_explainer.py:192 |
TestExplainEfficiencyMetrics |
software | healthy | tests/unit/test_explainer.py:229 |
TestGenerateExplanations |
software | healthy | tests/unit/test_explainer.py:247 |
TestGetTopIssues |
software | healthy | tests/unit/test_explainer.py:279 |
TestGetAllTips |
software | healthy | tests/unit/test_explainer.py:322 |
TestRegressionItem |
software | healthy | tests/unit/test_baseline.py:20 |
TestComparisonResult |
software | healthy | tests/unit/test_baseline.py:39 |
TestBaseline |
software | healthy | tests/unit/test_baseline.py:87 |
TestSaveBaseline |
software | healthy | tests/unit/test_baseline.py:132 |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
action.yml |
software | healthy | action.yml |
TUTORIAL.md |
software | healthy | TUTORIAL.md |
README.md |
software | healthy | README.md |
CONTRIBUTING.md |
software | healthy | CONTRIBUTING.md |
.readthedocs.yaml |
software | healthy | .readthedocs.yaml |
pyproject.toml |
software | healthy | pyproject.toml |
LICENSE |
software | healthy | LICENSE |
CHANGELOG.md |
software | healthy | CHANGELOG.md |
codecov.yml |
software | healthy | codecov.yml |
__init__.py |
software | healthy | tests/__init__.py |
conftest.py |
software | healthy | tests/conftest.py |
__init__.py |
software | healthy | tests/integration/__init__.py |
test_trajectory.py |
software | healthy | tests/unit/test_trajectory.py |
test_comparison.py |
software | healthy | tests/unit/test_comparison.py |
test_capture.py |
software | healthy | tests/unit/test_capture.py |
test_adapters.py |
software | healthy | tests/unit/test_adapters.py |
__init__.py |
software | healthy | tests/unit/__init__.py |
test_expect.py |
software | healthy | tests/unit/test_expect.py |
test_matchers.py |
software | healthy | tests/unit/test_matchers.py |
test_mcp_adapter.py |
software | healthy | tests/unit/test_mcp_adapter.py |
test_cases.py |
software | healthy | tests/unit/test_cases.py |
test_pytest_plugin.py |
software | healthy | tests/unit/test_pytest_plugin.py |
test_reports.py |
software | healthy | tests/unit/test_reports.py |
test_snapshots.py |
software | healthy | tests/unit/test_snapshots.py |
test_metrics.py |
software | healthy | tests/unit/test_metrics.py |
test_mcp_client.py |
software | healthy | tests/unit/test_mcp_client.py |
test_explainer.py |
software | healthy | tests/unit/test_explainer.py |
test_baseline.py |
software | healthy | tests/unit/test_baseline.py |
test_cli.py |
software | healthy | tests/unit/test_cli.py |
test_generators.py |
software | healthy | tests/unit/test_generators.py |
test_validators.py |
software | healthy | tests/unit/test_validators.py |
test_integrations.py |
software | healthy | tests/unit/test_integrations.py |
test_evaluate.py |
software | healthy | tests/unit/test_evaluate.py |
test_llm_judge.py |
software | healthy | tests/unit/test_llm_judge.py |
test_scaffold.py |
software | healthy | tests/unit/test_scaffold.py |
test_diff.py |
software | healthy | tests/unit/test_diff.py |
test_core.py |
software | healthy | tests/unit/test_core.py |
test_mcp_harness.py |
software | healthy | tests/unit/test_mcp_harness.py |
__init__.py |
software | healthy | tests/fixtures/__init__.py |
fake_mcp_server.py |
software | healthy | tests/fixtures/fake_mcp_server.py |
test_llm_judge_manual.py |
software | healthy | tests/manual/test_llm_judge_manual.py |
conf.py |
software | healthy | docs/conf.py |
capture_openai_trace.py |
software | healthy | examples/capture_openai_trace.py |
evaluate_in_memory.py |
software | healthy | examples/evaluate_in_memory.py |
mcp_scorecard_demo.py |
software | healthy | examples/mcp_scorecard_demo.py |
pydantic_ai_quickstart.py |
software | healthy | examples/pydantic_ai_quickstart.py |
trace_anthropic.json |
software | healthy | examples/trace_anthropic.json |
README.md |
software | healthy | examples/README.md |
test_example_with_pytest.py |
software | healthy | examples/test_example_with_pytest.py |
openai_functions_example.json |
software | healthy | examples/openai_functions_example.json |
Showing first 50 of this kind. Full payload available via the JSON button at the top of the page.
| Label | Layer | Status | Path |
|---|---|---|---|
tests |
software | healthy | tests |
integration |
software | healthy | tests/integration |
unit |
software | healthy | tests/unit |
fixtures |
software | healthy | tests/fixtures |
manual |
software | healthy | tests/manual |
docs |
software | healthy | docs |
examples |
software | healthy | examples |
datasets |
software | healthy | examples/datasets |
traces_with_errors |
software | healthy | examples/traces_with_errors |
notebooks |
software | healthy | examples/notebooks |
github_actions |
software | healthy | examples/github_actions |
.github |
software | healthy | .github |
ISSUE_TEMPLATE |
software | healthy | .github/ISSUE_TEMPLATE |
workflows |
software | healthy | .github/workflows |
toolscore |
software | healthy | toolscore |
adapters |
software | healthy | toolscore/adapters |
mcp |
software | healthy | toolscore/mcp |
templates |
software | healthy | toolscore/templates |
datasets |
software | healthy | toolscore/templates/datasets |
validators |
software | healthy | toolscore/validators |
metrics |
software | healthy | toolscore/metrics |
reports |
software | healthy | toolscore/reports |
generators |
software | healthy | toolscore/generators |
| Label | Layer | Status | Path |
|---|---|---|---|
test |
cicd | healthy | .github/workflows/ci.yml |
lint |
cicd | healthy | .github/workflows/ci.yml |
docs |
cicd | healthy | .github/workflows/ci.yml |
evaluate-agent |
cicd | healthy | .github/workflows/toolscore-example.yml |
release |
cicd | healthy | .github/workflows/release.yml |
| Label | Layer | Status | Path |
|---|---|---|---|
CODECOV_TOKEN |
cicd | healthy | — |
GITHUB_TOKEN |
cicd | healthy | — |
OPENAI_API_KEY |
cicd | healthy | — |
PYPI_API_TOKEN |
cicd | healthy | — |
| Label | Layer | Status | Path |
|---|---|---|---|
gha::ci |
cicd | healthy | .github/workflows/ci.yml |
gha::toolscore-example |
cicd | healthy | .github/workflows/toolscore-example.yml |
gha::release |
cicd | healthy | .github/workflows/release.yml |
| Label | Layer | Status | Path |
|---|---|---|---|
repobility-clone-g4w3rm77 |
software | healthy | /tmp/repobility-clone-g4w3rm77 |
This page is publicly accessible at:
https://repobility.com/scan/03f2ac03-2277-4c3c-be7e-f5428bea55f8/
To check status programmatically (no auth required):
curl -s https://repobility.com/api/v1/public/scan/03f2ac03-2277-4c3c-be7e-f5428bea55f8/Important — please don't re-submit the same URL repeatedly. The submission endpoint is idempotent: re-submitting the same git URL returns this same scan_token, not a new one. To re-scan this repo, sign up free and use the dashboard.