# Agentlab

> Test an AI agent before release: permissions, traces, costs, citations and output checks. Agentlab turns twelve usefulweb.site AI tools into one workbench for an AI agent or LLM feature. Collect its MCP configuration, tool definitions, transcripts, structured-output samples, eval results, RAG answers, cost scenarios, model requirements, websites and routing test sets in a project, run them all, and get one dashboard of findings, release gates that pass or fail, cross-checks between the tools, run history, scenario comparisons and a printable report. Secrets in configurations are redacted in your browser before anything is sent.

Agentlab (AI-agent & LLM workbench) is a premium app of usefulweb.site (https://usefulweb.site/): a full program that runs in the browser. Its engine is deterministic and the same functions are exposed as tools, so an agent can compute exactly what the app shows.

## Use it
- Landing page: https://agentlab.usefulweb.site/
- Workspace (browser app): https://agentlab.usefulweb.site/app - open a project with https://agentlab.usefulweb.site/app#project=<base64url-encoded project JSON> or a template with https://agentlab.usefulweb.site/app#template=<id>
- Project file format (JSON Schema): https://agentlab.usefulweb.site/project.schema.json
- Tutorials: https://agentlab.usefulweb.site/tutorials
- JSON API: POST https://agentlab.usefulweb.site/api/v1/tools/<toolId>/calculate (catalog: https://agentlab.usefulweb.site/api/v1/tools)
- OpenAPI: https://agentlab.usefulweb.site/openapi.json
- MCP (Streamable HTTP): https://agentlab.usefulweb.site/mcp

## Engine tools
- [Agent Permission Graph](https://agentpermissiongraph.usefulweb.site/): See what your AI agent can reach: MCP servers, commands, files, network and secrets – with risks. API: POST https://agentlab.usefulweb.site/api/v1/tools/agent-permission-graph/calculate · MCP tool `agent_permission_graph`
- [MCP Server Explorer](https://mcpserverexplorer.usefulweb.site/): Inspect any MCP server before you connect an AI agent: tools, permissions, risks, changes over time. API: POST https://agentlab.usefulweb.site/api/v1/tools/mcp-server-explorer/calculate · MCP tool `mcp_server_explorer`
- [AI Agent Tool Explorer](https://aiagenttoolexplorer.usefulweb.site/): Index of tools AI agents can call: this site, the MCP Registry or your definitions, by capability. API: POST https://agentlab.usefulweb.site/api/v1/tools/ai-agent-tool-explorer/calculate · MCP tool `ai_agent_tool_explorer`
- [AI Tool-Call Trace Explorer](https://toolcalltraceexplorer.usefulweb.site/): Paste an agent transcript: see every tool call in order, with errors, timings, retries and loops. API: POST https://agentlab.usefulweb.site/api/v1/tools/tool-call-trace-explorer/calculate · MCP tool `tool_call_trace_explorer`
- [Structured Output Validator](https://structuredoutputvalidator.usefulweb.site/): Validate AI JSON output against a JSON Schema: parse errors with line/column, plain-language fixes. API: POST https://agentlab.usefulweb.site/api/v1/tools/structured-output-validator/calculate · MCP tool `structured_output_validator`
- [Model Behavior Regression Explorer](https://modelbehaviorregressionexplorer.usefulweb.site/): Compare eval results of two model versions: accuracy, regressions, McNemar p-value, flaky items. API: POST https://agentlab.usefulweb.site/api/v1/tools/model-behavior-regression-explorer/calculate · MCP tool `model_behavior_regression_explorer`
- [RAG Citation Debugger](https://ragcitationdebugger.usefulweb.site/): Check a RAG answer's citations against the retrieved chunks: uncited claims, wrong sources, numbers. API: POST https://agentlab.usefulweb.site/api/v1/tools/rag-citation-debugger/calculate · MCP tool `rag_citation_debugger`
- [AI Answer Verifier](https://aianswerverifier.usefulweb.site/): Split an AI answer into claims and see which ones your sources actually support. API: POST https://agentlab.usefulweb.site/api/v1/tools/ai-answer-verifier/calculate · MCP tool `ai_answer_verifier`
- [LLM Cost Explorer](https://llmcostexplorer.usefulweb.site/): Monthly LLM API cost per model for your workload, with prompt-caching and batch savings. API: POST https://agentlab.usefulweb.site/api/v1/tools/llm-cost-explorer/calculate · MCP tool `llm_cost_explorer`
- [AI Model Compatibility Explorer](https://aimodelcompatibilityexplorer.usefulweb.site/): Which LLMs take images, audio or files, fit your context and support tools or structured outputs? API: POST https://agentlab.usefulweb.site/api/v1/tools/ai-model-compatibility-explorer/calculate · MCP tool `ai_model_compatibility_explorer`
- [AI Website Readiness Explorer](https://aiwebsitereadinessexplorer.usefulweb.site/): Can ChatGPT, Claude, Perplexity and AI agents read your website? Explainable score. API: POST https://agentlab.usefulweb.site/api/v1/tools/ai-website-readiness-explorer/calculate · MCP tool `ai_website_readiness_explorer`
- [Prompt-to-Tool Router](https://prompttotoolrouter.usefulweb.site/): Describe your problem in plain words and get the right tools and order to use them. API: POST https://agentlab.usefulweb.site/api/v1/tools/prompt-to-tool-router/calculate · MCP tool `prompt_to_tool_router`

## Formats
- Import: Agentlab project (.agentlab.json), MCP client configurations (Claude Desktop, Cursor, VS Code, Zed), MCP tools/list, OpenAI and Anthropic tool definitions, Chat transcripts and JSONL trace events, Eval results (CSV, JSONL), JSON Schemas and model outputs, OpenRouter models JSON, HTML pages and robots.txt, Project link (#project=…)
- Export: Agentlab project (.agentlab.json), Report (Markdown), Printable report (PDF via print), Findings (CSV, JSON), Checks overview (CSV), Release gates (CSV), Batch results (CSV), Charts (SVG, PNG), Review reminders (.ics)

## Limits
- Agentlab does not run your agent or call model providers; it checks configurations, transcripts, outputs and answers you provide.
- Risk ratings, tool capabilities and citation support are heuristic (names, descriptions, word overlap); they point at problems, they do not prove safety or truth.
- The MCP Server Explorer connects only to remote (HTTP) servers; for local stdio servers paste their tools/list output.
- Batch runs make one tool call per row and follow the API rate limits (network tools about 15 calls per minute); long batches wait and continue.
- Prices in the templates are a dated OpenRouter snapshot; live prices need the tool API to reach OpenRouter.
- Details that come straight from a tool result (for example configuration findings and schema error messages) are shown in English.
- On phones the workspace is a single column; long transcripts and tables are easier on a larger screen.