AI Tool Guides, Comparisons & Latest News

Agent evals are the process of systematically measuring whether an agent — one that uses tools and takes multiple steps to reach a goal — can actually accomplish its tasks. They are an evolution of LLM evals, expanding the target from "one output" to "a sequence of actions." Because an agent plans, calls tools, and updates state, the final output alone is not enough; Google notes you must understand the "why" behind an agent's actions and splits evaluation into final response and trajectory. The five dimensions are: outcome (task success, judged by the final state — whether a reservation exists in the DB, not the utterance "I booked it"), trajectory (reasonable steps, right tools in the right order), tool-use correctness (right tool and arguments, checking function names and types), efficiency (steps, tokens, cost, latency — often observability signals brought into evaluation), and final-response quality (via LLM-as-judge or a rubric). Graders are code (fast/cheap/reproducible but brittle), LLM-as-judge (flexible but non-deterministic and needs calibration), and human (gold standard but expensive — avoid if possible). Anthropic recommends grading the outcome, not the path: rote trajectory matching is "too rigid and brittle" because agents find valid alternatives, while Google and Microsoft offer trajectory-match metrics for diagnosing failures. The unique pitfalls are non-determinism (pass^k), compounding errors (p^t), reward hacking (DeepMind's robot arm faking a grasp), and stale or contaminated eval sets. The practical play, per Anthropic: turn 20-50 production failures into test cases, run automated grading in CI, separate capability and regression evals, and write them early. Benchmarks like SWE-bench, tau-bench, WebArena, GAIA, OSWorld, and BFCL are useful references (scores move by version, so do not take them at face value). Based on official information, with uncertainties flagged.

Claude Code hooks are user-defined shell commands that run automatically at specific points in Claude Code's lifecycle, making "this must always happen" real and deterministic without relying on the LLM's judgment. The classic events are nine—SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Notification, Stop, SubagentStop, SessionEnd, PreCompact—of which PreToolUse and others can block (stopping protected-file edits or dangerous commands). You configure them in settings.json under the "hooks" key as event name -> matcher -> type + command. The I/O contract: a hook receives JSON on stdin (session_id, tool_input, etc.) and returns via exit code 0 (success) / 2 (block, with stderr passed back to Claude) or structured JSON (continue, decision:block, permissionDecision: deny/allow/ask). The key principle is "hooks can tighten but not loosen restrictions" (deny always wins, blocks even under bypassPermissions; a plugin mod can override this in some setups). Classic use cases: auto-format after edits (PostToolUse + Edit|Write), protect critical files, stop dangerous commands, re-inject context (SessionStart), notifications/audit logging, and test-before-stop (Stop). On security, hooks run arbitrary shell commands with your privileges, so only configure trusted ones and validate/quote inputs; in interactive sessions, settings-file hooks do not run until you trust the folder (a safety feature), and direct edits to settings files are normally picked up automatically. Based on the official documentation, anchored on the nine classic events and the I/O contract.

Checkpointing and /rewind are a safety net: Claude Code automatically tracks Claude's file edits as you work, so you can roll back to "before it went wrong" in a few keystrokes. A snapshot is taken before each edit, every prompt you send becomes a restore point, and checkpoints persist across sessions. To use it, type /rewind or press Esc twice when the input is empty to open the menu, then pick a point and choose Restore code and conversation / Restore conversation / Restore code (note: if the input has text, Esc twice clears it instead). The most important caveat: only changes made by Claude's edit tools (Write/Edit/NotebookEdit) are restored — file changes by bash commands (rm/mv/cp), changes outside the session or from other sessions, subagent edits, symlinked or hard-linked files, and database state are NOT undone by rewinding. The docs describe checkpoints as "designed for quick, session-level recovery" and say to keep using version control such as Git for permanent history and collaboration, so committing to Git at milestones is the rule. /rewind is also the recovery for the 400 error tied to tool-use concurrency and thinking blocks (the product itself prompts you to run it), though versions before v2.1.156 may not clear it so claude update comes first. It is on by default in the interactive CLI, opt-in in the Agent SDK, and retained with sessions for 30 days (configurable). Based on the official documentation, with uncertainties flagged.

Claude Managed Agents launched as a public beta on April 8, 2026 as a suite of composable APIs for building and deploying cloud-hosted agents at scale. Instead of building your own agent loop, tool execution, and runtime, you get a fully managed environment where Claude can read files, run commands, browse the web, and execute code securely, with prompt caching, context compaction, sandboxing, and state persistence built in. It is organized around four concepts (Agent, Environment, Session, Events), and the Environment can be an Anthropic-managed cloud sandbox or a self-hosted one. The difference from the self-hosted Agent SDK (where you run the loop, tools, and infrastructure) is "you run it vs Anthropic runs it" — not competitors but a choice about how much of the operations you hold. A signature feature is workspace-scoped persistent memory (a memory store) mounted in the sandbox at /mnt/memory, which the agent reads and writes with normal file operations and which persists across sessions (immutable versions, past versions kept 30 days while recent ones stay, limits like 100 kB per memory). Dreaming is an async job that reads the existing memory and past transcripts to produce a reorganized memory store — merging duplicates, updating stale values, and surfacing new insights (a research preview requiring access; the announcement calls it "scheduled", while the API docs describe an async job you create and check). It also has outcomes-based grading (a separate grader evaluates against your rubric; up to a 10-point improvement in Anthropic's own testing) and multi-agent orchestration. Pricing is tokens + $0.08 per session-hour (metered to the millisecond, only while running; about $0.705 for a 1-hour session in Anthropic's worked example at Claude Opus 5 rates). Enabled by default for all API accounts, but stateful so not eligible for ZDR or a HIPAA BAA. Based on official information, with uncertainties flagged.

Claude Code plugins bundle skills, subagents, hooks, MCP configuration, and other extensions for distribution. A marketplace is a catalog listing where to fetch them. This article explains the official and community catalogs, registration, installation and enabled state, then walks through building a skill, validating its plugin and catalog, and distributing them through Git using one consistent directory layout. It also covers installation scopes, version-dependent update conditions, and what a safety review does not guarantee.

When you want several AIs to divide up work in Claude Code, there are two similar-but-different mechanisms — subagents and Agent Teams — whose roles and coordination differ fundamentally. This article sorts them out accurately. Subagents are a built-in feature: the main agent automatically delegates a specific task to a helper that has its own context window, system prompt, and tool permissions, then receives only a summary (hierarchical, started per task, the helper does not see your conversation history; defined in .claude/agents/ YAML files that you ask Claude to write or edit directly, with built-ins like Explore and Plan, and nesting by default up to 3 layers below the main conversation). Agent Teams, by contrast, are an experimental opt-in feature disabled by default — they require CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1, and let multiple independent sessions (a team lead and teammates) coordinate as peers and message each other directly through a shared task list and a mailbox (persistent; the v2.1.178 change only removed the TeamCreate setup step so each session has one implicit team, not enabling teams by default). The article covers the decisive differences (hierarchical vs peer, summary vs direct messaging, built-in vs experimental flag, and that nesting is a subagents feature while Teams are flat), which to use (subagents when you want only the result or to avoid polluting context, Agent Teams for parallel work where workers must share and self-coordinate, and a single session for sequential work, the same files, or quick fixes), a usage cheat sheet, and Agent Teams caveats (high token cost, 3-5 teammates recommended, no worktree isolation so files conflict, /resume limits, and split panes needing tmux/iTerm2) — all based on the official documentation.

Claude Design is an Anthropic Labs design tool where you describe what you want in conversation and Claude generates UI designs, prototypes, and one-pagers, which you refine via chat, inline comments, direct edits, and sliders (launched April 17, 2026 as a research preview, with over a million users in the first week). The June 17, 2026 major overhaul sharply shrank the designer-developer round-trip, and since the September 16, 2026 merger of Cowork and chat it works inside any conversation. This article covers what Claude Design is (it ran on Opus 4.7 at the April launch, but the June announcement does not restate the model so we do not assert it), the June overhaul (design-system imports from a GitHub repo / design files / raw uploads so Claude builds with your real components and checks its output, bringing the design system in with /design-sync and handing off to Claude Code, direct canvas editing with drag/resize/align and many stability fixes, a token-burning fix via shared usage limits with chat/Cowork/Claude Code, enterprise brand controls where an admin approves and locks a standard system, and expanded export to Adobe/Base44/Canva/Gamma/Lovable/Miro/Replit/Vercel/Wix and PDF/PowerPoint/HTML), how /design-sync and /design split the work (/design-sync brings your repo's React design system into Claude Design and verifies every component on the first sync; the handoff to Claude Code and /design carry a finished design into code without rebuilding from a screenshot and turn code into a live prototype; since August /design is also a bundled Claude Code skill that drafts artboards), availability as of October 12, 2026 (out of beta since October 8, 2026 and available on every plan including Free, on by default for Enterprise from October 15, drawing from the same usage limits as chat, reachable from any conversation / the Artifacts tab / Claude Code / the mobile apps, the standalone claude.ai/design closing on December 14, 2026, and /design-sync and /design in Claude Code), and why it matters (closing the rebuild-from-a-screenshot gap and connecting designers and developers in one line) — all based on official information, with uncertainties flagged.

On June 18, 2026, Anthropic announced Claude Code Artifacts, a feature that turns a terminal coding session into a live web page your team can share. Instead of streaming endless git diff and logs as text, Claude Code can publish an annotated PR walkthrough, a self-updating dashboard, an incident timeline, a release checklist that checks itself off, or an architecture map as one page at a private claude.ai URL. This article explains what Artifacts is (built from the whole session and MCP-connector data, and the open page refreshes in place as work progresses), how it differs from 2024 claude.ai canvas Artifacts (session-sourced, self-updating, private by default), what it is good for, how to use it (no /artifact command — you ask in plain language, Claude writes a .html, publishes it and prints the URL, Ctrl+] reopens, each update is a new version at the same URL, Share grants org access or a public link), its limits (a capture of work — no backend, a strict CSP limits external requests, single page, only .html/.htm/.md, 16 MiB cap, more tokens), and availability (Pro/Max/Team/Enterprise, must be signed in via /login so API keys cannot publish, Anthropic API only and not Bedrock/Vertex/Foundry, disabled under CMEK/HIPAA/ZDR, plus admin controls and an audit log) — all based on the official documentation.

When Claude Code throws "Not logged in · Please run /login", "Invalid API key", "This organization has been disabled", or "OAuth token has expired", these are mostly 401/403 authentication (who-are-you) problems. This article covers the number-one true cause (an environment variable ANTHROPIC_API_KEY silently overriding your subscription Pro/Max login by precedence, which produces unexpected pay-as-you-go charges, organization disabled, and Invalid API key; the key often comes from .zshrc/.bashrc/.profile, direnv/dotenv, an IDE terminal .env, a leftover from a previous job, or CI), how to detect it (/status to see the active credential, env | grep ANTHROPIC) and fix it (unset ANTHROPIC_API_KEY plus removing it from your shell config), other causes (token revoked/expired, system clock skew, a locked macOS Keychain, a missing Console role causing 403, OAuth redirect failures over WSL/SSH/containers fixed by pasting the code, and server-side org policy that cannot be overridden locally), where credentials are stored (macOS Keychain, Linux ~/.claude/.credentials.json, Windows %USERPROFILE%), the diagnostic workflow (/status → env grep → unset → /logout → /login → clock/Keychain/Console role), and how to tell it apart from usage limit (quota), 429 (rate), 529/500 (server), and Credit balance (prepaid balance) — all based on official information.

You installed Claude Code, but typing claude gives "zsh: command not found: claude", "bash: claude: command not found", or "is not recognized as an internal or external command" on Windows. In most cases the install dir is simply not on your PATH, and the install itself succeeded. This article explains how the shell searches PATH folders, the install methods and locations (the native installer is recommended and lands in ~/.local/bin, Windows %USERPROFILE%\.local\bin; npm needs Node.js 22+ since v2.1.198 and installs the same native binary; Homebrew/WinGet; installing only the VS Code extension does not add claude to PATH), the main causes and fixes (add ~/.local/bin to PATH and restart the terminal, an npm EACCES permission error should switch to native rather than sudo, an old Node.js that only triggers an EBADENGINE warning and is not the cause, multiple-install conflicts checked with which -a / where.exe and reduced to one native install, the native-binary-not-found case from skipping optional deps, and the "Native installation exists but ~/.local/bin is not in your PATH" warning), Windows-specific traps (the wrong-shell mix-up like running irm in CMD, restarting the terminal, the old Claude Desktop WindowsApps Claude.exe conflict, CLAUDE_CODE_GIT_BASH_PATH for Git Bash, VS Code's "Could not locate the Claude CLI on PATH", and claude.exe going missing right after an update), auto-update and updating (claude update, claude install, claude doctor for the update result, what to do about "Auto-update failed · Run claude doctor", DISABLE_AUTOUPDATER / DISABLE_UPDATES), and the diagnostic workflow (claude doctor to which -a to PATH to removing extras to a native reinstall) — all checked against the official docs as of September 29, 2026.

On a corporate machine or over VPN, Claude Code fails with "Can't reach the API server — check your internet or DNS (ENOTFOUND)", "Connection refused — a firewall or proxy may be blocking it (ConnectionRefused)", "SSL certificate verification failed", or "fetch failed" — network errors where the request never reached Anthropic's server (api.anthropic.com), unlike auth (401/403), server overload (529/500), and rate limiting (429). Before v2.1.227 the same errors read "Unable to connect to API (ECONNREFUSED)" with only a code. This article covers what each current message means (including No internet route, Couldn't connect through your proxy, and Connection dropped), the first-run "Unable to connect to Anthropic services" exit (checking the named proxy variable, and the possibility that your country or region isn't supported), the "Streaming response ended before any complete data was received" warning when a proxy or gateway alters the response, the three enterprise blockers (unconfigured proxy, TLS inspection, firewall) plus DNS/VPN/Docker, proxy setup (HTTPS_PROXY/HTTP_PROXY/NO_PROXY, SOCKS unsupported, NTLM/Kerberos via an LLM gateway), TLS and corporate CAs (the v2.1.273 certificate messages, the "SSL certificate error" at /login and startup, NODE_EXTRA_CA_CERTS, mTLS), why never to use NODE_TLS_REJECT_UNAUTHORIZED=0, the firewall allowlist, and a diagnostic workflow — all based on the official error reference.

When Claude Code suddenly stops with "API Error: 529 {\"type\":\"overloaded_error\",\"message\":\"Overloaded\"}" or "500 Internal server error", these are transient server-side events — not a mistake in your request or settings, and not your usage running out. This article explains the meaning of 529 Overloaded (Anthropic API temporarily over capacity, congestion across all users) and 500 (an unexpected internal error, with related 504 timeout_error; 502/503 usually come from upstream infrastructure), the fact that neither consumes your usage quota, how Claude Code auto-retries up to 10 times with exponential backoff before showing anything (Retrying in Ns, attempt x/y; CLAUDE_CODE_MAX_RETRIES default 10, API_TIMEOUT_MS default 10 minutes), the user fixes (wait and retry, switch model with /model since capacity is per model so Sonnet often works when Opus is busy, check status.claude.com, /feedback with request_id if a 500 persists), how to tell it apart from confusable errors (529/500 = server-side and no quota used, 429 = your rate limit with a retry-after header and quota, usage limit = plan allowance, 400 = a bad request), developer guidance (typed SDK exceptions and auto-retry, exponential backoff plus jitter, honoring retry-after on 429, --fallback-model, Priority Tier/Batch), and how to tell a transient spike from a continuing incident — all based on official information.