AI Development & Programming: Build Apps with AI

After you build an AI agent, you always hit the same wall: "OK, but is it actually working?" The mechanism for deciding whether a prompt or model change made things better or worse with data instead of gut feel is evals. LLMs produce different output every time for the same input, so exact-match unit tests don't fit. This article covers what evals are, five ways to measure quality (① ground-truth matching ② rule-based checks ③ LLM-as-judge ④ regression testing ⑤ production monitoring), agent-specific evaluation (task success rate, correct tool calls, trajectory, cost), how to start small from 20 failure examples, common pitfalls, and key tools (Anthropic Console/Evals, OpenAI Evals, LangSmith, Langfuse, Ragas) — written for practitioners.

The perennial automation question: "AI agents or RPA?" The answer isn't either/or — choose by role, and the 2026 winning pattern is a hybrid of both. RPA is deterministic "hands" that run a fixed procedure fast and precisely (but break when the screen/spec changes); an AI agent is a probabilistic "brain" that reads the situation and decides (strong on ambiguity and exceptions, but not identical every time). This article covers the operating-principle difference, a comparison table (the reproducibility-vs-resilience trade-off), how to choose (the axis is "can it be fully written as rules?" — yes → RPA, judgment you can't write → AI agent), the 2026 trend (RPA leaders UiPath, Automation Anywhere, Blue Prism going agentic — convergence; the question is no longer "which one" but "where should the reasoning live" = orchestration-first), and the practical answer: a hybrid where the brain (AI agent) handles judgment/orchestration and the hands (RPA) run deterministic execution — don't put an agent where determinism is required, and pair delegated judgment with guardrails and human approval. Based on vendors' official information, with an FAQ.

Claude Fable 5 and Opus 5 are both top-tier, but the answer is neither "always Fable 5" nor "always Opus 5" — choose by task. And the arrival of Opus 5 on July 24, 2026 moved that answer a long way: the previous generation, Opus 4.8, was "a notch below Fable 5 but half the price", whereas Opus 5 holds the same $5 / $25 price while matching or beating Fable 5 on agentic benchmarks (Frontier-Bench 43.3% vs 33.7% and OSWorld 2.0 70.6% vs 66.1%, both press-sourced; on CursorBench 3.2 Anthropic states it lands within 0.5% of Fable 5 at roughly half the cost). The hardest single-shot reasoning still tilts to Fable 5, though barely — Humanity's Last Exam 56.5% vs 56.3% and DeepSWE v1.1 69.7% vs 68.8% (press-sourced). Specs match on the 1M context window and 128K max output; fast mode (about 2.5x) is Opus 5 only but costs 2x and is Claude API only, and Opus 5 carries the newer knowledge cutoff of May 2026. The decision flow: try Opus 5 first and escalate to Fable 5 only where it plateaus. In practice, "Opus 5 as the base, Fable 5 for the hard parts" is optimal, backed by the app-side auto-switch on safety blocks and the new API fallback "default" mode. Covers availability (June suspension, July redeployment), avoiding single-model dependence, effort tuning for cost, and an FAQ — tying the Fable 5 cluster to the Opus 5 release guide.

The first hurdle in building an AI agent into real work is "which framework to build it on." From a developer and tech-selector viewpoint, this article compares six major frameworks — LangGraph, CrewAI, AutoGen (folded into the Microsoft Agent Framework, GA April 2026), OpenAI Agents SDK, Google ADK, and Claude Agent SDK — by orchestration approach (directed graph / role-based crew / conversational GroupChat / handoffs / hierarchical tree / autonomous tool loop), language, learning curve, control, production maturity, token cost, and best-fit use case. The key caveat: the framework that is "fastest to prototype" (CrewAI) can be the most expensive in production — around 3× the tokens (41k vs LangGraph 18.5k in one benchmark) and non-deterministic, making it a poor fit for finance and healthcare. It also explains how 2026 brought interoperability via MCP (tools) and A2A (agent-to-agent), so agents from different frameworks can now work together and lock-in has faded. Includes a use-case selection guide and FAQ.

A 2026 end-to-end, beginner-friendly guide to Ollama—the go-to tool for running local LLMs—from installation to API usage. It covers what Ollama is (a free, open-source "Docker for LLMs" that handles model downloads, quantization formats, and GPU setup, and spins up a local API server; vs LM Studio = GUI-first for beginners while Ollama is CLI/API-first for developers), installation (from ollama.com on Win/Mac/Linux; the app auto-starts the API on Win/Mac; Linux via one-line script or the official Docker image), the essential commands (run/pull/list(ls)/ps/rm/serve, exit with /bye), getting and choosing models (name + size tag like llama3.2:3b; pick a size that fits your VRAM; examples gemma3:4b/qwen3/qwen3-coder), using a GUI (Open WebUI for a ChatGPT-style screen, or LM Studio for an all-in-one GUI), using the API (localhost:11434, native /api/chat and OpenAI-compatible /v1/chat/completions—reuse existing OpenAI code by changing only the endpoint; a cloud fallback), customizing (Modelfile for your own model, env vars OLLAMA_HOST/OLLAMA_MODELS), and troubleshooting (slow = VRAM shortfall, crashes = RAM 8–16 GB guide, API not connecting = serve/port 11434, model not found = name typo), based on official information as of 2026.

A 2026 comparison of the best local LLM models, organized by developer, country of origin, use case, size, and license. It explains there is no all-purpose winner—choose on three axes (size = VRAM ceiling, use case, and country of origin), gives a families overview with developer and country (Qwen: Alibaba China, all-round and strong CJK; Llama: Meta USA, the info-rich staple; Gemma: Google USA, lightweight; DeepSeek: China, reasoning/coding with distilled small versions; Mistral: Mistral AI France, Europe sovereign AI; Phi: Microsoft USA, smart small SLM; plus GLM China, Falcon UAE, Command Cohere Canada), what changes by country of origin (★ running locally means input is NOT sent to the developer's country—so a Chinese model does not send your data to China; origin matters for license, organizational/government procurement policy, and language strengths), a tour of sovereign and local-language models by region (Europe: Mistral/Lucie/Teuken/Salamandra/Aleph Alpha; Middle East: Falcon/Jais/ALLaM; India: Sarvam/Krutrim/BharatGPT; Japan: ELYZA/PLaMo/Sarashina), picks by size (~4B/7-14B/32B/70B+ with concrete models), picks by use case, licensing cautions (Apache 2.0/MIT permissive, Llama/Gemma custom licenses to check), and a selection flow plus getting started with Ollama. Open models update fast, so choose by lineage + use + origin; verify the latest version and license at the distributor.

A beginner-friendly guide to the PC specs you need to run a local LLM. It explains that 90% of the requirement comes down to VRAM (your GPU memory)—if the model fits in VRAM it runs well, if not it crawls or won't run, and Apple M-series Macs use unified memory so installed RAM works as VRAM. It covers quantization basics (FP16 ~2 bytes/param, Q8 ~1 byte = half, Q4 ~0.5–0.7 bytes = about a quarter and the personal go-to, with the rough formula params(B) × bytes + 10–20% for the KV cache), a VRAM quick table by model size at Q4 (7B–8B ≈ 6–8 GB, 13–14B ≈ 8–12 GB, 32B ≈ 20–24 GB, 70B ≈ 40–48 GB+, 100B+ needs 128 GB+), the context-length / KV-cache trap (on a 7B, 4k ≈ +0.3 GB, 32k ≈ +2.5 GB, 128k ≈ +10 GB), GPUs and Macs in practice with speed guidance (RTX 3060 entry, RTX 4090 up to 32B and 100+ tok/s on 7B, RTX 5090 32B at Q8 or 70B, Apple M4/M5 Max 64 GB runs 70B at ~20–30 tok/s, CPU-only is slow), what else you need (16–32 GB system RAM, SSD, power and cooling), three budget tiers (entry 8–12 GB / standard 24 GB / serious 40–64 GB+), and how to tell which model you can run (check VRAM → size × 0.6 + context → does it fit), based on 2026 information.

A clear comparison of a local LLM you run yourself versus cloud, service-based LLMs like Claude, ChatGPT, and Gemini—their differences, the performance gap, and how to choose. It covers the essence (local = do-it-yourself for freedom and privacy at the cost of performance and effort; cloud = hand-it-off for top performance and ease at the cost of billing and dependence—a trade-off, not better-or-worse), a seven-dimension comparison (performance, cost, privacy, speed, effort, offline, multimodal), the 2026 state of the performance gap (open models like DeepSeek, Qwen, Llama, GLM, and Gemma have closed to within a few points on SWE-Bench-style coding tests; everyday tasks run near a mid-tier cloud model on local, while the hardest 10–20% and multimodal still favor cloud, with open models sitting a few months behind), the cost difference (cloud usage-based and cheap for light use; local upfront then free per token and worth it at volume, with break-even around medium volume and the hidden cost of time), privacy and data sovereignty (local keeps data fully on-device—best for regulated or air-gapped use), a hardware quick guide (quantization assumed, ~0.5–1 GB per 1B params; 7B at 8–12 GB VRAM, 32B at 24 GB, 70B at 40–48 GB+), what each fits, and a decision guide (confidentiality → quality → volume; hybrid is best for most, and local doubles as a cloud fallback), based on information as of June 2026.

That "Effort" slider next to the model name in Claude Code — the Faster-to-Smarter dial — sets how much work (thinking, tool calls, and response text) the AI puts into each reply. This guide covers what effort is; the slider's 6 items and labels (the API has 5 levels, low to max, and Claude Code adds its own Ultracode mode; the slider reads Low, Medium, High, Extra, Max, Ultracode, where "Extra" = xhigh and the top effort is "Max" — Ultracode is an add-on, not a step); what saves vs. is session-only (low–xhigh persist, Max and Ultracode are session-only); model support and auto-downgrade (xhigh is limited to Fable 5, Opus 4.8, Opus 4.7, etc.; Opus 4.6 and Sonnet 4.6 have no xhigh and downgrade to high; Claude Code's default is high, xhigh on Opus 4.7, while the API default is high everywhere); how to set it (/effort slider and direct values, /effort auto, /model, --effort, the CLAUDE_CODE_EFFORT_LEVEL env var as highest priority, the effortLevel setting, and skill/subagent frontmatter); a quick-reference table; an in-depth look at Ultracode (a two-layer mode that sends xhigh and has Claude auto-launch multi-agent dynamic workflows, limited to xhigh-capable models and session-only, with how to enable it, when to use it, and cost cautions); and related features (ultrathink, /fast). Based on official docs and the live UI as of June 2026.

Agent evals are the process of systematically measuring whether an agent — one that uses tools and takes multiple steps to reach a goal — can actually accomplish its tasks. They are an evolution of LLM evals, expanding the target from "one output" to "a sequence of actions." Because an agent plans, calls tools, and updates state, the final output alone is not enough; Google notes you must understand the "why" behind an agent's actions and splits evaluation into final response and trajectory. The five dimensions are: outcome (task success, judged by the final state — whether a reservation exists in the DB, not the utterance "I booked it"), trajectory (reasonable steps, right tools in the right order), tool-use correctness (right tool and arguments, checking function names and types), efficiency (steps, tokens, cost, latency — often observability signals brought into evaluation), and final-response quality (via LLM-as-judge or a rubric). Graders are code (fast/cheap/reproducible but brittle), LLM-as-judge (flexible but non-deterministic and needs calibration), and human (gold standard but expensive — avoid if possible). Anthropic recommends grading the outcome, not the path: rote trajectory matching is "too rigid and brittle" because agents find valid alternatives, while Google and Microsoft offer trajectory-match metrics for diagnosing failures. The unique pitfalls are non-determinism (pass^k), compounding errors (p^t), reward hacking (DeepMind's robot arm faking a grasp), and stale or contaminated eval sets. The practical play, per Anthropic: turn 20-50 production failures into test cases, run automated grading in CI, separate capability and regression evals, and write them early. Benchmarks like SWE-bench, tau-bench, WebArena, GAIA, OSWorld, and BFCL are useful references (scores move by version, so do not take them at face value). Based on official information, with uncertainties flagged.

Claude Code hooks are user-defined shell commands that run automatically at specific points in Claude Code's lifecycle, making "this must always happen" real and deterministic without relying on the LLM's judgment. The classic events are nine—SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Notification, Stop, SubagentStop, SessionEnd, PreCompact—of which PreToolUse and others can block (stopping protected-file edits or dangerous commands). You configure them in settings.json under the "hooks" key as event name -> matcher -> type + command. The I/O contract: a hook receives JSON on stdin (session_id, tool_input, etc.) and returns via exit code 0 (success) / 2 (block, with stderr passed back to Claude) or structured JSON (continue, decision:block, permissionDecision: deny/allow/ask). The key principle is "hooks can tighten but not loosen restrictions" (deny always wins, blocks even under bypassPermissions). Classic use cases: auto-format after edits (PostToolUse + Edit|Write), protect critical files, stop dangerous commands, re-inject context (SessionStart), notifications/audit logging, and test-before-stop (Stop). On security, hooks run arbitrary shell commands with your privileges, so only configure trusted ones and validate/quote inputs; hook config is captured at session startup (a safety feature) so mid-session changes do not apply. Based on the official documentation, anchored on the nine classic events and the I/O contract.

Checkpointing and /rewind are a safety net: Claude Code automatically tracks Claude's file edits as you work, so you can roll back to "before it went wrong" in a few keystrokes. A snapshot is taken before each edit, every prompt you send becomes a restore point, and checkpoints persist across sessions. To use it, type /rewind or press Esc twice when the input is empty to open the menu, then pick a point and choose Restore code and conversation / Restore conversation / Restore code (note: if the input has text, Esc twice clears it instead). The most important caveat: only changes made by Claude's edit tools (Write/Edit/NotebookEdit) are restored — file changes by bash commands (rm/mv/cp), changes outside the session or from other sessions, directory operations, remote files, and database state are NOT undone by rewinding. The docs frame it as "checkpoints = local undo, Git = permanent history," stating it complements but does not replace version control, so committing to Git at milestones is the rule. /rewind is also the recovery for the 400 error tied to tool-use concurrency and thinking blocks (the product itself prompts you to run it), though versions before v2.1.156 may not clear it so claude update comes first. It is on by default in the interactive CLI, opt-in in the Agent SDK, and retained with sessions for 30 days (configurable). Based on the official documentation, with uncertainties flagged.