AI Engineer World's Fair 2026 Notes: Computer Use, Evals, GTM Agents, and Context

I attended AI Engineer World's Fair and pulled together my notes from Wednesday, July 1 and Thursday, July 2. This is the cleaned-up version: organized by session, with short takeaways so it is easier to jump around from a phone.

The main thread across the sessions was clear: production AI is becoming less about one impressive model call and more about systems around the model. The real leverage showed up in context, evals, verification, interface design, and operational loops.

Start Here

If you only skim this, these were the repeated themes:

• Computer use is becoming a practical bridge for the long tail of websites and software that will never expose clean APIs.
• Evals are shifting from static benchmarks toward production simulations, route-specific QA, trace analysis, and online tuning loops.
• Context is becoming infrastructure: something companies either rent repeatedly or own as a reusable knowledge graph.
• Agent systems need durable execution, scoped retrieval, strong logging, human review paths, and rollback.
• UX still matters. A capable agent that ships generic, ugly, or untrustworthy output will lose users.

Wednesday, July 1

Yutori Computer Use: Why APIs Will Not Cover the Long Tail

The Yutori session argued that computer use agents will navigate the web through vision and interaction, not through universal API coverage.

The web is too large and too slow-moving for clean API access to cover everything. There are roughly 200 million active websites, and the long tail includes local restaurants, school districts, government offices, vendors, and institutions that will not suddenly expose modern endpoints for every workflow.

The session framed web complexity in three tiers:

• Easy mode: the data is plain text and scrapable.
• Medium mode: the data is hidden behind a PDF.
• Hard mode: the data is rendered as pixelated JPEGs in a gallery, where OCR struggles.

HTML scraping also breaks down because modern sites often load placeholders first and fetch data later. A score on NBA.com or a stock state on an ecommerce page may never exist as plain text in initial HTML. The visible pixel state is the real source of truth.

Yutori's Navigator model follows that premise. The first version, launched last November, took screenshots as input and produced clicks or scrolls. Navigator M1.5 adds on-demand JavaScript execution, so the agent can write code to fill multiple fields at once while still using screenshots as the verification layer.

The important architecture point is that code is a tool inside the visual loop. The screenshot remains the judge of whether the page is actually in the right state.

Multi-agent mode was the other major unlock: an orchestrator can launch multiple Navigator instances across sandboxed browsers, letting hundreds of web tasks run in parallel.

Takeaway: computer use is not just a backup for missing APIs. For the messy, institution-heavy, long-tail web, it may become the practical interface layer.

Uber Eats: Eval Design for Visual Content Agents

The Uber Eats session was about evaluating an image enhancement pipeline at marketplace scale.

The business problem is large: Uber Eats has a massive catalog, millions of items added monthly, and many small merchants that do not have high-quality food photography. Better visuals can improve click-through and add-to-cart, but the system has to preserve authenticity, merchant brand, marketplace diversity, and customer trust.

The agent pipeline had three stages:

• Image understanding and routing: score the image and decide whether to keep it or enhance it.
• Image editing: generate an enhancement prompt, run the edit, then use LLM QA to approve or reject it.
• Post-processing and publish QA: final quality gate before the image reaches the menu.

The most practical advice was to start with logging. All agents write flat JSON traces that can be read by technical and non-technical teammates. The traces capture both publish and do-not-publish paths.

The evals matched the routing structure:

• Routing eval: classic precision and recall confusion matrix.
• Routing guardrail: recall, because bad images should not slip through.
• Enhancement eval: pass@K, measuring how quickly an image passes QA across retries.
• QA criteria: faithfulness, unchanged portions, natural plating, preserved colors and textures, sharpness, and clarity.

The failure examples made the risk concrete. A high-quality image routed for enhancement can be degraded. A six-wing photo described as eight wings can cause a model to remove real food. A model can add fake ingredients or remove real sauce.

The auto-tuning loop was production-oriented. Sample live traffic, have humans verify decisions, let a diagnoser tune routing thresholds, benchmark against current production, ship config without code changes, and rerun on fresh traffic. Prompt tuning used labeled samples, an eval engine, disagreement diagnosis, and prompt synthesis before registering a new agent config.

Takeaway: the eval design had to match the business risk. The point was not a single quality score. It was routing safety, retry efficiency, and publish readiness.

Snorkel AI: From Agent Traces to Agent Simulations

Snorkel's core point was that production traces are evidence, not experiments.

A trace shows what happened once, under one state of the world. If you replay a task later, user intent, database state, API state, tool versions, context, history, and wall-clock time may all drift.

Public benchmarks are useful, but they do not represent your company's domain, tools, policies, or users. The goal is to turn production traces into repeatable offline simulations.

The proposed flow was:

• Collect production traces through observability tools.
• Extract and cluster representative tasks.
• Package those tasks into an offline simulation benchmark.
• Run agent configurations A, B, and C against the same task, environment, seed, tools, and state.

That makes comparisons apples-to-apples. It also shifts the benchmark target from model-only to full configuration stack: model, thinking level, prompt, policy, harness, context, memory, skills, and tools.

The phrase that stuck: benchmark configurations, not models.

Each benchmark task is a packaged artifact:

• Agent-visible layer: instruction file and environment.
• Verifier-only layer: oracle solution and hidden checks.
• Metadata layer: limits, tags, and baseline results.

The oracle matters because it proves the task is solvable and the verifier is correct before agents run against it.

The verifier mix included deterministic checks for final outputs, tool calls, and constraints; LLM judges for trace quality and recovery behavior; and optional human review.

Takeaway: serious agent evals need controlled environments, not just interesting traces. The benchmark becomes a data engine for improving the agent.

Context as a Service: Renting vs Owning Context

This session framed Context as a Service as vertical search plus knowledge graphs.

The CaaS provider indexes, structures, deduplicates, enriches, and merges entity data from multiple sources. It is distinct from AI search in the same way AI search is distinct from Google: the value is in structured, reusable context, not one-off links.

The benchmark compared AI search and CaaS for company enrichment:

• 100 runs across event sponsors.
• 25 fields per company.
• Fields ranged from easy facts like domain and HQ to harder signals like hiring, layoffs, and personnel changes.
• The harness used Claude field-by-field retrieval across search providers, SERP, native search, and CaaS providers.

The counterintuitive result was that CaaS underperformed on coverage. The reason was recency. CaaS can be limited to what it has already collected, while search can go find new public information.

Cost had a separate shape. Native search was expensive because unstructured results burn tokens during structuring. Premium CaaS was also expensive. Budget CaaS was cheaper but lower quality.

The hidden cost driver is frequency. If your team asks about the same accounts daily, weekly, and monthly, every repeated query costs again. The real formula is records times frequency, not just record count.

The build-vs-rent tipping point from the session was around 15,000 entities or queries at an estimated $5,000 setup cost. Below that, renting is cheaper. Above that, owning context starts to compound.

Takeaway: use AI search for ad hoc, always-changing questions. Own or build context for persistent, high-frequency, well-defined workflows.

CUA: Computer Use 2.0

The CUA session covered Quadriver, an open-source project built around Codex and computer use models.

The goal is to connect agents to the operating system at a middle-ground level, not just as screen overlays.

The architecture uses two execution modes:

• Accessibility tree interface as the preferred path.
• Screenshot-level fallback when the structured interface fails.

The execution loop is:

• Take a snapshot of the current UI state.
• Retrieve an XD3 accessibility-tree representation plus execution context.
• Try the action through XD3 first.
• Fall back to screenshot-level execution if needed.

The team emphasized schema management because OS and app update cycles can break behavior. They tested early releases across eight application harnesses.

Takeaway: the strongest computer-use systems will likely combine structured OS interfaces with visual fallback, rather than choosing only one layer.

Hassan: Stop Letting Agents Ship Ugly UIs

Hassan's session was a practical design warning: AI-generated UI has recognizable patterns, even when people cannot name why it feels wrong.

The common slop patterns included:

• Purple gradients and glow effects.
• Italic words in headers.
• Three-card feature rows.
• Spaced-out all-caps pills.
• Invented stats and fake testimonials.
• Emoji icons and vague copy.
• Random graphics and long scroll-to-floor layouts.

The proposed fix was Hallmark, a codified design skill or prompt layer that bakes in more than 60 anti-slop gates. It refuses the common patterns and uses curated design themes as inspiration.

The model choice point was also pragmatic. Cheaper open models like GLM 5.2 can be good enough for iteration. More expensive models can be reserved for final polish.

The concrete workflow advice:

• Learn the bad patterns by name so you can explicitly block them.
• Save your preferences in a skill file or Markdown doc.
• Keep an inspiration vault of screenshots.
• Use longer, more specific prompts.
• Break features into separate prompts.
• Treat the first output as a base, not a finished product.

Takeaway: design quality is not a cosmetic layer. It affects trust, adoption, and whether users can take an AI-built product seriously.

Zoox: From Self-Driving Cars to Self-Driving Code

Amit Navindgi from Zoox described an AI-first SDLC that spans spec writing, coding, PRs, review, CI, deploys, observability, and on-call.

The tooling posture was intentionally agent-agnostic. Cursor, Claude Code, and GitHub Copilot serve different users. Zoox does not want to lock the organization into one coding agent.

The operating lessons were very concrete:

• Use Sonnet as the default coding model for most work.
• Disable fast mode when token burn needs control.
• Treat prompt caching as the most important cost lever.
• Keep sessions short and scoped to preserve context quality.
• Keep root AGENTS.md minimal, with team-specific rules in subdirectories.

Hooks are used to block restricted files and deploy commands, and to educate users during a session. But hooks run after every message, so they need to be used sparingly.

For MCPs, the rule was pragmatic: internal services get MCPs; external services can often be handled through CLIs.

The plugin marketplace was especially interesting. It lives outside the monorepo to make contribution and CI faster. About 100 skills were created within one month of launch. A review-plugin skill enforces a hard bar:

• The skill must justify why it needs an LLM.
• It cannot access secrets or tokens.
• It cannot run polling loops.
• It cannot trigger deploys.
• It needs a useful README and name-scope match.

Background agents are the next scaling step. A Slack message can trigger routing to a Cursor cloud agent, which then creates a PR. The PR is the handoff point, not the finish line.

Zoox is also experimenting with Bugbot as CodeOwner. Teams opt in small surfaces first, like pure utility functions. Human escape hatches remain. Human-curated rules beat auto-learned rules because institutional knowledge needs to be encoded explicitly.

Spend management is still evolving. The current model is a $1,500 per developer monthly cap, with over-limit forms reviewed centrally. The planned system gives managers and directors more direct visibility into productivity and spend.

Takeaway: enterprise coding-agent adoption is mostly an operating-system problem: rules, hooks, skills, review, spend, and ownership.

LLM Wiki: Raw Notes as a Personal Knowledge Engine

The LLM wiki session was about capturing messy raw material first, then letting agents structure it later.

The philosophy was simple: collect as much raw material as possible. Voice dictation, meeting transcripts, research notes, and podcast transcripts all count. Formatting can happen later.

The "Enrich Note" skill added structure:

• Add a timestamp so unenriched notes can be detected.
• Generate tags from a curated preferences folder.
• Search the web and attach source URLs.
• Find related notes through key-term search.
• Add backlinks.

The wiki generation step takes raw notes from a focus area and synthesizes them into an organized page. It extracts people, places, and concepts, then groups them into topics.

The overnight processing workflow used a cloud agent or local Codex automation:

• Pull down the Markdown folder through Obsidian sync or GitHub.
• Run enrichment across notes missing timestamps.
• Generate backlinks and update the wiki.
• Sync everything back.

The visualization layer was an HTML graph view generated from Markdown files. It showed clusters across AI engineering, startups, books, faith, and post-COVID reflections.

Takeaway: personal knowledge systems get much more useful when raw capture is cheap and enrichment is automated.

Thursday, July 2

Ramp: GTM Agents and Coordination Cost

Ramp's GTM session focused on reducing coordination cost. The goal is to describe intent once and let execution federate across teams and channels.

The three root problems were:

• Data inconsistency: every team has a different source of truth.
• Operational overhead: sales teams have no time for execution work.
• Distribution cost: coordinating action across teams and channels is too expensive.

Ramp built an internal CDP-like data foundation:

• Salesforce and interaction data.
• External signals like funding, new hires, news, and champions.
• Third-party enrichment and public web.
• Kafka for real-time ingestion.
• Snowflake and dbt for warehouse processing.
• Reverse ETL into Postgres and Turbopuffer for online retrieval and vector search.

The hard engineering problems were referential integrity, fuzzy account matching, and embedding unstructured data like calls, emails, and notes.

Every GTM agent thread runs as a durable Temporal workflow:

• Trigger from human chat, events, or schedules.
• Optionally hydrate accumulated state.
• Call the model.
• Execute tools.
• Repeat until no tool calls remain.
• Pause for human review when needed, then resume.

The first vertical was pre-meeting briefs for Account Managers. The brief includes attendees, tech stack, agenda, account vitals, recent comms, open tickets, product usage, and suggested implementation tasks. It runs nightly, spawning one Temporal agent per upcoming meeting.

The architecture extends naturally to AEs by swapping in opportunity data and third-party enrichment instead of product usage data.

The broader vision is GTM orchestration: describe a motion once, then spin up audiences, outbound sequences, landing pages, ad creative, and channel-specific work across SD, AE, AM, and web.

Takeaway: GTM agents need data infrastructure before they need clever prompting. Without a consistent entity layer, every agent inherits the same coordination mess.

Cross-Session Takeaways

1. Pixels, Context, and Traces Are Becoming Production Inputs

The Yutori and CUA talks treated pixels and accessibility trees as real interfaces. The Snorkel talk treated traces as raw material for simulation. The LLM wiki talk treated messy notes as raw material for a knowledge graph.

The shared pattern: production AI systems are only as good as the inputs they can repeatedly observe, structure, and verify.

2. Evals Need to Match the Workflow

Uber Eats did not use one generic image score. It used routing recall, pass@K, QA criteria, feedback traces, and online threshold tuning.

Snorkel made the same point for agents. If the real system depends on tools, memory, prompts, policies, and environments, the eval needs to benchmark the configuration, not just the model.

3. Owning Context Compounds

The CaaS talk and Ramp talk converged on the same idea from different sides. If the workflow is frequent and persistent, rented context gets expensive and stale. Owned context becomes reusable infrastructure.

4. Agent Adoption Is an Operating Model

Zoox made this clearest. The hard part is not giving every developer a coding assistant. It is building the rules, skills, hooks, review surfaces, spend controls, and team ownership patterns that let agent-generated work scale.

5. UX Is Part of Trust

Hassan's UI talk belonged with the eval and production talks. Users judge capability through interfaces, images, briefs, notes, and artifacts. Bad design makes the output feel less credible, even when the model is strong.

My Take

The conference made production AI feel less like a model race and more like systems engineering.

The best teams are not just asking which model is smartest. They are asking what context the model receives, how the environment is controlled, what the verifier checks, how failures are logged, where humans review, which configs can roll back, and how the workflow compounds over time.

That is the real shift for AI engineers. The job is becoming less about prompting one model and more about designing the loops around it.