ep 05 field notes
Show Us Your Agent Skills / EP 05 / guest dossier
JOHN BERRYMAN ARCTURUS LABS EARLY GITHUB COPILOT ROOK PI

JOHN BERRYMAN

Rook is John's personal agent, and it follows him around. Open Obsidian and it sees the people vault. Move to Wikipedia and it can find the BERT passage he cares about. Put it on his phone and a grocery store can become more than a dot on a map: inventory, aisles, substitutions, and John's own shopping list in one route. The early GitHub Copilot story is still in there too: John helped work on Copilot when there was no chat, just a tiny completion model and a lot of context-packing tricks.

▶ WATCH SEGMENT $ AGENTS-FOLLOW-YOU
2048early Copilot context had to be packed hard TERMINALthe harness is still too code-bound ROOKone client wrapped around multiple runtimes OBSIDIANthe vault lends the agent local powers BERTWikipedia becomes an agent-readable page KROGERplace context meets the shopping list
EP 05 · JOHN BERRYMAN · early Copilot, Rook, Obsidian, Wikipedia, and place-aware agents

"This is the first tool we've ever made that can talk back to us."

John came from the pre-chat Copilot world. Completion models needed tricks. Talking tools made the whole product surface feel open again. 00:07:51

FROM PSEUDO-DOCS TO PERSONAL AGENTS

the Copilot context-packing story behind the Rook demo

John worked on early GitHub Copilot at GitHub before ChatGPT made chat the default shape of AI products. The model did document completion. The context window was tiny. Useful behavior came from building pseudo-documents: comments, neighboring file facts, and just enough surrounding state for a completion model to infer the job.

Rook starts from the opposite frustration. The model can now talk, call tools, and run inside agent harnesses, but the harness still spends too much of its life in a terminal working on code. John wants the same agent to move with him through the places where his tasks actually live.

John Berryman showing Rook on the right side of his screen
Rook on the right of John's screen: a portable client over Pi and other runtimes, with the active app or page deciding which capabilities appear. [00:16:24]

"By and large, our agent harness lives in the terminal and works on code."

Rook is John trying to pull the agent out of the harness without losing the useful parts: tools, skills, local state, and permission. 00:13:08

AGENTS THAT FOLLOW YOU

the agents-that-follow-you workflow
Walk into the roomOpen Obsidian, a website, or the phone app. Rook notices where John is and changes what the agent can see. 00:18:39
Grant scoped permissionJohn allows the current place for the visit before the agent acts inside it. The permission moment is part of the interface. 00:19:41
Let local skills do local workObsidian has a CLI, and the local skill knows it. Rook can open Hugo's people-vault page instead of staying in generic chat. 00:20:30
Recover missing powers liveThe LinkedIn attempt finds a missing search capability. Pi appears to write and load what it needs mid-session. 00:22:44
Make web pages activeOn Wikipedia, Rook can act inside the page: find the original BERT model-size passage and highlight it. 00:24:58
Carry it into the worldOn the phone, a grocery store can add location, inventory, aisle layout, substitutions, and a route through the list. 00:28:02
Rook following John into Obsidian and exposing local vault affordances
Rook follows John into Obsidian and picks up vault-specific skills. The people vault gives the agent a local job instead of another blank prompt box. [00:18:47]
John holding up Rook on his phone while explaining physical environment context
Rook on the phone: the same personal agent can meet a place, a shopping list, and local store data without becoming a new agent every time. [00:28:02]

"Whenever it follows me to a new environment, it gets the superpowers of that environment."

Obsidian gives vault powers. Wikipedia gives page powers. A grocery store could give aisle powers. Rook stays John's agent while the place lends the tools. 00:18:47

WIKIPEDIA BECOMES AN ENVIRONMENT

the wikipedia-discovery skill

John asks Rook to open the BERT article, then asks for the part about the original model size. Wikipedia lookup is ordinary. The live move is the page-local action: find the right passage and put John's attention back on the evidence.

He calls the web affordance "open agent protocol," then immediately laughs it off as a name for something that should exist. In the version he wants, a website sends state to the user's agent, receives requests back, and lets the user reshape what is visible.

John using Rook to highlight the BERT model-size passage on Wikipedia
The BERT page gives the agent a page-local job: find the model-size passage and highlight it for John. [00:24:58]

"I was actually lying about the open agent protocol. It's just a thing that absolutely should exist."

The joke matters because the missing standard is visible in the demo. Web pages want a way to become places an agent can visit with the user. 00:25:36

ROOK'S FIELD KIT

the workflow, the reconstructed skill, and the places John wants agents to enter
workflow

agents-that-follow-you

A portable agent harness that follows the human across apps, websites, and places, picking up local affordances from each stop.

skill

wikipedia-discovery

A reconstructed page-local skill from the demo: open the requested article, find a passage, and make the evidence visible in context.

skill-shaped

obsidian people-vault skills

Local Obsidian affordances for John's people records, using the vault and CLI so the agent can navigate and update the place he already works.

concept

kroger place skill

A grocery-store concept: merge John's list with store inventory, aisle shape, item locations, and substitutions.

concept

zoom participant AI

The moment that changed John's view of skills: use English to author behavior, then add enough app chrome to seat it inside Zoom.

practice

task to skill

John turns small tasks into skills, then pays for it when the five-minute task becomes two days of skill maintenance.

"My life has become kind of like an agentic nightmare."

The joke carries the maintenance cost: every five-minute task can become a reusable skill, and every reusable skill can become a two-day repair project. 00:14:20

JOHN'S STACK

what Rook wraps, visits, and wants the world to expose
Pithe runtime John hacks with most Obsidianlocal people vault and CLI affordances Wikipediapage-local discovery demo Krogerphysical place-context example MCPbridge for systems he does not control Copilotthe early context-packing origin story