Why I Built Faye: Human-in-the-Loop in the Age of Fully Autonomous AI Agents

This is an introduction of a Slack-first, local-only bridge for Claude Code, Codex and Visual Studio Code inside an enterprise that trusts humans more than harnesses.

The world wants you to let go of the wheel

It’s May 2026, and we are living in the golden age of autonomous coding agents. The competition between Claude Code and Codex has been brutal in the most interesting way: every time one ships a feature, the other answers within weeks. Basic tools like web search, slash commands, and tab completion already seem obsolete. Granular configurations for bash command permissions, agent skills, subagents, external plugins and project-wide RAG search, all those capabilities that felt like a distant dream just a few months ago, have already become standard. After Codex shipped the /goal command and received overwhelming praise, Anthropic quickly fired back with its own /goal feature and a “Proactive Output Style.” Both platforms now offer managed cloud services where you don’t even need to boot up your PC — your AI agents simply spin up and start working on your behalf.

The message from every vendor is identical: stop babysitting the agent. Set a goal. Walk away. Come back to a pull request. Developers and vibe coders start to run two or three accounts with dozens of subagents operating in parallel in an agent team. They set up brutal “harness” scripts or skills that effectively crack a whip at the model whenever it tries to pause, chasing a workflow where the agent never stops until the objective is reached. The dream pitched at every tech conference is identical: while you sleep, travel, or simply refuse to touch a mouse, Claude Code and Codex will vibe-code twenty-six hours non-stop until your exact idea turns into reality. Both companies have also leaned hard into this with remote-control capabilities and cloud-hosted agent sessions so you can program from your phone, from a beach, or from an international flight — basically anywhere.

Meanwhile, I spent the last two weeks building the exact opposite: a system that makes the agent stop and ask me things, routes those questions to my company’s Slack workspace as interactive cards, and waits — patiently — for a human to tap “Approve” on a phone screen before a single line of code is written. I even forked an already-sunset open-source coding agent, Roo Code, just so a new hire could safely and comfortably use an AI coding assistant on day one.

What on earth am I doing?

The Illusion of “Free” Productivity

Late last year, our company experimentally provided unlimited Claude Code access to all employees. We quickly learned a painful lesson: unmanaged AI access leads to massive token inflation.

Under the vibe coding hype — and against the backdrop of a weak yen — some people burned through more in API costs than their monthly salaries. They learned prompt engineering in a single morning and started producing mountains of “just-in-case” prompts. They learned agent skills, felt the magic, and built dozens of elaborate agents and workflows they never touched again. They spent weeks tweaking AGENTS.md and CLAUDE.md, desperately searching for a magic prompt and the perfect workflow. By the time they started the actual task, the preparatory overhead alone had cost ¥100,000–200,000 — and it was all “free service” from the company, so nobody noticed.

Meanwhile, our company had a global cap, which created a first-come-first-served race. The people who just needed Claude to help with a real, boring task — refactoring a legacy module, writing documentation, debugging a production issue — found their quotas already consumed by Monday morning. Reasonable users lost out to heavy and active experimenters. Hundreds of thousands of yen vanished into workflow scaffolding that never touched production code.

I had raised this in an internal channel several times, asking for some kind of governance so the responsible users would stop subsidizing the irresponsible ones. This month, the president finally stepped in: all AI usage now counts against each project’s budget, not a shared corporate pool. The free lunch is over.

The Reality of Enterprise Constraints

Under the new policy, bypassing the system simply isn’t viable. If I route requests through the public OpenAI API or Gemini AI Studio on my own account, I end up footing the bill myself, and proprietary company code could get exposed to my personal environment. If I try to save money by using a cheaper DeepSeek model via a personal API key, I violate the company’s code-security guidelines.

The company’s security guidelines from last year explicitly recommend using models deployed through Microsoft Foundry, Amazon Bedrock, or Vertex AI — the three blessed enterprise channels — to ensure source code never reaches an AI vendor’s training pipeline.

Furthermore, company policy forbids storing source code on personal phones. You cannot use a personal Claude subscription or a ChatGPT account to touch company repositories. You cannot install third-party Android SSH clients to tunnel into the corporate network. You certainly cannot spin up a Linux VM hosting company code on your personal device — the configuration is a headache, and if a leak occurs, the liability falls entirely on your shoulders.

But fortunately, company policy allows you to read and respond to Slack messages from anywhere, on any device, at any time. Slack is already MFA-protected and company-managed. It is the single remote surface that is both explicitly permitted and already seamlessly woven into everyone’s daily rhythm.

Ultimately, enterprise projects are not side projects. In a personal repo, you can afford to ship fast and fix fast. In a client-facing system, the cost of a production bug exceeds the cost of slower development by orders of magnitude. The margin of error for assuming “the agent probably got it right and I can just trust it” is practically non-existent here.

Onboarding in the Age of Vibe Coding

Around the same time, I was asked to onboard a new junior team member.

When a newcomer arrives, establishing healthy AI norms is critical. Especially for someone without AI development experience, requesting unlimited Claude Code quota is hard to justify, and Claude Code’s workspace-level quota means a new hire could inadvertently eat into the project’s token budget. Worse, they might fall into the trap of vibe coding before understanding the project context, and start generating massive blocks of “mostly working” code that become a nightmare to review and maintain.

So I did something deliberate: I chose not to give him Claude Code or Codex at the start. Instead, I sent him a short survey — six questions about his AI habits, his spending, his trust level, and how he actually wants to use these tools going forward. I was trying to understand his relationship with AI before handing him anything.

His response was remarkably mature. Crucially, he expressed a healthy skepticism: “I’m afraid of the situation where I can no longer understand what is supposed to be my own code.” He seems to view AI not as an autonomous architect, but as a sparring partner to deepen their own understanding.

This developer wants to drive the car. He wants AI as a copilot who explains the road, not one who takes the wheel. Giving him Codex or Claude Code — tools designed to run autonomously for hours — would directly conflict with how he wants to learn. And I needed to provide him with the exact tools that align with his learning style.

Faye — a Roo Code fork for constrained environments

Roo Code was sunsetted on May 16, 2026. The original team archived the VS Code extension, shut down Roo Code Cloud and Router, and moved on to Roomote. But Roo Code’s local-agent core — the ability to work inside VS Code, read and edit a repository, use tools, ask questions, and help with real development — was exactly what we needed.

So I decided to fork Roo Code, stripping away everything that doesn’t belong in a constrained company workflow. I removed all cloud telemetry, marketplace integrations, and upselling features to create a hyper-focused, local-only agent named Faye. There are no cloud logins and no webviews—just pure local execution. I also streamlined the prompts, removing verbose rules and markdown formatting boilerplate, as 2026-era models perform better with less prescriptive scaffolding.

But the most philosophically important change I made was removing the “forced tool use” constraint.

The original upstream system prompt and harness script required the agent to call a tool in every single response. The problem with these pure harness-driven loops is that when they go wrong, they go wrong expensively and silently. An agent that’s been whipped through 30 turns desperately trying to call a tool may produce code that technically passes the condition while being architecturally incoherent. The reviewer opens the pull request, sees 2,000 lines of generated code with no narrative, and has to reverse-engineer what the agent was trying to do. It’s a nightmare. Furthermore, this constraint prevents newcomers from using Faye normally as a sparring partner. When the AI is relentlessly whipped toward an objective, it has no room to breathe, pause, or patiently teach the developer.

What I actually want from an AI colleague is closer to what you’d want from a junior engineer: someone who does the work, but reports back proactively when something feels off. Not someone who plows ahead in silence because the harness told them not to stop.

To fix this, I introduced a new TaskTextOnlyResponse event. Faye can now pause and talk naturally — ask a clarifying question, explain a concern, or just say “I’m confused” — without being forced into weird actions. It means the agent is allowed to think out loud with you instead of being whipped toward a goal. You are allowed to pause. You are allowed to ask a question before acting.

That solved the behavioral problem, but a well-mannered agent running on the wrong model is still the wrong tool. The next question was equally deliberate: which models should I actually put behind Faye for a newcomer who wants to learn?

The first decision was what not to provide. I intentionally left out the state-of-the-art GPT-5.5. It is the most expensive model available, and recent reports document higher hallucination rates and output instability compared to its predecessor. Coupled with widespread user complaints about quality degradation in the weeks following launch, the model appears to have been quietly “lobotomized” — a depressing but familiar pattern of post-release cost management across the industry this year. For a newcomer already skeptical of AI tools, GPT-5.5’s expensive, “confident-but-wrong” failure mode is the absolute worst possible first impression.

What I wanted him to reach for instead was GPT-5.1. It is OpenAI’s mid-tier reasoning model: It does not try to impress you with novel abstractions; it reads what is there, reasons about it carefully, and gives you a straight and clean answer. For the tasks that actually matter in his first months — refactoring a legacy module, drafting a pull request description, tracing an easy bug — GPT-5.1 is the kind of colleague who says “I think this is right, and here’s why” instead of rewriting your entire file with silent confidence.

Around GPT-5.1, I placed a small supporting cast. GPT-4.1-mini for throwaway tasks — translating a variable name, explaining an error message. GPT-4.1, distilled from GPT-4.5, as a patient conversational partner for when he wants to understand something rather than produce something. GPT-5.4 for the rare moments when 5.1 genuinely cannot see deep enough — a complex architectural review or a subtle issue spanning multiple modules. And Kimi K2.6, one-sixth the price of Claude and strong on frontend work, for the admin-panel pages improvement.

How to deploy these models? The enterprise security constraints I described earlier already ruled out personal API keys from OpenAI or Anthropic. Since I hold the Owner role for our Azure subscription, I could deploy exactly the models I wanted through Microsoft Foundry, with usage rolling into the existing project billing and per-user consumption visible in the Azure portal. I set an explicit ¥3,000/month budget for him — tight enough to discourage waste, generous enough to never block real work.

A few days later, I checked the Azure dashboard. He had settled on GPT-5.1 as his primary model — exactly as I had hoped. Not because I told him to, but because he tried the others and arrived at the same conclusion on his own. The dashboard also confirmed something quieter: his daily spend was well under ¥300.

The Slack Bridge: Solving the Mobile Constraint

We had our local agent now, which perfectly solved the safe onboarding problem for junior developers. But for my own workflow as an experienced developer managing larger tasks, I faced a different challenge: how do we solve the physical constraint towards automation and mobility?

When handing off a complex, long-running task to an agent, you naturally want to step away from the screen. My ideal workflow is to have a focused initial brainstorming session with the AI, agree on a plan, and then go downstairs to feel the breeze and soak up some sun while the agent executes it. But if the agent unexpectedly pauses for a permission check while I am away, development halts. This creates a lingering anxiety that forces me to stay anchored to my desk.

As established earlier, the only trusted remote surface allowed on our phones is the company Slack. So instead of bypassing security and trying to use other AI providers or Tailscale, I intentionally designed a Slack Approval Bridge workflow for Faye:

1. I manually start a sandboxed agent session on my company PC via pnpm remote:codex, pnpm remote:claude or pnpm remote:vscode.
2. The Slack bridge connects locally via IPC socket. No inbound ports are exposed.
3. When any of the AI agents needs approval, Faye pushes an interactive message card to a dedicated Slack channel.
4. Slack enforces MFA. The bridge verifies the Slack interactivity and my Slack username.
5. Using /faye codex send <prompt> and /faye claude status <number>, or more directly, through the interactive message card I designed, I can view terminal outputs, answer Faye’s questions, or click “Approve”/“Deny” directly from my phone.

No source code leaves the machine. No credentials are put at risk. The security is airtight, yet what makes the bridge satisfying to use is the interaction design:

• Bidirectional push. Any agent pause — permission request, clarifying question, task completion — posts immediately to Slack as a styled message card.
• Color-coded cards. Amber sidebar for permission requests. Cyan for follow-up questions. Moegi for completions. A random fairy-palette color when Faye is just listening. When an ask is resolved, the card folds with a status line and a matching color. This turns the black-and-white CLI approval flow into something visually scannable.
• Interview modals. When Claude Code asks a multi-choice question, Faye translates it into a Slack dialog with radio buttons (single-select) or checkboxes (multi-select). You tap your choices; the bridge converts them to the correct terminal keystrokes. No typing on a phone keyboard.
• Approve/Deny with one touch. Diff previews are rendered in the card at a readable font size. You see what’s changing. You don’t blind-approve.

• /faye status captures the last n lines of the terminal, even if a hook didn’t fire. You can always see what’s happening.
• /faye send injects keystrokes directly (Enter, Escape, Tab, arrow keys). If the agent is stuck in an unexpected state, you can recover without walking back to your desk.

• Auto-close stale cards. When a new event arrives, all previous open asks fold automatically with a status line — “Faye was lured away by another whisperer.”
• Clean text copy. Terminal output is reformatted before posting to Slack. No broken line wraps, no ANSI escape codes. Text you copy from Slack is text you can paste anywhere.

I know I don’t need an agent that codes while I sleep. I need a workflow that lets me step outside without losing control. That is still automation. It’s just automation with a checkpoint — a human who reviews the direction before the agent burns another 1,000,000 tokens going the wrong way.

Breaking the Physical Limit: New workstyle unlocked

When I finished building this and finally got it running, I realized it had completely changed how I experience my workday.

I no longer have to sit anxiously watching a terminal output, babysitting an AI to ensure it doesn’t run off a cliff. Instead, I can pack up my laptop, step out of the office, and take a walk around Minato Mirai. With the beautiful views of the bay stretching out before me, I can actually stop and take photos of the surreal, towering summer cumulus clouds. While I’m strolling near the water, feeling the breeze, my phone vibrates. It’s a Slack message from Faye. She has finished reviewing a massive Pull Request and found a subtle edge case.

Standing there by the bay, I read the summarized diff on my screen. I use voice-to-text to reply in the Slack thread, asking her to verify the database transaction rollback logic and go through the authentication flow via the Playwright agent skill, and then slip my phone back into my pocket. By the time I return to the office, the updated report is waiting for me, perfectly aligned with the project’s strict constraints. I do a quick sanity check and hit merge.

Another evening, I was at the local supermarket doing my grocery shopping. I was holding a shopping basket, standing in the produce aisle trying to decide what to cook for dinner, when my phone buzzed. It was Faye again. She had reached a critical checkpoint while refactoring a legacy module and needed permission to execute a batch of search bash commands and kick off the test suite. An amber-colored card popped up in Slack showing the exact bash command she wanted to run.

With my free hand, I simply tapped “Approve” right there in the aisle, and went back to picking out vegetables and bentos. By the time I got home, unpacked the groceries, and washed my hands, the tests had passed, and the commit was pushed and waiting for my final review.

No tunnel. No SSH app. No remote desktop. No copying code to my phone. No sitting at my desk waiting for an agent to ask a question it could have asked 20 minutes ago. I can walk to Minato Mirai and take photos along the waterfront. I can sit in a meeting in a different building. I can get a haircut. The agent works. When it needs me, it finds me. When it doesn’t, I know she is doing her job.

About Faye

Faye is named from the old word “fae,” meaning fairy — a gentle spirit who slips into your editor like moonlight through a window. She reads the shape of your thoughts, tidies what is tangled, and whispers the answer just before you think to ask. You may forget she is there — and that is how you know she has done her job well.

Faye isn’t an argument against automation. It’s an argument that the shape of automation should fit the shape of the constraints, not the other way around. We didn’t need to chase the chaotic, fully autonomous “vibe coding” trend to find real value in AI. By accepting our enterprise constraints and intentionally keeping the human firmly in the loop, we didn’t just protect our codebase and our token budget. We completely removed the physical anchor of the desk, creating a workflow that fits seamlessly and securely into real life.

For a newcomer who wants to learn, that means an agent that explains each step and waits for understanding. For a senior developer who wants freedom of movement, that means a Slack bridge that brings the agent’s questions to wherever they are. For a project budget, that means Azure billing integration instead of shadow API keys. For a security audit, that means code that never leaves the local machine and a model endpoint behind Microsoft Foundry. These are all the same principle: the human stays in the loop not because the agent can’t work alone, but because the organization works better when someone is accountable for each decision.

Now, when Faye posts an amber card asking whether to run a migration script, my first reaction isn’t “I need to get back to my desk.” It’s a glance at my phone, a tap, and back to whatever I was doing — a walk along the harbor, a conversation with a colleague, a cup of coffee in the sun.

The agent is working. I’m living. And the code has never left my machine.