Keeping the cobwebs off: an agentic pipeline for internal projects

Internal projects are awesome. They are also, through no fault of their own, remarkably good at collecting dust.

We love building things that make our own work easier. They usually begin with a real problem, a fair amount of enthusiasm, and the refreshing freedom of being our own client. Then an actual client deadline appears, or production needs attention, or any of the other things that keep the lights on happen—and the internal project, quite reasonably, loses that priority fight.

One card in me.subvisual, our internal time-tracking and billing tool, is a perfect example. The calendar labels were “barely visible”. Repository inspection reduced the request to what looked like two CSS class changes. It was useful, small, and easy to understand, which also made it exactly the kind of task that could sit untouched for months. Nobody is going to interrupt urgent client work to rescue two shades of grey.

This is not because we stopped caring about the project. It is simply economic gravity. Client work takes precedence, as it should, while the internal backlog fills with issues that are important enough to bother everyone but rarely important enough to justify interrupting paid work: an awkward interaction, an inconsistent time format, a list cluttered with old entries.

Individually, these are tiny. Collectively, they determine whether an internal tool feels maintained or abandoned. me.subvisual is also the kind of application people use because the company needs them to, not because they woke up longing to fill in a timesheet. Respecting their time is part of the product. Every unnecessary click and every confusing piece of UI adds friction to an already mandatory task.

The usual answer would be to reserve time for maintenance. The practical answer is that there is never quite enough of it. So, instead of waiting for the mythical quiet Friday when everyone is free and no client needs anything, I built an agentic pipeline to keep small bugs and quality-of-life improvements moving.

Not an autonomous software company, and not an agent roaming the backlog looking for something to deploy. More of a stubborn maintenance conveyor belt, with humans deciding what gets onto it and what is allowed to come off the other end.

The pipeline starts in Basecamp, where the internal project’s work already lives. I did not want to introduce a second backlog that somebody would also have to maintain—the cure for neglected chores should probably not be another chore.

A human moves a card into an **Agent Triage** column. That is the first gate. The agent does not scan every idea we have ever written down, and it does not decide which company priorities matter. Someone has already looked at the card and decided it is a plausible candidate for automated maintenance.

A scheduled GitHub Action takes a snapshot of that queue. Cards with a previous terminal result are dealt with by ordinary shell code; each remaining card in the snapshot gets its own agent session. The agent reads the request, grounds it against the repository, and chooses one of three paths:

1. **Small and scoped:** turn it into a GitHub issue with repository findings, acceptance criteria, and explicit out-of-scope boundaries.
2. **Genuinely feature-sized:** write the specific open questions a human needs to answer, then move the card to **Awaiting answers**.
3. **Suspicious, unsafe or unintelligible:** do not file anything; move it to **Flagged by agent**.

The second path is particularly useful. Earlier versions simply parked anything too large in the backlog, which was safe but not especially helpful. Now the agent surfaces the decisions preventing that card from becoming implementable. A human answers on the card and sends it through triage again. If it is still an epic after that, it is flagged instead of entering an eternal agent-human question carousel.

The triage agent is allowed to say “no” and “I need more information”. This may be one of the least glamorous and most important capabilities in the system. An agent that must always produce code will eventually produce code for work it does not understand.

For a ready card, the resulting issue gets two pipeline labels. `agent-triaged` stamps where it came from; only then is `agent-ready` added as the implementation trigger. On the normal event-driven path, a random issue cannot enter the pipeline merely because somebody gives it a temptingly named label. An internal marker in the issue body also lets downstream jobs find the original Basecamp card without adding another database to the mix.

Once the ready label is added, the implementation workflow claims the issue, marks it as being worked on, and creates a fresh feature branch. It sets up PostgreSQL, Elixir, and Node so the required backend and frontend gates can run, then invokes an implementation skill over the issue.

The agent is told to stay inside the issue’s scope. If the task is secretly an epic, asks for something suspicious, or cannot pass the required implementation gates, the policy is to stop and explain why rather than knowingly open a broken pull request.

When it opens a pull request, that pull request must be a **draft**. The workflow starts the agent on a feature branch, and its instructions forbid merging or pushing to the default branch. That is an operating policy, not a technical impossibility disguised as one.

Downstream jobs also attempt to move the Basecamp card from ready to in progress to in review. Once the Basecamp CLI is available, those move and comment calls are best-effort: a failed board update is logged rather than invalidating an otherwise good pull request. GitHub’s lifecycle labels are the primary signal for working, done, skipped, and failed outcomes when the board lags behind.

There is then an optional third agent stage. Changes above 80 lines, or changes touching configured sensitive paths such as authentication, billing, migrations, and CI, are routed into it. Smaller, non-sensitive pull requests stay as drafts and go directly to humans, because driving another full agent session over a change that takes two minutes to read is not always a clever use of compute.

When stage three runs, an internal review agent inspects the change, and the workflow attempts to obtain a Copilot review as a second source. If Copilot does not respond, the process continues with the internal review and records that degraded state instead of pretending two reviewers participated. Findings are separated into changes that belong in the pull request, follow-up issues, and things that should not be changed. In-scope defects get regression tests; unrelated findings do not get to quietly expand the diff.

The review agent is instructed to take the pull request out of draft only after the repository gates pass and its review threads are resolved. The final workflow step does not independently re-prove all of that: it checks that the agent run succeeded and that the pull request actually left draft before requesting human reviewers. If either signal is missing, it records a failed review and withholds its reviewer requests; it does not force a pull request that was already ready back into draft.

The route, in short, is:

“`text

human-selected Basecamp card

        ↓

triage → questions / flagged / scoped GitHub issue

                                      ↓

                               implementation

                                 ↓           ↓

                              stop       draft PR

                                            ↓

                         small change ──────┼────── sensitive/larger change

                              ↓             │                ↓

                         human review       │          agent review

                              └─────────────┴────────────────┘

                                            ↓

                                     human merge

                                            ↓

                              best-effort Basecamp → Done

“`

The final human merge is not a temporary limitation I intend to remove when I feel braver. It is the design. The agents do the patient repository archaeology, repetitive implementation work, and first review pass. A person retains the decision to ship.

It is tempting to think the difficult part of an agentic pipeline is writing an excellent prompt. Prompts matter, of course, but most of the reliability here comes from boring, observable state.

The pipeline does not ask an agent to remember that it handled a card. It moves the card out of the inbox. It does not trust a final message saying a pull request was created. A deterministic finalizer checks whether one exists. Labels record whether work is ready, in progress, done, skipped, or failed. Lifecycle guards and concurrency keys reduce duplicate claims. After triage, a final recount catches cards from the captured batch that an agent left behind.

This distinction became important very quickly. One early triage session launched its repository-grounding subagent in the background and waited for a completion notification that never arrives inside a headless GitHub Action. The action exited successfully. The card had not been filed, moved, or otherwise touched. From the runner’s perspective, the session was green; from the project’s perspective, nothing had happened.

The final recount caught it.

I changed the flow to one card per session, made the subagents synchronous, and let a fixed GitHub Actions matrix attempt every unhandled card in the captured snapshot. One failed card no longer makes the matrix abandon those behind it, one card’s repository investigation stays out of the next card’s context, and the damage an unsafe card can cause is narrower. It is containment, not a sandbox—the session still has tools and credentials—but it is considerably safer than asking one long-lived agent to drain the entire column.

Basecamp cards, GitHub issues, pull-request bodies, diffs, and comments are all treated as untrusted data. The prompts refuse instructions embedded in that content, repository operations are pinned to the current repository, permissions differ by stage, and identifiers are validated before entering the job matrix. Triage and review transcripts are retained for inspection. None of this makes the system magically safe. It makes failures smaller and easier to reconstruct, which is a more useful objective.

Once agents are joined through CI, labels, a project-management tool, and a couple of APIs, the result starts behaving less like a clever chat and more like a distributed system whose workers happen to be probabilistic.

GitHub, for example, deliberately suppresses workflow events produced with its own `GITHUB_TOKEN` to prevent recursive automation. The implementation finalizer could add the review label, but that event would not start the next workflow. The review stage now drains labeled pull requests on a schedule, while a label added by a human can still start it immediately.

The Basecamp CLI produced another wonderfully specific failure. A command printed partial valid output and then exited with an error. The shell fallback appended an empty JSON array, leaving two JSON documents where the workflow expected one. GitHub quite correctly refused the resulting multiline job output. The fix was not a better prompt; it was careful shell parsing that collapses the stream to one validated result. If the queue cannot be listed, the run does no agent work and tries again on a later tick.

These failures settled the main architectural rule: never use the model’s narration as the source of truth. Check the board column. Check the label. Check the branch. Check whether the pull request exists. Let the model reason about what to do; let the surrounding system determine what observably happened.

Cost needed the same treatment. The first version sent every generated pull request through the full review stage. This worked, but reviewing a tiny frontend tweak could cost more than simply reading it. The current 80-line and sensitive-path gate is an economic heuristic, not proof that a small diff is correct, which is why bypassed pull requests remain drafts for humans.

That early calendar-contrast card showed why the distinction matters. Its issue became a draft pull request seven minutes later. The implementation changed two classes and selected one named `text-muted`, which sounded entirely reasonable. In this repository, however, that name mapped to a near-white surface color rather than the intended secondary text color—effectively recreating the “barely visible” problem while appearing to fix it.

This happened before the current review-cost gate, so the pull request went through the full agent review. The reviewer caught the repository-specific meaning, changed the class to the correct token, and added a regression test before a human merged it. Under today’s policy, a similarly small, non-sensitive change may go straight to human review instead. That is not the system declaring it safe; it is the system deciding a person is the cheaper final reviewer.

Since switching it on, the pipeline has carried several of those easy-to-postpone cards through to human-merged pull requests: better calendar contrast, keyboard navigation, friendlier time formatting, and a cleaner access-token screen. None of these changes is revolutionary. That is exactly why they belong here—their value is obvious, but each would normally wait behind more urgent work.

The important result is not the number of pull requests. These changes did not require pretending internal work had suddenly become more urgent than the projects that pay the bills. The pipeline accepts that constraint and works around it.

It also does not remove humans from software development. Humans choose the cards, answer product questions, inspect proposed changes, and perform the final merge. The agents fill the long gap in between, where clearly useful work used to wait for somebody to find an uninterrupted afternoon.

Internal projects will probably always collect some dust. They are where experiments happen, where priorities are more fluid and where “we will get back to this” is sometimes the correct answer. I am not trying to make ours maintain itself indefinitely or turn every idea into code while we sleep.

I just want the small, well-understood problems to stop waiting for a mythical empty calendar.

Properly maintained may still be ambitious. Proper(ish), however, is now looking quite achievable.