Context Matters: Turning AI-Assisted Changes into Trustworthy Product Behavior

I was looking for dinner on the way home when a practical limitation turned into an engineering task. ChatGPT could suggest restaurants, but it could not reliably retrieve their KitchenGrade grades, even when I supplied direct links. KitchenGrade was my own app. It already had the inspection data, independently calculated grades and a public API.

From my phone, I kicked off a read-only Model Context Protocol (MCP) connection. A coding agent implemented the connection and automated checks. I defined the requirements, reviewed the validation and approved the release. Later, ChatGPT retrieved a restaurant's grade directly.

The phone made the work convenient to initiate. The engineering decisions made the result usable: which data to expose, how to identify the right restaurant, what a grade means, and how to verify that the new connection preserved the existing behavior.

Across my independent products, AI agents help implement changes and run checks. I supply problem definition, product direction, architecture, UX refinement, hands-on validation and release judgment. The useful question for me is what evidence connects an implemented change to the behavior a person needs.

Bound the contract before the code

KitchenGrade provides a concrete example. Its published project account describes the dinner decision and the integration that followed. The app calculates its own letter grades from public restaurant inspection records. Those grades have a dated basis and source links. Keeping that context attached matters when another application consumes the result.

The integration exposes two operations: search for restaurants and retrieve a grade for an exact establishment. Search helps resolve identity; the exact identifier anchors the subsequent lookup. A restaurant name alone is an inadequate contract for a question about a particular location's inspection history.

I kept the scope around existing public data and grading logic. The connection does not need access to private favorites, subscriptions or administrator functions. It also does not need to invent another grading method. Those boundaries make the intended behavior understandable before considering the implementation.

The tool results carry the grade's context, including dates and source information. An unavailable or outdated current grade needs to remain qualified; a historical letter should not silently become a current recommendation. This is part of the data contract between the application and the AI client.

The implementation includes bounded inputs and read-only operations. Validation covered the transport and compared results against the existing public grade API through actual application HTTP tests. I then confirmed the connection in the original ChatGPT workflow. Each check answered a different question: does the endpoint handle requests correctly, does it preserve the app's grade results, and can the intended client use it?

Test the interrupted interaction

That distinction became especially clear in Scantrast, an app that scans food products and interprets their nutrition against a person's daily targets. During a scanner update, confirmation required consecutive matching barcode reads. The candidate was cleared on an unreadable frame. It was an understandable attempt to avoid accepting a bad read.

On my phone, I could see the camera image, but capture would not complete. The important sequence was a valid read, an unreadable frame, then another valid read of the same barcode. Requiring consecutive matches treated an ordinary interruption as a reason to start over.

The correction accepted two matching sightings within a two-second window, allowing intervening misses. A different valid code or an expired candidate still reset confirmation. The regression checks included an unreadable frame between successful reads and were observed failing before the correction. I confirmed capture worked in the preview.

This changed the specification as well as the code. The original tests could support the consecutive-read rule while the rule itself failed the intended interaction. Reproducing the observed sequence gave the automated checks a more useful job: preserve behavior that had been established through device testing.

The Scantrast project overview explains the underlying nutrition problem. Its scanner is an example of why implementation evidence needs to include the environment in which a product operates. Camera optics, focus and intermittent decoding add conditions that a clean stream of synthetic successful reads can omit. Campare, my before-and-after photo comparison tool, tackles a related framing problem: it uses an earlier shot as a guide to help recreate the position and angle, so changes are easier to compare.

Match the evidence to the claim

WaterGrade exposed a different kind of mismatch. An early tester found the review form too complicated. We made the rating the prominent review action and moved supporting details into a collapsed section. She tried the revised form and found it much better.

The WaterGrade account records that feedback. The decision was about hierarchy: make the main contribution clear, then provide access to additional context. It gave the implementation a concrete UX requirement rather than a general instruction to simplify the page.

The follow-up is useful qualitative evidence from one tester. It supports saying that her feedback informed a simpler flow and that she preferred the revision. Completion-rate and retention claims would require different evidence. Separating those claims helps me describe the improvement accurately and decide what remains to learn.

Make room for human review

Mock Band adds the constraints of an interactive music product. Its case study separates agent implementation and automated checks from my product direction, physical-device acceptance testing and merge decisions. Hearing the music and operating real hardware were part of deciding whether the interaction worked.

Returning to the project with Codex also increased the amount of work I could coordinate in the background. That created pressure on review: some fixes caused regressions, and the queue became difficult to manage. I required QA steps in pull-request descriptions and asked for related changes and manual checks to be combined into batches.

A manageable review unit became a workflow requirement. An agent can prepare another change while I am assessing the previous one. I still need enough context to understand their interaction, perform the relevant acceptance checks and make a release decision. The review capacity around implementation belongs in the architecture of the development process.

Choose when automation runs

The same attention to context shaped a professional code-review rollout. In the public case study, I describe initially enabling GitHub Copilot review across more than 30 repositories. Repeated automatic reviews during development were consuming developers' usage allowances.

We refined the defaults in a small number of repositories. Some stopped automatic reviews on draft pull requests. Others made reviews opt-in so a reviewer could request a final sweep when the pull request was ready. My contribution included enabling the broad coverage and helping adjust those defaults around the way the repositories were used.

The relevant design decision was when the capability should run. A review trigger affects attention and usage allowances as well as coverage. Those tradeoffs became clearer through actual use, so the defaults evolved. That is an engineering outcome I can describe without inventing a percentage improvement.

A process for reviewable changes

Together, these examples suggest a practical process for making AI-assisted changes reviewable:

1. Define the decision or activity the change should support. For KitchenGrade, it was obtaining a verified grade for the restaurant under consideration. For WaterGrade, it was making the main review action understandable.
2. State the boundaries before implementation. Identify the allowed inputs, data, actions and existing behavior to preserve. KitchenGrade's public-data scope and reuse of its grading logic made that boundary concrete.
3. Turn observed failure into acceptance criteria. Capture the sequence that failed, including interruptions and recovery. The scanner's readable-unreadable-readable sequence was more informative than another uninterrupted successful scan.
4. Match verification to the claim. Use automated checks for behavior they can establish, and device or user testing where the environment matters. Record what each check demonstrates so a passing result has a clear meaning.
5. Review the change and its operating cost before release. Consider related changes, the human acceptance workload and recurring automation. Preserve the ability to explain why the result is ready and which questions remain open.

This is a synthesis of the work, rather than a claim that every project started with a formal five-step template. The pattern became clearer through implementation, feedback and corrections. It also gives me a better way to direct the next change: a specific problem, explicit boundaries and an appropriate standard of evidence.

Context guides the next change

I organize these independent products under Signal & Habit. Context matters in the product and in the development workflow: the restaurant's exact identity, an interrupted camera read, a person's first encounter with a form, or a developer's limited review allowance.

I want to shrink the distance between “this should exist” and a useful product. AI-assisted implementation helps me move along that path. My engineering responsibility is to make the resulting behavior understandable, testable and supported by evidence before I ask someone else to rely on it.