Code review shows repo changes. Who reviews product behavior?

A PR can look boring and still change the product. A copy branch changes a disclosure. A routing cleanup changes who sees a paywall. A prompt/config update changes what an AI workflow returns. The diff may be understandable. The customer-facing behavior may still be unreviewed.

Keep code review. GitHub describes pull request reviews as a way to comment on, approve, or request changes to proposed changes. That workflow is valuable because it gives teams a place to reason before merge. The problem is that a repository diff is only one view of a release.

The review gap is product behavior

The question I keep coming back to is simple: after the PR preview runs, what did the product do?

Code review can tell you a function moved, a branch condition changed, or a dependency was updated. Tests can tell you whether known assertions still pass. AI code review can add useful comments on implementation. GitHub's Copilot code review documentation says Copilot provides review feedback and leaves a comment review. Human approval still needs a durable record of the live preview flow.

That record matters most in the places teams already worry about: billing, onboarding, auth, permissions, AI behavior, workflow automation, and disclosure-heavy UX. A small mismatch there can create a confused customer, a bad release note, or a merge approval the reviewer regrets.

What a product-behavior reviewer should ask

Before approving a risky PR, I want the review to answer a few concrete questions. These checks help even without ReleaseNotary.

• Which customer flow changed, and why is it risky enough to inspect?
• Which preview URL did we check, and which commit or release state did it represent?
• What did the core flow do when someone ran it?
• What evidence was captured: text, screenshots, network state, artifacts, or reviewer notes?
• What changed in access, outcome, pricing, disclosure, or AI behavior?
• What should a human reviewer inspect first before approving the merge?

NIST's Secure Software Development Framework is useful here because it treats review and testing as related but distinct ways to find and record issues in the development workflow. Product teams need that same separation: review the code, test the executable product, and preserve the evidence reviewers need.

Where ReleaseNotary fits

ReleaseNotary is my narrow answer to this gap. It starts from the GitHub PR or release preview, runs configured checks and core flows, captures evidence, computes a Product State Hash, raises Notary Flags, and produces a hosted Notary Packet plus hosted release notes. GitHub stays compact. The packet carries the detail.

The point is a better review surface for human approval. If the reviewer wants to understand the behavior behind the PR, give them evidence instead of forcing them to reconstruct the flow from diffs, screenshots in chat, and a memory of what someone clicked during QA.

The current wedge is small on purpose: product behavior proof for teams shipping through GitHub PRs and previews. The best fit is a few sensitive use cases. You can also see how it compares with AI code review, visual QA, observability, and compliance tools, how packet access is handled in the security model, and what the founder-led pilot costs.

A useful rule even without ReleaseNotary

For any PR that touches a sensitive flow, add one product-behavior note to the review. Keep it short: preview URL, flow checked, expected outcome, observed behavior, and evidence someone can revisit.

That habit changes the approval conversation. Instead of asking, "Does the diff look fine?" the team can ask, "Did the product do the thing we are about to release?"

Send me the examples code review missed

I am collecting examples of PRs where code review looked reasonable but runtime or product behavior surprised the team later. Public examples are easiest. Private ones are welcome if you can describe them without secrets, customer data, signed packet URLs, tokens, raw provider output, or private emails.

The pattern I am looking for is simple: what changed in the repo, what changed in the product, and what evidence would have helped the reviewer before release?