Add required_state to gh aw checks --json to isolate CI verdict from optional third-party commit statuses

[Copilot]

gh aw checks --json collapsed any failing third-party commit status (Vercel, Netlify deployment previews, etc.) into state: "failed", even when all GitHub Actions check runs passed — causing false REQUEST_CHANGES from the review agent.

Changes

• New required_state field in ChecksResult JSON output — computed from check runs plus policy/account-gate commit statuses only (classifyCheckState(checkRuns, policyStatuses(statuses))). Optional third-party deployment statuses (Vercel, Netlify, etc.) are excluded, but branch-protection and account-gate commit statuses are still included and can yield policy_blocked. The existing state field is unchanged.
• New policyStatuses() helper filters commit statuses to only those matching isPolicyCheck(), reusing the existing policy-pattern logic.
• Updated help text documents both fields and directs consumers to use required_state as the authoritative CI verdict in repos with optional deployment integrations.

JSON output shape

{
  "state": "failed",           // aggregate: check runs + all commit statuses
  "required_state": "success", // check runs + policy commit statuses only — Vercel/Netlify excluded
  ...
}

required_state can return any of the normalized states including policy_blocked when branch-protection or account-gate commit statuses fail. Downstream automation (e.g. review agent) should gate on required_state rather than state when optional deployment integrations are present.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>gh aw checks --json collapses optional third-party failures into top-level state</issue_title>
<issue_description>## What happens

When external services like Vercel or Netlify post a failing commit status on a PR head commit, gh aw checks <PR_NUMBER> --json returns state: "failed" even if all required CI checks have passed.

classifyCheckState() marks any non-policy failed commit status as failed. There is no distinction between:

• Required checks that should block review/merge
• Non-required third-party deployment statuses

That makes gh aw checks unsuitable as an authoritative CI verdict in repos that have optional deployment integrations posting statuses.

What should happen

gh aw checks should either:

• Distinguish required checks from non-required third-party deployment statuses, or
• Provide a mode/output field that reports required-check state separately from raw overall status aggregation

At minimum, a failing optional deployment status should not be collapsed into the same failed result as a required CI failure.

Where in the code

All references are to main at 55e326bff (verified unchanged from 2d91393f3).

• pkg/cli/checks_command.go:59 — ChecksResult is a single top-level state with no separate required/optional field
• pkg/cli/checks_command.go:286-295 — policyCheckPatterns only exempts policy/account-gate names like required status check, branch protection, mergeability, etc. It does not model required-vs-optional checks.
• pkg/cli/checks_command.go:336 — In classifyCheckState(), any commit status with state failure or error sets hasFailed = true unless isPolicyCheck(context) matches. That means a failing Vercel/Netlify-style status is collapsed into top-level failed.
• pkg/cli/checks_command_test.go:96 — Existing tests encode that behavior: failing commit statuses are expected to return CheckStateFailed.

Relevant logic:

• Failed check runs -> hasFailed = true unless treated as policy
• Failed commit statuses -> hasFailed = true unless treated as policy
• No required-vs-optional filtering exists for Vercel/Netlify-style statuses

Evidence

Production evidence of the underlying problem (PR github/gh-aw#145, 2026-02-27 ~22:41 UTC):

• PR [copilot] generated by ai footer message #145 implemented a feature with all 7/7 acceptance criteria passing
• Vercel (connected from a prior Next.js run) posted failure on the head commit — Vercel cannot build a C# project, which failed on this project in that run
• The review agent issued REQUEST_CHANGES citing CI failure, despite all required checks passing
• Same pattern hit PR github/gh-aw#141 minutes earlier — identical outcome
• Consumer-side workaround applied at 81b741b: review agent prompt updated to ignore third-party deployment statuses

Source-level verification (2026-03-01): The current classifyCheckState() on main at 55e326bff would collapse that same Vercel failure status into top-level failed in gh aw checks --json output. The code path has no required-vs-optional filtering — any non-policy commit status with state failure sets hasFailed = true.

This is separate from auto-merge gating in #19020. That issue is about merge behavior based on mergeable=UNSTABLE. This one is about reporting/classification in gh aw checks itself.

Proposed fix

Possible approaches:

1. Classify only required checks/statuses when computing the top-level state
2. Add a second field such as required_state alongside the current aggregate state
3. Add an ignore/filter mechanism for known third-party deployment contexts like Vercel, Netlify, etc.

Impact

Frequency: Affects repos that have optional third-party deployment integrations (Vercel, Netlify, Cloudflare Pages, etc.) posting commit statuses and use gh aw checks --json as a CI verdict.
Cost: Medium — false failed results mislead downstream automation. In our pipeline, this caused REQUEST_CHANGES reviews on PRs where required CI passed.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes gh aw checks --json collapses optional third-party failures into top-level state #19158

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Copilot]

Pull request overview

Adds a separate JSON-reported CI verdict for gh aw checks --json that reflects GitHub Actions check runs only, avoiding false failures caused by optional third-party commit statuses (e.g., Vercel/Netlify).

Changes:

• Added required_state to ChecksResult JSON output, computed from check runs only.
• Updated checks command help text to document state vs required_state.
• Added unit tests covering the intended required_state semantics vs aggregate state.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
pkg/cli/checks_command.go	Adds required_state to JSON output and documents it in command help; computes it during result assembly.
pkg/cli/checks_command_test.go	Updates ChecksResult shape test and adds new tests validating required_state behavior.

Comments suppressed due to low confidence (1)

pkg/cli/checks_command_test.go:259

• The section header for the classifyGHAPIError tests was replaced by the new required_state section, but the classifyGHAPIError tests still follow immediately after. Consider reintroducing a header comment before TestClassifyGHAPIError_* to keep the test file structure clear.

// ---------------------------------------------------------------------------
// required_state — commit status failures do not affect check-runs-only state
// ---------------------------------------------------------------------------

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

TestChecksResultJSONShape doesn't actually marshal ChecksResult to JSON, so it won't catch a wrong JSON tag/name (e.g., required_state vs requiredState) or missing fields. Since this PR adds a new JSON field, please marshal/unmarshal and assert that the encoded JSON contains the required_state key with the expected value.

[Copilot]

Fixed in 7f35c19. TestChecksResultJSONShape now marshals the struct to JSON, unmarshals into map[string]json.RawMessage, and asserts that all expected keys (state, required_state, pr_number, head_sha, check_runs, statuses, total_count) are present with correct values.

[pelikhan]

@copilot review and apply comment. Surface policy_blocked as a field.