Building a Hermes-backed GitHub PR reviewer

The first concrete workflow in this series is a GitHub PR reviewer. A pull request event arrives, a worker gathers context, Hermes generates a review, and the service can post the result back to GitHub.

The useful implementation details are mostly around that model call.

This is an architecture walkthrough of the repository, not a claim that every supported option is enabled in production. The code supports a one-shot worker and an opt-in reviewer-only orchestrator. I will start with the simpler path.

Keep the receiver small

The basic flow is:

GitHub pull request event
  -> validate request and select supported events
  -> identify the repository, PR, and head commit
  -> deduplicate and dispatch a worker
  -> gather PR context and generate a review
  -> publish a result for that revision

The receiver accepts pings and pull request events. Supported PR actions include opening, synchronizing, reopening, and marking a PR ready for review. Draft PRs are ignored.

Signature verification uses X-Hub-Signature-256. The implementation calculates an HMAC over the raw request bytes and compares it with hmac.compare_digest. It verifies signatures when a webhook secret is configured; that conditional matters. For an internet-facing deployment, I would require the secret rather than treat it as an optional convenience.

The receiver logs a sanitized event envelope instead of the full webhook payload. That reduces unnecessary retention of submitted content. It does not make every downstream worker log safe to publish.

There is also a repository allowlist. An empty allowlist preserves the broader existing behavior, so a restricted deployment needs to configure it explicitly.

Give each revision an identity

Jobs use repo#pr@head_sha as their idempotency key.

That is more useful than identifying work by PR number alone. A new commit deserves a new review. Receiving the same event again should not automatically produce another independent review of the same commit.

The one-shot worker uses gh for GitHub access and hermes chat for review generation. Posting is explicit. The documented manual interface can generate a review without --post, or publish with it.

I like that separation because inspecting an output and writing to GitHub are different actions. It gives the workflow a useful place to stop during development.

Deduplication should not be confused with a universal exactly-once guarantee. Writing a local record and making an external API call are separate operations. Crash recovery and ambiguous API outcomes still need attention.

The queue has a limitation worth spelling out

The receiver records jobs in JSONL, but calling that a queue can suggest more than it provides.

In the automatic dispatch path, the receiver first tries to reserve a worker slot. If the concurrency limit is reached, it records concurrency_limited. It does not enqueue that request for later execution.

It still responds with HTTP 202 for that pull request event.

That means an accepted HTTP request is not proof that a review will eventually happen. The current admission path does not provide a durable backlog for overflow work, and the successful response should not be treated as a retry mechanism.

A worker cap is useful for controlling resource use. If I need guaranteed eventual processing, I need a durable pending-job mechanism and a dispatcher that drains it. I should not describe the present implementation as though it already has those properties.

Keep orchestration separate from review generation

The repository also supports an opt-in orchestration path. It persists run state in SQLite and writes structured artifacts to disk. Its documented webhook integration does not mutate PR branches; automatic fixing is not enabled there.

That is a useful boundary for expanding the system. Multiple reviewer outputs can become inputs to a controlled decision process without immediately granting the process permission to edit code.

The default documented dispatch path remains the one-shot worker unless the agentic option is enabled. Support in the repository and activation in a deployed service are separate facts.

Make failure visible where the work happens

The receiver monitors dispatched processes and supports timeouts and a concurrency cap. When posting is enabled, generation failures have a deduplicated failure-comment path. The orchestrator also publishes commit statuses so a person can distinguish a pending review from one that needs attention.

Local logs and artifacts provide the detail behind that GitHub surface. The repository includes an operations CLI for recent runs, failures, and health, plus a retention command with a dry-run mode.

None of this measures review quality. A successfully completed run can still produce a weak review. But it does make the service easier to inspect than a background model call that may or may not leave a comment.

For this first implementation, that is the point: a review tied to a specific change, explicit publishing, and enough evidence to investigate what happened. The next step is to examine the structured contracts between reviewers and the code that decides what to do with their results.