AI can draft this Kafka deduplication refactor, but someone needs to test it against the real service.

The pattern — idempotent upserts with a deduplication key, Kafka offset commit after successful DB transaction — is well-established and structurally similar across services. However, the exact schema, consumer group setup, and crash-recovery semantics vary per codebase, requiring some bespoke judgment each time.

The success criterion (<99.9% duplication rate, rollback on crash) is numerically defined, which is good. But key design decisions are underspecified: what constitutes the deduplication key, whether to use ON CONFLICT DO NOTHING or DO UPDATE, and how Kafka offsets are committed relative to DB commits all require choices the task doesn't resolve.

The agent almost certainly lacks access to the actual Go source files, PostgreSQL schema, Kafka topic configuration, and existing consumer group state. Without these, it must write against plausible assumptions, producing code that may need significant adaptation before it compiles or runs correctly.

A flawed refactor in a production Kafka consumer can cause data loss, offset mismanagement leading to message skips or reprocessing storms, or silent duplication that defeats the entire purpose. At 10k messages/hour, even a brief bad deploy has real downstream impact and may be hard to fully reverse.

Choosing the right deduplication strategy (e.g., message ID vs. content hash vs. business key) and deciding on exactly-once vs. at-least-once semantics with idempotent consumers requires understanding the business domain and acceptable trade-offs. A senior engineer review of the agent's output is genuinely necessary before deployment.