Go on Mengboy Tech Notes

Go Dual-Provider LLM Routing (OpenAI + Claude): Timeout Tiers, Cost Caps, and Fallback Control

Wed, 08 Apr 2026 01:22:53 +0000

If your Go service relies on one LLM provider, two failures hurt the most, timeout spikes and billing spikes. A real production setup is not just “add another provider”, it is a single control plane for routing, timeout tiers, cost caps, and fallback.

This guide gives you a practical OpenAI + Claude dual-provider pattern with one priority, keep uptime first, then optimize quality.

OpenAI Responses + Go Stream Recovery: Delta Persistence, Resume Tokens, and Duplicate Chunk Dedup

Mon, 23 Mar 2026 01:13:09 +0000

In production, the painful part is not “streaming is slow.” It’s “streaming breaks and then duplicates output after reconnect.”
This guide gives you a practical recovery loop: delta persistence + resume token + idempotent dedup, so reconnection does not replay garbage.

Go + OpenAI Responses Agent Memory Layering: Short-Term Context, Long-Term Index, and Cost Caps

Wed, 18 Mar 2026 16:33:52 +0000

In production Go agents, the first thing that breaks is usually not model quality. It is memory management: context grows, bills spike, and answers drift.

Use a 3-layer memory design:

L1: short-term conversational window (seconds)
L2: rolling summary (minutes)
L3: long-term retrieval memory (days)

Handling OpenAI 429/5xx Storms in Go: Token Bucket, Exponential Backoff, and Circuit Breakers

Wed, 18 Mar 2026 01:14:00 +0000

Most Go teams are not killed by a single API error. They are killed by a retry storm they created themselves.

OpenAI Batch API with Go: Offline Batching, Failure Replay, and Cost Boundaries

Fri, 13 Mar 2026 01:08:00 +0000

Short answer: if your workload is delay-tolerant, batchable, and replay-safe, move it from online calls to Batch API. The savings are real, but only if you design splitting, failure routing, and replay first.

Many teams treat Batch API as a cheaper sync endpoint. That usually creates a replay mess instead of stable savings. A conservative rollout starts with cost boundaries and SLOs, then implements offline batching and controlled replay.

OpenAI Responses Structured Outputs with Go: Schema Evolution, Bad-Case Fallbacks, and Gradual Rollback

Wed, 11 Mar 2026 01:08:00 +0000

The hardest part of Structured Outputs is not getting JSON once. It is surviving schema changes without turning production into a small fire with excellent logs and terrible business results.

Once a Go service starts evolving prompts and response contracts, the usual failure modes show up fast: a new required field breaks older consumers, an enum expands and strict validation kills valid requests, or one bad sample drags the whole chain into retries and rollback panic.

OpenAI Realtime + Go in Production: WebRTC Token Rotation, Interruption Recovery, and End-to-End Latency Budgets

Mon, 09 Mar 2026 01:13:00 +0000

If you plan to put OpenAI Realtime into production, do not let a passing demo fool you.

What usually breaks the system is not the model itself. It is non-rotating short-lived auth, missing interruption state, and zero end-to-end latency budgeting. Miss those three and your voice UX starts sounding like an angry walkie-talkie.

Go + OpenAI Responses: Connection Pooling and Timeout Budgets from HTTP/2 Reuse to Error-Budget Control

Fri, 06 Mar 2026 01:13:12 +0000

When Go services call the OpenAI Responses API in production, the real failures are rarely about model quality. Most incidents come from transport instability: weak connection pooling, conflicting timeout layers, and retry storms.

This guide gives you a practical baseline: HTTP/2 reuse, layered timeout budgets, bounded retries, and error-budget driven operations.

OpenAI Responses + Go: Taming Retry Storms with Idempotency Keys, Jittered Backoff, and Circuit Breakers

Wed, 04 Mar 2026 01:10:40 +0000

The most expensive outage is not a single failure — it is a failure amplified by retries.

In an OpenAI Responses + Go tool-calling stack, missing idempotency, jittered backoff, and breaker thresholds can turn 10 failing requests into 1000 downstream calls in minutes.

Taming Context Explosion in OpenAI Assistants/Responses with Go: Truncation, Summary Backfill, and Cost Caps

Mon, 02 Mar 2026 12:44:00 +0000

Long-running agent sessions usually fail the same way: context keeps growing, latency spikes, costs blow up, and answer quality gets worse.

That is rarely a model-quality issue. It is almost always missing context governance.

Go + OpenAI API Timeout Troubleshooting: DNS, TLS, Proxy, and Connection Pool

Mon, 02 Mar 2026 01:12:10 +0000

When OpenAI API calls start timing out in production, the real problem is usually not “OpenAI is down.”

The real problem is you don’t know which hop is failing: DNS, TLS handshake, proxy path, or your own connection pool.

OpenAI Agents SDK with Go: Tool Calling, Session Memory, and Error Recovery

Wed, 25 Feb 2026 01:18:00 +0000

Most teams can connect an LLM in a demo. The real pain starts in production: multi-step tasks, flaky tool calls, unclear retries, and rising cost.

This guide gives you a pragmatic Go-first blueprint for shipping an Agent workflow that can survive real incidents.