Field Notes

Asking directly for “the best option” returns the most typical, generic answer — aligned models keep only ~24% of the base model’s diversity, vs ~67% with this technique. Only for creative work, never facts, code, or math.

Each persona made a true but different point, so the piece had five lenses and no spine — and the drift was felt even though every point held up alone. Write the thesis in one sentence first; every contribution must deepen it, not argue a new point about the same topic.

Returned a 3-bullet summary and got “I don’t think you’ve done enough thinking about this.” Work out which findings contradict what you’ve built, which expose a wrong number, which don’t actually apply, and the one thing they prove wrong.

Sequential single-agent builds look slow on screen even when they’re fast, and silence during runtime reads as “broken.” Pre-stage research into files, engineer the prompt to force visible parallel subagent dispatch, and script 60–90s of narration for the beat where agents work silently.

Repeatedly asking for values already sitting in a local config file or CI secret store. Check dotfiles, secret stores, config files, and docs before asking — and when you do ask, name exactly what you already checked.

Stacked watcher loops each kept their own successor alive forever. Use the runtime’s built-in completion notification, or poll a sentinel file the target writes on finish — never a process-name grep that can match the watcher itself.

A webhook returned “not found” for an unknown span while several jobs and a smoke test pointed at it — none alerted, because each caller only checks its own success. Treat “job succeeded but no one heard” as its own failure class, and periodically ping every endpoint directly.

A self-push used a token that silently lacked write access; the || true hid the 403 for weeks. Let consequential steps fail loudly; if a push is genuinely best-effort, log the failure visibly — and verify the token scope on any self-push.

A daily pipeline lost its output when the provider deprecated a pinned model; the exception logged as a generic LLM failure. When expected output goes missing, check the model constants first; prefer “-latest” aliases over dated strings.

A “never crashes” wrapper let a helper’s sys.exit(1) on a missing dependency kill an entire multi-worker run, while a separate error-swallowing pattern hid it behind a green status for days. Catch (Exception, SystemExit) together; don’t let library code call sys.exit at all.

A summarization step called the LLM helper bare; when every provider was exhausted at once, that one unguarded call crashed the whole downstream pipeline. Wrap every call site and degrade gracefully — proceed with fewer inputs rather than let the exception propagate.

Hand-patched a live generated page; the fix disappeared on the next scheduled regeneration because the template was never touched. Cap and validate at the point of generation; treat bulk “fixer” scripts as a safety net, not the primary defense.

A morning automation burned one provider’s entire daily quota partway through its run because it was listed first, starving the capacity needed for interactive daytime use. Order by which quota to burn versus protect, not by preference.

Every place the old name lives — data schema, pipeline code, bot config, public pages, docs — has to move together or the live product goes inconsistent. Treat it as a feature branch with a dependency map, and migrate additively (add the new name, migrate, then drop the old).

“Agentic loop” framing gets pitched for taste-driven work (design, copy, angles) where “done” is subjective. Only loop mechanical work with real exit codes; for judgment work, write an actual verifier or admit it needs a human.

A market-maker built state machines and kill switches before the one honest test: a queue-aware fill simulation. Build the cheapest test of the load-bearing assumption first, before the machinery around it.

A confidence floor built to prevent losses was shadow-tracked against the bets it filtered out — it was killing the most profitable bet type. Position-aware thresholds beat one universal floor when win rates are asymmetric.

The instinct was the on-brand domain; the data showed another had far more markets near real uncertainty resolving fast. Optimize for volume of training examples and speed of feedback — one-off events teach a model nothing repeatable.

After four months and two strategies, the honest framing was “we can make a few dollars, but not at the cost of focus.” Pre-register shelve criteria and dates before you start; when “this is the real shot” fires three times in a row, the problem is structural, not parametric.

A multi-agent chat had several personas answering from canned scripts (no model call at all) alongside one genuinely live backend. Architect and disclose the boundary between “live” and “performed” explicitly, rather than letting the UI imply everything is equally live.

A live-build demo pre-loaded reference files the agent read in seconds and acted on as if researching from scratch. If asked “did this happen live?”, point honestly to the pre-work stage — legitimate staging is the trick, not deception. Being upfront is more credible than false spontaneity.