qgallouedec HF Staff commited on
Commit
2910d5b
·
1 Parent(s): ce1fbc2

README: name the class of bug that mattered most

Browse files

Three separate bugs each billed the user, reported success and returned something
broken. None was a crash, and none showed up in a benchmark — only in end-to-end
runs. Worth saying plainly at the top level rather than only in RECIPES.md.

Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -99,6 +99,12 @@ Three results worth knowing, all measured rather than assumed:
99
  1 of 20 answers short); from 1e-4 up it works and the value barely matters. The
100
  literature's rate clears it with ~2x margin, and lowering it "to be safe" would
101
  break the product silently.
 
 
 
 
 
 
102
 
103
  ## The endpoint
104
 
 
99
  1 of 20 answers short); from 1e-4 up it works and the value barely matters. The
100
  literature's rate clears it with ~2x margin, and lowering it "to be safe" would
101
  break the product silently.
102
+ - **Silent failures are the real enemy.** Three separate bugs each produced a run
103
+ that billed the user, reported success, and returned something broken: a merged
104
+ gpt-oss missing 23 of 24 layers' experts, a repo left holding a previous run's
105
+ stale artifact, and a run diverged to NaN that pushed a model answering every
106
+ prompt with "!!!!". None was a crash. All three are guarded now, and all three were
107
+ found by running the thing end to end rather than benchmarking it.
108
 
109
  ## The endpoint
110