Spaces:
Running
Running
Commit ·
2910d5b
1
Parent(s): ce1fbc2
README: name the class of bug that mattered most
Browse filesThree separate bugs each billed the user, reported success and returned something
broken. None was a crash, and none showed up in a benchmark — only in end-to-end
runs. Worth saying plainly at the top level rather than only in RECIPES.md.
README.md
CHANGED
|
@@ -99,6 +99,12 @@ Three results worth knowing, all measured rather than assumed:
|
|
| 99 |
1 of 20 answers short); from 1e-4 up it works and the value barely matters. The
|
| 100 |
literature's rate clears it with ~2x margin, and lowering it "to be safe" would
|
| 101 |
break the product silently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
|
| 103 |
## The endpoint
|
| 104 |
|
|
|
|
| 99 |
1 of 20 answers short); from 1e-4 up it works and the value barely matters. The
|
| 100 |
literature's rate clears it with ~2x margin, and lowering it "to be safe" would
|
| 101 |
break the product silently.
|
| 102 |
+
- **Silent failures are the real enemy.** Three separate bugs each produced a run
|
| 103 |
+
that billed the user, reported success, and returned something broken: a merged
|
| 104 |
+
gpt-oss missing 23 of 24 layers' experts, a repo left holding a previous run's
|
| 105 |
+
stale artifact, and a run diverged to NaN that pushed a model answering every
|
| 106 |
+
prompt with "!!!!". None was a crash. All three are guarded now, and all three were
|
| 107 |
+
found by running the thing end to end rather than benchmarking it.
|
| 108 |
|
| 109 |
## The endpoint
|
| 110 |
|