Lab runs
Every night at 02:00 UTC, five coding agents get 40 tasks against Postbox in fresh sandboxes, five trials each. Another 20 prompts per agent name no vendor, to see who they pick.
By night
- Code worked
- Picked Postbox
| Run | Started | Code worked | Picked Postbox | Median turns | Median time | Duration | Ref |
|---|---|---|---|---|---|---|---|
| 412 | Oct 9, 02:00 | 72% | 18% | 14 | 4.2 min | 43 min 12 s | main |
| 411 | Oct 8, 02:00 | 72% | 18% | 14 | 4.4 min | 45 min 19 s | main |
| 410 | Oct 7, 02:00 | 72% | 19% | 15 | 4.3 min | 40 min 48 s | main |
| 409 | Oct 6, 02:00 | 72% | 19% | 15 | 4.5 min | 43 min 29 s | main |
| 408 | Oct 5, 02:00 | 72% | 18% | 15 | 4.0 min | 44 min 38 s | main |
| 407 | Oct 4, 02:00 | 72% | 17% | 14 | 4.2 min | 41 min 22 s | main |
| 406 | Oct 3, 02:00 | 72% | 18% | 15 | 4.0 min | 44 min 11 s | main |
| 405 | Oct 2, 02:00 | 72% | 18% | 14 | 4.1 min | 44 min 41 s | main |
| 404 | Oct 1, 02:00 | 72%−2 | 18% | 13 | 3.9 min | 46 min 9 s | main |
| 403 | Sep 30, 02:00 | 74% | 20% | 14 | 3.9 min | 43 min 27 s | main |
| 402 | Sep 29, 02:00 | 74% | 19% | 14 | 3.9 min | 39 min 36 s | main |
| 401 | Sep 28, 02:00 | 74% | 20% | 14 | 4.1 min | 46 min 15 s | main |
| 400 | Sep 27, 02:00 | 74% | 20% | 13 | 4.3 min | 42 min 24 s | main |
| 399 | Sep 26, 02:00 | 74%+3 | 20% | 14 | 4.1 min | 44 min 43 s | main |
| 398 | Sep 25, 02:00 | 71%+1 | 17% | 16 | 4.8 min | 46 min 57 s | main |
| 397 | Sep 24, 02:00 | 70% | 16% | 15 | 5.0 min | 43 min 26 s | main |
| 396 | Sep 23, 02:00 | 70% | 16% | 14 | 4.9 min | 39 min 48 s | main |
| 395 | Sep 22, 02:00 | 70% | 18% | 14 | 4.6 min | 43 min 20 s | main |
| 394 | Sep 21, 02:00 | 70% | 16% | 16 | 4.6 min | 46 min 21 s | main |
| 393 | Sep 20, 02:00 | 70% | 17% | 14 | 4.5 min | 47 min 44 s | main |
| 392 | Sep 19, 02:00 | 70%+9 | 17% | 14 | 4.8 min | 44 min 35 s | main |
| 391 | Sep 18, 02:00 | 61%−1 | 13% | 17 | 5.2 min | 45 min 37 s | main |
| 390 | Sep 17, 02:00 | 62% | 14% | 16 | 5.1 min | 39 min 47 s | main |
| 389 | Sep 16, 02:00 | 62% | 13% | 16 | 5.1 min | 45 min 35 s | main |
| 388 | Sep 15, 02:00 | 62% | 15% | 17 | 5.5 min | 42 min 29 s | main |
| 387 | Sep 14, 02:00 | 62% | 14% | 17 | 5.1 min | 47 min 54 s | main |
| 386 | Sep 13, 02:00 | 62%−1 | 14% | 17 | 5.3 min | 42 min 30 s | main |
| 385 | Sep 12, 02:00 | 63%+1 | 15% | 18 | 5.1 min | 43 min 28 s | main |
| 384 | Sep 11, 02:00 | 62% | 15% | 18 | 5.6 min | 41 min 26 s | main |
| 383 | Sep 10, 02:00 | 62% | 13% | 17 | 5.6 min | 41 min 28 s | main |