DeepSeek V4.1 Flash test: 3 of 3 runs green, under 2 cents
Can DeepSeek V4.1 Flash fix a real bug? 3 of 3 runs passed 18 failing time-zone tests, in about 90 s and under 2 cents each.
Senior Full-Stack Engineer- Published
- 8 minReading time
- Artificial IntelligenceCategory
Listen to this post· AI-narrated
0:00 / 9:08Can DeepSeek V4.1 Flash, the cheap open-weight model DeepSeek updated on 17 October, fix a real bug on its own? The launch post shows benchmark tables. I wanted to see it on a job I wrote the spec for. So today, 18/10/2026, I gave the new 1017 snapshot a time-zone bug that breaks real scheduling code: 18 failing tests and one rule, don't edit the tests. I ran it three times in the OpenCode CLI and checked every run against a clean copy of the tests. All three runs went green on the first test run, in about 90 seconds each, for under two cents a run. NVIDIA's free Nemotron model, run beside it, never finished: its free endpoint stalled twice, and its one real attempt hung on the 01:30 case.
TL;DR: DeepSeek V4.1 Flash 1017 fixed a real time-zone bug in 3 of 3 runs: 18 of 18 tests green each time, median 88.7 seconds, about $0.015 a run on OpenRouter. NVIDIA's free Nemotron never finished: two runs stalled on the free endpoint, one hung on a 32-bit bug. Test it on your own failing tests first.
What is DeepSeek V4.1 Flash 1017?
DeepSeek published the 1017 snapshot of V4.1 Flash, its cheaper model tier, on 17 October 2026; the number is the date. The model card (opens in a new tab) calls it a minor update with better reasoning, coding and agent results, shorter outputs and faster task completion. The weights are MIT-licensed, so anyone can host it. On OpenRouter (opens in a new tab), an API that routes requests to many model providers, it lists at $0.05 per million input tokens and $0.60 per million output tokens.
At that price a short agent session costs cents. The release got 32,876 likes on DeepSeek's launch post (opens in a new tab) and 808 points on Hacker News (opens in a new tab) in a day, and most of the replies argue about benchmark tables.
How I tested DeepSeek V4.1 Flash on a real bug
The task is a small TypeScript module with two empty functions: toUtcIso turns a date, a time and a time zone into a UTC timestamp, and tomorrowAt does the same for tomorrow. I schedule my own posts with this logic. In Lisbon the clocks jump from 01:00 to 02:00 on the last Sunday of March, so 01:30 that night doesn't exist, and code that assumes every wall-clock time exists breaks.
I wrote 18 tests for it with Node's built-in test runner, all failing at the start. They cover summer and winter time, the day of each switch, the missing hour in March, the repeated hour in October, New York and Kolkata's half-hour offset, a leap day, bad input, and "tomorrow" across a switch and a month end. The tests also pick a policy for the missing hour: 01:30 moves to 02:00 local, the first valid time, which is 01:00 UTC. Some libraries shift by the size of the gap instead; here the tests decide. The one that matters most:
const L = "Europe/Lisbon";
test("spring-forward gap moves to the first valid instant", () =>
assert.equal(toUtcIso("2026-03-29", "01:30", L), "2026-03-29T01:00:00.000Z"));
Every run got the same prompt, in a fresh copy of the folder:
Implement the two functions in schedule.ts so that every test in schedule.test.ts passes.
Do not edit schedule.test.ts. Use only Node built-ins, no packages.
Run `node --test schedule.test.ts` to check your work. Stop when all tests pass.
The agent ran in OpenCode 1.18.34, an open-source coding agent for the terminal, with opencode run --pure --auto, on my MacBook Pro M3 Max, with a 15-minute limit. When it stopped, a script compared the test file with the original, copied the original tests next to the agent's code and ran them again, one test at a time with an 8-second limit, so a hang counts as a failed test. I ran DeepSeek and NVIDIA's Nemotron 3.5 Super VL (the VL means it also reads images) three times each, alternating them. Nemotron is one of the models OpenCode offers for free, and free is what a cheap model competes with.
One setting I couldn't hold equal is the route to the model. DeepSeek ran through my paid OpenRouter key. Nemotron ran on OpenCode's free endpoint, the only way it's free.
Did DeepSeek V4.1 Flash fix the bug?
Yes, every time. Each run read the two files, wrote schedule.ts once, ran the tests once, saw 18 passes and stopped:
| Run | Tests passing (clean copy) | Time | Cost (OpenCode log) | Test file edited |
|---|---|---|---|---|
| 1 | 18 of 18 | 90.3 s | $0.0153 | no |
| 2 | 18 of 18 | 87.6 s | $0.0144 | no |
| 3 | 18 of 18 | 88.7 s | $0.0165 | no |
Median 88.7 seconds, range 87.6 to 90.3. The first test run came about 1.3 minutes into each session, and it was green every time, so there was nothing to iterate on. The three solutions differ in size, 139, 191 and 223 lines, and all three handle the missing hour in March and the repeated hour in October.
Three runs is a small sample, and one task isn't a benchmark. It is the job I care about, though: a precise spec in tests and a model that has to meet it without bending them.
How did the free Nemotron model do on the same test?
Nemotron 3.5 Super VL, free in OpenCode, didn't finish any of its three runs within 15 minutes, though only one of them was a real attempt.
Runs 1 and 3 never started working. In both, the model read the two files and then got no further response from the free endpoint for the rest of the 15 minutes: one step and 102 output tokens each. The model never got a second turn, so those two failures belong to the free service. Either way, the free route didn't produce a fix today.
Run 2 got further. It wrote a solution that passes 17 of the 18 tests and hangs forever on the spring-forward test, the 01:30 that doesn't exist. Its binary search finds the midpoint like this:
const mid = (lo + hi) >> 1;
>> works on 32-bit integers in JavaScript, and lo and hi are millisecond timestamps around 1.8 trillion. The shift chops them, the midpoint lands nowhere near the range, and the loop never ends. The agent noticed the hang, isolated it to the 01:30 Lisbon case, and was still working on it when the limit hit. I changed that one line to Math.floor((lo + hi) / 2) afterwards, and its code passed all 18 tests.
What does DeepSeek V4.1 Flash cost per fix?
About a cent and a half. OpenCode logged $0.0144 to $0.0165 per run, $0.046 for all three, from OpenRouter's reported cost per step. My OpenRouter account's usage rose by $0.030 between the start and the end of testing. I can't tell from the outside why the two figures differ, and both put each fix under two cents. Most of the input was cached, 203,000 to 353,000 tokens per run, which is cheaper than fresh input.
Two things this test doesn't tell you. A long agent session in a big repository will cost more than 90 seconds on a small module. And I didn't run a paid frontier model on the same task today, so this isn't a comparison with whatever you pay for now. That's the next test.
How to run this test on your own code
You can run the same test on your own code in an afternoon:
- Pick a bug you understand and write failing tests that define "fixed". Make them precise enough that a lazy fix fails.
- Write one rule into the prompt: don't edit the tests.
- Run each model at least three times, in a fresh copy each time, with a time limit.
- Grade every run yourself on a clean copy of the tests, and run each test with its own timeout so a hang counts as a failure.
- Record time, cost and whether the test file changed, and keep the code.
I built SharpOS, the platform my studio SharpHaw runs on, test-first with agents, and this test runs on the same rule: tests are the spec, and they decide which run you keep. I want three green runs in a row before a model gets a place in my workflow.
What I'd do next
For a well-specified bug like this one, DeepSeek V4.1 Flash 1017 did the job three times out of three for about a cent and a half each, and that's enough for me to hand it more of this kind of work. I'd still check it on a long session in a real codebase before making it my default, because longer sessions cost more and give a model more room to wander off the task.
If you want more tests like this, follow me on X (opens in a new tab) or Instagram (opens in a new tab), or read the rest of the blog on gabs.app. If you're hiring a senior full-stack engineer who builds with agents, or you need a product built, get in touch on gabs.app.