you finish something. it is fine. you still feel bad.
in your head there is a better version of it. that better version is the ruler you hold yourself against.
the ruler is bent.
look back at a hard problem and it looks easy. you know the ending now. the you from back then did not.
the ruler gets drawn after the fact and then held against somebody who was working blind.
it assumes you decide to focus and then focus. getting into a good state is work on its own. the ruler charges you nothing for it.
the clean version of a day. no tiredness. no being stuck. no hour lost to your phone. that hour happens every time and the ruler has no mark for it.
i did something good last month. it took a normal amount of time. i felt slow the whole way through.
imagine a button. press it and the guilt goes. you also stop caring about the work. i would not press it. so the caring was never made of guilt.
picture a friend standing exactly where you are. you would tell them to ease up and you would be right.
guilt is a punishment. you learn from punishment slowly. you learn faster when something good happens too.
put the mark a little above where you normally land. not where a perfect person would land.
you will not wake up twice as good tomorrow. people move in small steps. the mark only needs to sit one step up.
clear it often and raise it a little. that is the shape of getting better.
the line is noisy. expect the noise.
not by deciding to. with facts.
write down how long things take. in a file. your head lies about this.
write down a good day. next time you feel slow hold yourself against that real day.
ask somebody. you are the most biased person about you.
guess a time then bet on it. imagine real money riding on you missing. what number do you say now.
you lose the morning and then raise the mark to make up for it. you miss. then you throw the whole day out.
i do this one a lot.
stop and draw the line again from here. you are not in the morning any more.
sometimes it is the thing being measured. a class you learn nothing in. a book that went dull four chapters back. the mark is fine. the goal is wrong. drop the goal.
research and building are both slow and both look obvious the moment they work.
with a bent ruler every day reads as a failure. nobody keeps walking into that.
so i track time. i ask people. and after a bad morning i draw the line again instead of pressing harder.
]]>most deployed systems are two things. a model that can write anything. and a classifier in front of it that reads the request, sometimes the answer too, and decides whether to let it through.
the classifier is trained on examples of bad requests. that is the whole problem, and everything below follows from it.
we asked eighteen models the same harmful request written five different ways. one way was plain english. another was the same request in formal bengali, the register of newspaper prose.
gemma-4-26b answers 18.5 per cent of them in english. in bengali it answers 90.8 per cent.
same model, same request, seventy-two points apart. the model did not get more capable in the second case. it got past something, and whatever that something was it had not read the request. it had read the shape of the request.
it holds across the table. the mean swing is 17.5 points and seventeen of the eighteen models move the same way. it is not really about bengali, because the gap turns up between two registers of bengali. it is not one grader being odd, because a judge from another vendor gives the same ranking. and it is not only small or careless models, because the safest model in our set still moves from 0.6 per cent to 37.9 per cent when the clothes change.
the numbers are all public and i am not going to relitigate them here. one thing from them is worth saying out loud though. the least safe model we measured is one built and served in europe, a continent that runs on a lot of languages and only one of them is english.
a classifier has to be right every time. whoever is probing it needs one phrasing.
and the phrasings are not a finite set you can work through. language, register, framing, transliteration, code mixing, indirection, politeness. they multiply against each other. we moved one of them and a seventy-two point hole opened, and we were not even hunting for the clever ones. we picked the five ways a normal person writes.
a classifier is fitted to a distribution of attempts, which means everything outside that distribution passes by construction. you cannot enumerate your way out, because the thing you are enumerating is produced by people who watch what your filter does and move somewhere else.
the two sides also scale differently, and this is the part i find hardest to argue around. capability comes from compute. filter coverage comes from somebody sitting down and writing examples of things that should be refused. one of those has a scaling law behind it. the other has a headcount. the distance between what a model can do and what its filter has seen gets wider every generation, for free, without anybody deciding it should.
satya nadella said something in a recent interview that i keep returning to. he said he had assumed alignment would follow from scale the way capability did, and that it has not turned out that way. he argued consistency has to be built in from the start rather than fitted at the end.
that is the whole thing in one sentence. the scaling bet paid out on capability. it did not pay out on alignment as a side effect. so alignment is something you spend training on deliberately or you do not get it, and a classifier is what it looks like when a company would rather not spend it.
a filter is a separate component. open weights ship without it.
whoever runs the model locally runs it with nothing in front. every number in our table is the model with its guard off, which is the only way an open model is ever actually used.
and weights can be trained further. a few hundred examples of the behaviour you want moves a model a long way, so safety in the weights is not a guarantee either. someone with the weights and a gpu can push it back out.
but there is a real difference between deleting a file and retraining a model, and plenty of open releases today do not even make you do the second one. nadella’s version of this is that an unconstrained open release is a problem for everyone, including whoever shipped it. i think that is right and i do not think it is an argument for closing everything. it is an argument for putting the safety somewhere it travels with the weights.
a refusal that comes from the model’s own understanding of what is being asked should not care what clothes the question wore, because by the time the model has understood it, it has mapped it to the same place whatever the language.
that is the claim worth testing and i cannot prove the mechanism from behaviour alone. what our data says is weaker and still useful. the models that barely move when you change the register are the ones with the most post-training behind them. the ones that swing thirty points are not. whatever the first group is doing, it is not a wrapper.
keep the filter. it catches the lazy attempts and it costs almost nothing.
stop reporting it as the safety property. report the model with the filter off, because that is what an open release is and what a jailbroken deployment is.
test in the languages you ship in, and inside each language test the registers. one phrasing per language tells you close to nothing.
and build the harm taxonomy from the law of the place the language belongs to. ours comes from bangladeshi statute. a north american content policy would have missed several of the categories outright.
]]>a door you walk past every day. you decided long ago it is locked. you have never once pulled the handle.
that is what i do with my biggest problems.
it feels like weather. a fact about the world. not a door at all.
i never weigh it up and decide against it. i flinch before i get that far.
what i want my life to be. my health. my work. being lonely.
i will fix a small squeaky thing today and leave the big door shut for a year.
i am not trying to get through. i am trying to see what is behind it.
open it is a heavy phrase. look at it is not. and you can start anywhere.
quiet room. phone somewhere else.
for most of my big doors i have never given ten honest minutes. so i do not know if they are locked. i only know they feel locked.
an hour is cheap. lose it and i lost an hour. win it and years change.
blank page. write everything you think about it. no standards.
write the problem out as if you were explaining it to someone. the holes show themselves.
imagine it is a month later and the door is open. what happened in between.
ask a friend for the obvious first thing. they say it in five seconds and you have dodged it for five years.
list ten things you could try. pick one. run it for a week. a failed one still tells you the shape of the room.
what is the most important problem in my life. why am i not working on it.
no good answer means a shut door.
or imagine a stranger living my life. what would i tell them to go and open.
a few problems are too big for now. that is real.
but you cannot tell from this side. you find out by pulling the handle once.
my instinct is to work on whatever is in front of me. small things get done. big doors stay shut.
so when i catch myself complaining about something i have never given an hour to i book the hour.
]]>you are at a bus stop. three buses come. all of them go to your town.
you want the fastest one so you let all three go past.
you are still at the stop.
almost any option gets you most of what you wanted. the gap between the good one and the best one is small. the gap between any bus and no bus is the whole journey.
i spend my effort on the small gap and none on the big one.
reading about buses feels productive. asking around feels productive. nothing moves.
that is the trap. the stalling wears the costume of care.
i can write one of these notes in about two hours. so ten hours spent picking is five notes i did not write.
put it in units you actually spend and it stops feeling free.
it feels like i am asking which bus. i am actually asking whether i ever go.
say it is two months from now and you still have not gone. are you surprised. if not then this was never now or later. it was now or never.
ask somebody which bus. they answer in five seconds from outside the stop.
take the obvious one. the one you would name without thinking.
or jump forward. it is a month later and you have been riding the same bus the whole time and it was fine. which one was it.
a fixed time. a reminder. somebody who charges you when you skip.
once it runs by itself the choosing stops costing anything.
some choices are expensive to undo and worth a real look.
most are not. most of mine were a bus.
i stall hardest on the things with the most upside. the vaguer and bigger it is the longer i stand there.
so when i notice i am comparing instead of moving i take the first one that goes to my town.
]]>