This is mostly a side effect of post-training models to not hallucinate, which has obviously been a major priority for a while now. They are highly incentivized to double check things to avoid accidentally making stuff up.
If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?
Except it just happened. Boeing was found to have massive safety issues since they were granted the right to self-certify. It made a lot of news but nothing much changed, they can still self-certify a bunch of stuff.
Runway incursions and midair collisions are another example.
Only airliners are required to have TCAS, smaller planes and helicopters don't even need radios or transponders unless in certain airspace. Midair collisions do lead to fatalities, enormous fatalities if an airliner is involved.
Runway incursions and overruns are similar. They cause lots of fatalities and injuries but only the busiest and largest airports have automated systems to warn when a runway is occupied or end of runway (overrun) arrestor systems. Most still rely on human voice to deconflict.
Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.
Airplanes have pretty bounded damage. Generally you kill at most a few hundred people. Even weaponized a few thousand. This is a risk profile that allows risk taking with near misses and waiting until something goes wrong to fix it (though doing so is rightfully uncomfortable and frequently unethical).
The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people". That's not a risk profile that allows for waiting to see if the risk is real, you have to prevent it before it happens. It's akin to the risk of the cold war going hot, not even "just" a nuclear reactor irradiating half of europe (which has yet to happen, but is a risk with nuclear reactors, chernobyl got uncomfortably close but ultimately was well contained).
Humanity does not have a great track record for "globally coordinated, collective action to solve/prevent global catastrophe." Look at how Climate Change is going.
The story of the safetyists has the convenient property of being unfalsifiable, so they can always claim doom is just around the corner. It’s the secular/EA version of the second coming.
huggingface stores weights and its service is replicated by /countless/ other sites. The world will not change even by one byte if it vanishes tomorrow
I have no horse in this race, but for fun on a literal rainy sunday afternoon I went in and confirmed bits of what happened myself. Besides huggingface, a bunch of wikis and url shorteners got hit too. My sympathies to the people who had to revert out all that mess.
The Huggingface incident strongly suggests that OpenAI is not able to make its products consistently law-abiding.
It's not like OpenAI carelessly neglected to consider that its products might break the law: it has spent literally billions of dollars in scientific and engineering talent and compute resources over many years to make its products law-abiding and otherwise "safe". For example there is a section titled "comply with applicable laws" in this document: https://model-spec.openai.com/2026-08-18.html#stay_in_bounds
The worry is that the only reason the Huggingface incident caused "zero actual damage" is because AI is not yet more ingenious than human criminals are, and of course our society has been hardened against human criminals. The worry is that as they become more capable, AI systems will start to be able to cause considerable damage, which will make the fact that the industry does not know how to control its
products enough to prevent that damage a big problem for all of us.
Open Weights is where the action is at in the past couple months, I’d have to think the US frontier labs are getting nervous. Like Anthropic hasn’t released anything pushing the frontier since “the event” earlier this summer.
If you don’t like your government’s policy vote for someone else. If you try to change it by force I will cheer on your suppression and incapacitation by force.
I don't think a country is well-served by an overly broad definition of terrorism.
You might disagree: the fact that you mention the military "shooting back" when we're discussing a jet-engine being sabbotaged suggests that you believe a state is allowed to be / ought to be disproportionate in its violence towards its citizens.
> the fact that you mention the military "shooting back" when we're discussing a jet-engine being sabbotaged suggests that you believe a state is allowed to be / ought to be disproportionate in its violence towards its citizens.
That is in no way a good faith interpretation of what I said.
The fact is that demand for tokens at electric bill rates so far outstrips what can be supplied currently not just with frontier models, but with open weights cheap models too. Running an always on Deepseek flash agent would cost three figures a month at API prices.
Total costs sure, electricity only costs no. My two DGX Sparks run DS4 Flash at about 50tok/s concurrency=1 which is more than suitable; at about 150W total wall power when generating.
That’s about A$16 a month in electricity if I ran it 7x24x30.
"In mice" is the right caveat for outcomes and the wrong one for mechanisms. Nobody’s selling you a drug here--they’re describing a chromatin change, and that’s the part rodents are actually good for.
The alternative study design is randomizing infants to early-life stress and biopsying their hippocampi at 30. Committee’s still reviewing it.
I built my machine in 2022 with 64GB of memory and upgraded to a 4090 at MSRP in 2023 so at this point I'm good but I fear the back end of the bathtub curve. I currently expect to have to hold on until 2029 and at that point the mobo and CPU will be 7 years old, and the GPU almost 6.
same-ish. In 2023 128 gigs of RAM and a 4090 and 4x 4TB ssds. I just looked up prices and 4090 alone is like 2/3 of what I paid for the whole system. Crazy times.
Maybe it could, maybe it could not. If it could and we had no defence against, it could be disastrous, even world ending (for us). The precautionary principle applies.
I'm not sure what advantage you see to a government in restricting this.
I have been doing this a lot even without a skill, having Fable write a planning document, then spawning an Opus subagent with instructions to strictly follow the plan and report any deviance at the end. It also helps that then the plan is always saved in an md file so any future agent can look at it and see what happened.
Interestingly, this was tackled in this blog post[0] a month ago. They claim that plan files aren't token-efficient, because after reading the plan the workhorse model then reads all the relevant files anyways.
That link just says the planning stage should vet the idea concretely so that the plan focuses on a solution that won’t immediately have to pivot.
And I think plan files should focus on general ideas and invariants, not do “implementation as prose”. That way they perform as mini-ADRs that are useful historically, especially to mine why the system is the way it is.
My plan and implementation files are task specific (so specific workhorse reads only its own slice), and the workhorse itself is spawned from an orchestrator with a very specific small prompt.
I managed even the orchestrator to NOT read the plan whole, at once, but in sections.
The most useful thing is the task ledger the task agent leaves behind, which alongside its structured status message makes a very resilient handoff between all stages.
reply