Writing is fluff. What do you want, a list of bullet points? There would be no books. Maybe you have a career in writing 2 page books?

I like books but I don't find KJV's writing style engaging at all. The Hobbit was much better IMO

You ever wonder why recent claude models speak in riddles? I dunno, maybe all those "rare" books? They may have been rare for a reason.

Rare like a steak not like a treasure.

Well, I can tell what sort of angle you most enjoy. Anyways -- I think there is GOOD writing and BAD writing, but only subjectively. So if you enjoy it, power to you. It's certainly not random, but it is the sort of verbosity that turns off 99 percent of the people that would read it given a comparison. I find the former rather eloquent.

Like every downtown in the US. In short: not good.

I don’t want to say you guys overstate things, but literally every downtown in the US? Might be time to move. I was regularly visiting San Francisco in the late 2010s when the tech bro community was really ginning this story up. Was it an ugly clash of both the promise and the failure of US economics? Yeah. Was their shit on the streets? Also yeah. Did I ever feel unsafe? Not really. Anecdata but I remain skeptical because of these Helen Lovejoy-level overstatements.

Another thing that sort of puzzles me about benchmarks is that LLMs are not deterministic and do not always complete a problem. So what are the results actually representing? The best run? The average? It is all in some ways a falsehood

That's a good point and conventionally if benchmarks aren't run as "one shot", it is denoted as "benchmark@K". Inference time scaling has historically shown improvement.

Generally though, many of these fairness complaints do go away if there is "3rd party testing". Right now, companies reporting their own benchmarks has all the problems that 3rd party testing resolves in many other industries.

I don't think it is vague in the slightest. Take the most simple examples, how many LLM's have you tested making them? There are stylistic choices pertaining to games that is well beyond a 0/1 reward. Even something as basic as breakout or flappy bird can have wildly different quality between models. Yeah, you could call this animal on a bike benchmarking, but I don't think it is. IMO the problem space occupies an interesting area where you can ignore the pass/fail and focus on the actual level of the model to do something beyond that.

I doubt the OP meant something like creating the whole tech stack for WOW.

You seem to think I was disagreeing somehow.

I was just asking what kinds of games and with which technology.

Neither is stated in the original comment, and the answer obviously isn’t “every kind with every technology”.

Your first failure was trying to get claude to do anything :)

It ain't bearing enough load, any load on this one.-

There is, in my personal opinion, a reason that reasoning is not front and center. Partially related to distillation.

From what I understand pre-training is totally irrelevant to this and as far as post training goes there will be multiple steps, for claude and codex and the like that ship with a harness, the harness is definitely included in evaluation. However, they will definitely include evaluation from a variety or even none, and settle on something that works the "best" for a release.

Disclaimer: No first hand knowledge

Tmux is good because: 1) sessions stay running if you disconnect 2) window management - you can split your screen, have an octobox a-la redzone style, and focus/unfocus etc.

This is based on developing on a remote server - but even locally, I find it invaluable. Multiple terminal windows are fine, but some times you want multiple windows. Even in a pre-ai world, you might want to run a process, see the code, edit, and maybe have htop or something like that. If you ever NEED multiple terminal windows for the same thing, tmux is really the answer.

One reason I find it hard to grasp is that when I work on Windows, it's almost always through RDP or VNC. There is never any question of a terminal (or a process launched from it) staying open between login sessions, because it never depended on any particular connection to begin with. It's just there, on the desktop, which might stay up for months before I need to connect to it again. Any terminals or other processes I launch on that system will be there until I close them intentionally, or until Microsoft decides to reboot the system without my permission.

So when I dip my toes into the Linux world, the first thing I do (or rather ask an LLM harness to do, being much less familiar with Linux than I am with Windows) is set up an RDP host. That way I can follow the same basic workflow on both OSes. I don't need to do anything special with terminal windows.

I guess if I were running truly headless systems that don't even have a VGA frame buffer, I'd be more interested in multiplexing. But so far I haven't had to do that, and would probably resist it because a persistent desktop strikes me as better in general than letting applications including terminals worry about their own lifetimes.

I have been in RDP land but I would never go back unless required (windows). While it could just be personal experience/failures, latency was not good and reliability was not good. Things are probably better as of late though I hesitate to assume software has improved... It is just much simpler to live in the terminal for dev work, especially with agents. That being said, I use MacOS and Windows daily for non-dev work. Either way, your points are valid, and whatever works for you works for you friend :)