> This isn't the death of open source. It's the death of the global village were everybody can freely roam and it's easy to interact. It's the resurrection of small, social, trusted communities. I hope this spreads to all of the internet.
This is definitely a microcosm of what's happening to the entire Internet.
I did this exact same thing for porting a compiler from one language to another with Codex. I run tests at every step, and verified that bytecode output was byte-for-byte identical. I was very impressed at the results, and this is coming from someone who's always pointing out issues with AI programming.
You can run an LLM locally (and distributed compile systems, where the compiler runs in the cloud, are a thing, too) so that doesn't really produce a distinction between the two.
Likewise, many optimization techniques involve some randomness, whether it's approximating an NP-thorny subproblem, or using PGO guided by statistical sampling. People might disable those in pursuit of reproducible builds, but no one would claim that enabling those features makes GCC or LLVM no longer a compiler. So nondeterminism isn't really the distinguishing factor either.
My favorite benchmark for LLMs and agents is to have it port a medium-complexity library to another programming language. If it can do that well, it's pretty capable of doing real tasks. So far, I always have to spend a lot of time fixing errors. There are also often deep issues that aren't obvious until you start using it.
Comments on here often criticise ports as easy for LLMs to do because there's a lot of training and tests are all there, which is not as complex as real word tasks
This is definitely a microcosm of what's happening to the entire Internet.
reply