Great job! 32-to-64-bit conversions are always fun :) One question though. If this is intended for desktop, why bother with WASM at all? Do you gain anything other than less performance?
Funny enough the customer is one of my dads friends that has been running the same shop for 20 years, wanted bigger tables and keep milking that for the foreseable future. I wanted something simple, no jit stuff, no gc complexity. Just a stack based interpreter. Yes it was LLM assisted like most stuff nowadays.
Because the same module runs in three places: inside the Electron IDE, in the shipped app, and in the test suite under jsdom, with no native build per platform. The other reason is the boundary itself: wasm exports can't re-enter, which forced the design where every side effect is yielded to the host and the VM is never on the stack while a dialog is up. That's what makes MESSAGEBOX not freeze the window. Perf isn't where these apps hurt, they're I/O and UI bound, and VFP itself was a p-code interpreter. The crate is plain Rust, the CLI runner is native, so a native build is a cargo flag away if it ever matters
No but almost all good ideas can be reduced down to a few sentences if you're good at explaining things. It's a different kind of intelligence than what's commonly called IQ but it's something like that regardless.
Sure the explanation will oversimplify a lot but then you can expand it recursively if needed, you gotta start somewhere.
Most musicians in history didn't know how to read music, yeah. Less so in the present day but just about every folk song has spread by the word of mouth, and even in pop music many genres have a strong oral tradition (jazz, blues, rock, you name it), they play by ear and not by sheet.
Well, that's not really what I mean. Obviously they had some concept of communicating music, somewhat standardized instruments, etc. Pythagoras didn't phrase his teachings using algebraic symbols either. I suppose what I mean is that
> Reading math however does not guarantee ideas or intuition.
is true, but the converse, having the mind for good mathematical ideas or intuition, almost surely means that you'll have very little trouble picking up mathematical notation (not just notation, but the reasoning and ability to write a proof, be it in natural language or some formal proof assistant language).
Like yeah, before the written language there were great storytellers who couldn't write. But now that humanity does have standardized languages, it's way more rare to come across such a person.
With /LTCG /GL I don't think it's possible to get matching (or at least it's a very tall order), the codegen is wayy too volatile for an exact match and since inlining and reg alloc work on heuristics with thresholds it really cascades. Even stuff like what order you declare your locals in or the exact frontend syntax can mess things up...
Most people are working on builds where debug information was either inadvertently or sometimes intentionally included (e.g. for beta releases sometimes debug builds with debug info would ship for sake of making things easier.) These builds usually don't have all of the normal release optimizations on. That makes it much more likely to get a match.
I haven't tried this, but I also suspect that once you have a lot of code fully matching, it might make it possible to ratchet your way up further into builds that you don't have debug information for, that may have more aggressive compilation options. I am not sure if you would manage to get /LTCG builds fully matching even with this advantage, but it's going to be the best shot at it. You're possibly 90% of the way there already.
Hey I'm not working in matching something with LTCG atm :) I'm just saying in general.
And yes if you have a pdb / an Od build then things are much easier, I was assuming arbitrary game i.e. release binaries.
The "knowledge laundering" approach you describe might help in reconstructing headers, class layouts and function names which is a godsend although I don't think it would be enough to get a match. Getting functionally equivalent code is muuuch easier (although there's the problem of "how do you verify that without running every function")
Yeah, this is probably true. I've done non-matching decompilations of modern software up to a few hundred kilobytes worth of code - it is challenging but doable. I have no idea how hard it would be to get to matching with LTCG no matter where you start from. If it was genuinely not practically possible for computational reasons I would be unsurprised.
AI is pretty powerful for decompilation, especially because you can also just have an LLM go and start reverse engineering bits of the linker and compiler if you want. (I suspect this decomp is AI assisted if the Clauded out README is any indication.) Maybe future models will be able to come up with clever and novel ways to reduce the number of possibilities and converge faster on possible matching source codes. Or maybe not; I think Astra is the best LLMs have ever been at decompilation and yet I find LLMs frustrating and prone to getting deeply stuck in local maxima in my experimentation.
Yeah IMO the next big thing will be a way of verifying equivalence while filtering out the "noise" differences in optimisation. Otherwise you can't really decide whether you got it correct or not.
Correct, this is just hacks for stuff you didn't manage to match exactly. Either that or your build environment isn't the same. Sadly, there are things which aren't really possible to reproduce in a byte-identical manner, things like exact file layout or variable declaration order, compilation order and that kinda stuff. And they might cause small but equivalent changes like different inlining/optimisation decisions, so it's really tricky to get it byte-exact.
"assign all these pointers to the correct types, a wrong guess leads to different COMDAT folding"
"get the order of local variables in this function right, otherwise the register allocation doesn't match. Oh and there's 150 local variables just in this function, good luck trying them all"
"Find out the translation unit boundaries exactly (assume there's no pdb otherwise this is trivial) and after doing so, figure out the order they were compiled in, otherwise it won't match"
"brute force the compilation flags for the project and if you're done, also bruteforce it for the CRT or any other middleware which usually came prebuilt so it doesn't match the main game"
Indeed it's a shitload of work. It's also possible and has been done. You don't have to only use global brute-force - you could also reverse engineer the compiler. The OOT/MM decomps achieved completeness without //COMPILERDIFF.
Correct me if I'm wrong (I'm not very well-versed in game decomp scenes) but aren't all those bytematched decomps from 90s or at the latest early 2000s games? They didn't have global optimisation (MSVC introduced it in VS .NET or 2003 I think and many games didn't use it until later)
So these are mostly problems with more advanced compilers yk
Correct, N64 and PS1 era projects can essentially always be strictly tackled one function at a time. There are very minor things (like string literal sharing) where some earlier function can affect later function, but the workarounds are comparatively simple.
On the other hand, I have heard even old MSVC is a nightmare. Various things are affected by hash table ordering (so the names of variables matter in some situations), stuff like order of #includes mattering, etc.
Yuuuup exactly. I guess this is masked because old 90s games are often more culturally significant so people haven't found these problems yet but yes, these are serious challenges.
TL;DR: You got promoted or you wanted to get promoted. You decide you want to make a bigger project to show ambition, advance your career and whatnot. You embark on making some grand project, disappearing from view. This often leads to you getting stuck with something, falling behind and various negative effects like getting a PIP or becoming depressed.
His prescription to avoid this is to keep checking in and show some progress on something every day so you don't lose touch.
(My analysis: I don't think this is particularly a senior engineer problem, to me it just reads like ADHD-coded problems with time and interests)
The real prescription to stopping this is to only commit to looking at the problem for a few days regardless of how much progress you’re making. And never take on open-ended tasks with no sunset date. A CTO had me take an open-ended dev project once and I learned that the lack of a time box made it really miserable.
Sure, but what are you going to do with such a binary? It is probably high distributed. For instance, i don’t think we are ever going to get any games that is not streamed.
At some point, the cost of reverse engineering a binary becomes way too cheap. Frankly speaking, assembly code might as well be source code today.
You don't have a buffer overflow problem if you read it in a memory-safe way i.e. read it in chunks and realloc when you reach the size of your allocation.
What you will have is a potential denial-of-service attack - although this one isn't particularly great because there's zero amplification (they might as well just send garbage into your firewall)
I think the DoS has pretty common amplification vectors in the form of APIs that split or otherwise copy (e.g. materializing code points for Unicode regex searching).
Additionally, OOM inside a low level routine can be a troublesome attack, since OOM handling in many applications does questionable (nee vulnerable) things when crashes occur in not-known-to-be-memory-intensive code. Sure, that’s sloppy engineering, but it’s common.
Memory allocation?? Why? Real world, you'll bump into limits based on fonts long before you'll get a buffer that's too large for the stack and worth using the memory allocator for. I can't see any reason to support more than 2^64 characters and lots of headaches from trying to beyond that. You check your buffer writes and reject the character if it's too long.
They can write any EULA they want but it's not like they will be able to enforce it in a meaningful way, to be fair. You can't legislate modding away, that's like legislating piracy away, which has famously failed. Music piracy wasn't killed by banning it, it has shrunk in importance because Spotify is fairly cheap and easy. Despite that, many people do still pirate music.
In the hypothetical GTA VI case, what can they do about it? Take just about any historical data leak, in pretty much zero cases anyone has been able to scrub all copies of it from the internet. There's plenty of lost media, the stuff no one cares about. But if enough people care about, it can't really be scrubbed away.
Sorry, I'm just being realistic. "Whenever there's a will there's a way" and so on. People have tried to legislate all sorts of various things, even stuff like "PI = 3" without much success because they weren't viable.
I said it's not possible to legislate some things away, i.e. unviable legislation. A hard ban on modding / AI reimplementation isn't viable, because you can't really keep data off the internet, evidenced by things like leaks, Anna's Archive etc. remaining online. People can always reupload the remade GTA or whatever onto noname fileshares, code forges, torrent and so on.
reply