The implementation of a language model is usually good enough for running _a single forward pass_ through the model, but to host it via an inference engine you typically need to convert a few operations. For instance the MLP can be easily split across GPUs (tensor parallelism) and MoEs also have a way of parallelizing.
Most of it is pretty standard, since not that many different layers and primitives are used in LLM architectures, but once in a while something new comes along that needs more effort. MoEs are one example, they are sparse and allow for completely different inference patterns, which takes a while to figure out.
Last year I started a blogpost series about this topic (that I hope to update some time). I start from a minimal gpt implementation by Karpathy and build the engine around it, you might like it: https://pieter.ai/blog/2025/nanogpt-inference/
yes. this is just raw implementation of the model arch as described in papers. for complete model training with back propogation we need training pipeline with optmizer and loss calculation.
I only see an edit to the readme (13h ago) and removal of the weights, so the repo is now empty.
I am willing to give them the benefit of the doubt, but we've seen this before: a model gets released that is supposedly state-of-the-art, yet seems to be a an other repackaged model without any training. Reflection 70B was the most similar example, all they now need is an api that rewrites "Claude" to "Rio".
There is currently very little evidence that morphological tokenizers help model performance [1]. For languages like German (where words get glued together) there is a bit more evidence (eg a paper I worked on [2]), but overall I start to suspect the bitter lesson is also true for tokenization.
I never understood why people want this in the first place. Sure, making this step more human explainable would be nice and possibly even fix some very particular problems for particular languages, but it directly goes against the primary objective of a tokenizer: Optimizing sequence length vs. vocabulary size. This is a pretty clear and hard optimization target and the best you can do is make sure that your tokenizer training set more closely mimics your training and ultimately your inference data. Putting english or german grammar in there by force will only degrade every other language in the tokenzier, while we already know that limiting additional languages will hurt overall model performance. And the belief that you can encode a dataset of trillions of tokens into a more efficient vocabulary than a machine is kind of weird tbh. People have also accepted since the early convnet days that the best encoding representation for images in machine learning is not a human understandable one. Same goes for audio. So why should text be any different? If you really think so, you might also wanna have a go at feature engineering images. And it's not like people haven't tried that. But they all eventually learned their lesson.
We usually build the tokenizer by optimizing for one goal (space-efficient encoding of text), then use it in a model that is trained for an entirely different goal (producing good text, "reasoning", "coding", etc). It is not immediately clear that the optimization goal for the tokenizer is actually the one that best serves the training of the llm.
That's what all these attempts boil down to. They don't presume to be able to find a more space-efficient encoding by hand, they assume that the optimization goal for the tokenizer was wrong and they can do better by adding some extra rules. And this isn't entirely without precendent, most tokenizers have a couple of "forced" tokens that were not organically discovered. Moving around how digits are grouped in the tokenizer is another point where wins have been shown.
This is where projects like nanochat are really valuable for quickly and (relatively) cheaply trying out various tweaks
>It is not immediately clear that the optimization goal for the tokenizer is actually the one that best serves the training of the llm.
Except that is exactly what research has shown. Besides, the tokenizer's training goal is literally just to encode text efficiently with fewer tokens by increasing the vocabulary, which obviously directly benefits the attention mechanism if you look at the dimensions of involved matrices. The biggest issues so far have stemmed from variances between tokenizer and LLM training sets [1] and the fact that people primarily work with character based text and not word-part based text (even though that gets muddy when you look at what is actually happening in the brain) when doing anything in writing.
If you want to make it more human-explainable, then ditch the entire tokenizer and just feed the models raw characters. Because now there is nothing to explain.
Then that means you need at least 4x the compute to achieve the same results as state of the art. Meaning that if I can train my frontier model with my normal tokenizer in 3 months, it will take you a year. When major releases across all competing providers are measured in months, there's simply no incentive to do that just to capture these fringe edge cases.
Yes, OK. But all the tutorials start with explaining how a tokenizer works. This is not necessary. And in fact makes the message of why a tokenizer is necessary not come across as well.
This is an interesting blogpost. While the general conclusion ("We need batching") is true, inference of mixture of experts (MoE) models is actually a bit more nuanced.
The main reason we want big batches is because LLM inference is not limited by the compute, but my loading every single weight out of VRAM. Just compare the number of TFLOPS of an H100 with the memory bandwidth, there's basically room for 300 FLOP per byte loaded. So that's why we want big batches: we can perform a lot of operations per parameter/weight that we load from memory. This limit is often referred to as the "roofline model".
As models become bigger, this does not scale anymore because the model weights will not fit into GPU memory anymore and you need to distribute them across GPUs or across nodes. Even with NVLink and Infiniband, these communications are slower than loading from VRAM. NVlink is still fine for tensor parallelism, but across nodes this is quite slow.
So what MoE allows is expert parallelism, where different nodes keep different experts in memory and don't need to communicate as much between nodes. This only works if there are enough nodes to keep all experts in VRAM and have enough overhead for other stuff (KV cache, other weights, etc). So naturally the possible batch size becomes quite large. And of course you want to maximize this to make sure all GPUs are actually working.
You could load different "experts" in a round-robin way on a single node and only aggregate "batches" opportunistically, when you just have multiple requests in-flight that all happen to rely on the same "expert". The difference being that instead of "batches", you would only really have queues. Of course this would come with a sizeable increase in latency, but that's acceptable for many applications (such as for "deep research" workflows)
This is very much like Erlang's actor model. The same compute can be run in parallel, or managed via queues. With Erlang's strong support for FFI and process control, I wonder if it's being used as a dispatcher for these sorts of workloads.
> As models become bigger, this does not scale anymore because the model weights will not fit into GPU memory anymore and you need to distribute them across GPUs or across nodes. Even with NVLink and Infiniband, these communications are slower than loading from VRAM. NVlink is still fine for tensor parallelism, but across nodes this is quite slow.
Inference works by computing layers and then have a very small vector that you send to the next layer as input. When a model does not fit in a single GPU, you just divide it into layers and send the vector over a fabric to the GPU holding the next layer. The transfer happens so quickly that there is a negligible amount of idle time and then the next layer can be computed. The fastest inference on the planet at Cerebras uses this technique to do 2500T/sec on Llama 4 Maverick.
Groq and Cerebras both take a big chip approach to architecture and, at least in the case of Groq, they only make economic sense under high batch loads.
There is nothing big about Groq’s chips. Their individual chips have only 230 MB RAM. Unlike Cerebras, which can load multiple layers into a single chip, grok must divide a layer across many chips.
Distributing inference per layer, instead of splitting each layer across gpus, is indeed another approach, called pipeline parallelism. However, per batch there is less compute (only 1 gpu at a time), so inference is slower. In addition, the orchestration of starting the next batch on gpu #0 while gpu #1 starts is quite tricky. For this reason, tensor parallelism as I described is way more common in LLM inference.
The idea is to replicate the weights of the network in the electronics. Somehow like our brains work? This way an analog input signal could lead to a neural network processed output signal without the digital emulation on an gpu. As this is very much simplified, the question is if this could work for modern llms?
And this is the investment case for AMD, models fit entirely in a single chassis, and side benefit: less tariffed network equipment to interconnect compute. Map/reduce instead of clustered compute.
Edit: when downvoting, please offer some insight why you disagree
This means that you can fit larger and larger models into a single node, without having to go out over the network. The memory bandwidth on AMD is also quite good.
"driver" is such a generic word. tinygrad works on mi300x. If you want to use it, you can. Negates your point.
Additionally, ROCm is a giant collection of a whole bunch of libraries. Certainly there are issues, as with any large collection of software, but the critical thing is whether or not AMD is responsive towards getting things fixed.
In the past, it was a huge issue, AMD would routinely ignore developers and bugs would never get fixed. But, after that SA article, Lisa lit a fire under Anush's butt and he's taking ownership. It is a major shift in the entire culture at the company. They are extremely responsive and getting things fixed. You can literally tweet your GH issue to him and someone will respond.
What is true a year ago isn't today. If you're paying attention like I am, and experiencing it first hand, things are changing, fast.
I have been hearing this about AMD/ATI drivers for decades. Every year, someone says that it is fixed, only for new evidence to come out that they are not. I have no reason to believe it is fixed given the history.
Here is evidence to the contrary: If ROCm actually was in good shape, tinygrad would use it instead of developing their own driver.
ROCm isn't part of AMD drivers, its a software library that helps you support legacy compute APIs and stuff in the BLAS/GEMM/LAPACK end of things.
The part of ROCm you're interested in is HIP; HIP is the part that does legacy CUDA emulation. HIP will never be complete because Nvidia keeps adding new things, documents things wrong, and also the "cool" stuff people do on Nvidia cards aren't CUDA and it is out of scope for HIP to emulate PTX (since that is strongly tied to how historical Nvidia architectures worked, and would be entirely inappropriate for AMD architectures).
The whole thing with Tinygrad's "driver" isn't a driver at all, its the infrastructure to handle card to card ccNUMA on PCI-E-based systems, which AMD does not support: if you want that, you buy into the big boy systems that have GPUs that communicate using Infinity Fabric (which it, itself, is the HyperTransport protocol over PCI-E PHY instead of over HyperTransport PHY; PCI over PCI-E has no ability to handle ccNUMA meaningfully).
Extremely few customers, AMD's or not, want to share VRAM directly over PCI-E across GPUs since most PCI-E GPU customers are single GPU. Customers that have massive multi-GPU deployments have bought into the ecosystem of their preferred vendor (ie, Nvidia's Mellanox-powered fabrics, or AMD's wall-to-wall Infinity Fabric).
That said, AMD does want to support it if they can, and Tinygrad isn't interested in waiting for an engineer at AMD to add it, so they're pushing ahead and adding it themselves.
Also, part of Tinygrad's problem is they want it available from ROCm/HIP instead of a standards compliant modern API. ROCm/HIP still has not been ported to the modern shader compiler that the AMD driver uses (ie, the one you use for OpenGL, Vulkan, and Direct family APIs), since it originally came from an unrelated engineering team that isn't part of the driver team.
The big push in AMD currently is to unify efforts so that ROCm/HIP is massively simplified and all the redundant parts are axed, so it is purely a SPIR-V code generator or similar. This would probably help projects like Tinygrad someday, but not today.
> ROCm isn't part of AMD drivers, its a software library that helps you support legacy compute APIs and stuff in the BLAS/GEMM/LAPACK end of things.
AMD says otherwise:
> AMD ROCm™ is an open software stack including drivers, development tools, and APIs that enable GPU programming from low-level kernel to end-user applications.
It loads various firmware blobs, manages part of the initialization process, manages memory, writes to registers, etcetera. These are all things a driver does.
AMD is extremely bad at communications. The driver already contains everything ROCm requires to talk to the GPU, and ROCm itself is only a SDK that contains runtimes, libraries, and compilers.
This part of TinyGrad is not a driver, however it tries to hijack the process to do part of that task. You cannot boot the system with this, and it does not replace any part of the Mesa/DRI/DRM/KMS/etc stack. It does reinitialize the hardware with a different firmware, which might be why you think this is a driver.
I consider it to be a driver, or at least part of one. Userspace drivers exist. Graphic drivers originally were entirely in userspace, until portions of them were moved into the kernel for kernel mode setting and DRM. These days, graphics drivers themselves have both kernel mode and user mode components. The shader compiler for example would be a user mode component.
They were doing this before, the difference with this is, the version of ROCm you use is locked to the driver versions that are supported, which is a very narrow range.
With this new thing, the backend API is now formalized and easier to support wider range of difference.
We have all been hearing things for decades. Things are noticeably different now. Live in the present, not in the past.
Tinygrad isn’t a driver. It is a framework. It is being developed by George however he wants. If he wants to build something that gives him more direct control over things. Fine. Others might write PTX instead if using higher level abstractions.
Fact is that tinygrad runs not only on AMD, but also Nvidia and others. You might want to reassess your beliefs because you’re reading into things and coming up with the wrong conclusions.
Under driver quality for AMD, they say “developing” and point to their git repository. If AMD had fixed the issues, they would instead say the driver quality is great and get more sales.
They can still get sales even if they are honest about the state of AMD hardware, since they sell Nvidia hardware too, while your company would risk 0 sales if you say anything other than “everything is fine”, since your business is based on leasing AMD GPUs:
Appreciate you diving more into my business. Yes, we are one of the few that publishes transparent pricing.
When we started, we got zero sales, for a long time. Nobody knew if these things performed or not. So we donated hardware and people like ChipsAndCheese started to benchmark and write blog posts.
We knew the hardware was good, but the software sucked. 16 or so months later, things have changed and sufficiently improved that now we are at capacity. My deep involvement in this business is exactly how I know what’s going on.
Yes, I have a business to run, but at the same time, I was willing to take the risk, when no-one else would, and deploy this compute. To insinuate that I have some sort of conflict of interest is unfair, especially without knowing the full story.
At this juncture, I don’t know what point you’re trying to make. We agree the software sucked. Tinygrad now runs on mi300x. Whatever George’s motivations were a year ago are no longer true today.
If you feel rocm sucks so badly, go the tinygrad route. Same if you don’t want to be tied to cuda. Choice is a good thing. At the end of the day though, this isn’t a reflection on the hardware at all.
I hope your business works out for you and I am willing to believe that AMD has improved somewhat, but I do not believe AMD has improved enough to be worth people’s time when Nvidia is an option. I have heard too many nightmares and it is going to take many people, including people who reported those nightmares, reporting improvements for me to think otherwise. It is not just George Hotz who reported issues. Eric Hartford has been quiet lately, but one of the last comments he made on his blog was not very inspiring:
> Know that you are in for rough waters. And even when you arrive - There are lots of optimizations tailored for nVidia GPUs so, even though the hardware may be just as strong spec-wise, in my experience so far, it still may take 2-3 times as long to train on equivalient AMD hardware. (though if you are a super hacker maybe you can fix it!)
> A conflict of interest (COI) is a situation in which a person or organization is involved in multiple interests, financial or otherwise, and serving one interest could involve working against another.
You run a company whose business is dependent entirely on leasing AMD GPUs. Here, you want to say that AMD’s hardware is useful for that purpose and no longer has the deluge of problems others reported last year. If it has not improved, saying such could materially negatively impact your business. This by definition is a conflict of interest.
That is quite a large conflict of interest, given that it involves your livelihood. You are incentivized to make things look better than they are, which affects your credibility when you say that things are fine after there has been ample evidence in the recent past that they have not been. In AMD’s case, poor driver quality is something that they inherited from ATI and the issues goes back decades. While it is believable that AMD has improved their drivers, I find it difficult to believe that they have improved them enough that things are fine now, given history. Viewing your words as being less credible because of these things might be unfair, but there have been plenty of people whose livelihoods depended on things working before you that outright lied about the fitness of products. They even lied when people’s lives were at risk:
You could be correct in everything you say, but I have good reason to be skeptical until there has been information from others corroborating it. Blame all of the people who were in similar positions to yours that lied in the past for my skepticism. That said, I will keep my ears open for good news from others who use AMD hardware in this space, but I have low expectations given history.
Funny to see you quoting Eric, he’s a friend and was just running on one of our systems. AMD bought credits from us and donated compute time to him as part of the big internal changes they’re pushing. That kind of thing wouldn’t have happened a year ago. And from his experience, the software has come a long way. Stuff is moving so fast, that you aren't even keeping up, but I am the one driving it forward.
And sigh, here we are again with the conflict of interest comments, as if I don’t get it. As I said, you don’t know the full story, so let me spell it out. I’m not doing this for money, status, or fame. I’m fortunate enough that I don’t need a job, this isn’t about livelihood or personal gain.
I’m doing this because I genuinely care about the future of this industry. I believe AI is as transformational as the early Internet. I’ve been online since 1991 (BBS before that), and I’ve seen how monopolies can strangle innovation. A world where one company controls all AI hardware and software is a terrible outcome. Imagine if Cisco made every router or Windows was the only OS. That’s where we’re headed with Nvidia, and I refuse to accept that.
Look at my history and who my investor is, this isn’t some VC land grab. We truly care about decentralizing and democratizing compute. Our priority is getting this previously locked up behind supercomps HPC into the hands of as many developers as possible. My cofounder and I are lifelong nerds and developers, doing this because it matters.
Right now, only two companies are truly competing in this space. You’ve fairly pointed out failures of Cerebras and Groq. AMD is the only one with a real shot at breaking the monopoly. They’re behind, yes. But they were behind in CPUs too, and look where that went. If AMD continues on the path they’re on now, they can absolutely become a viable alternative. Make no mistake, humanity needs an alternative and I'll do my best to make that a reality.
Ask Eric to consider writing a new blog post discussing the state of LLM training on AMD hardware. I would be very interested in reading what he has to say.
AMD catching up in CPUs required that they become competent at hardware development. AMD catching up in the GPGPU space would require that they become competent at software development. They have a long history of incompetence when it comes to software development. Here are a number of things Nvidia has done right contrasted with what AMD has done wrong:
* Nvidia aggressively hires talent. It is known for hiring freshly minted PhDs in areas relevant to them. I heard this firsthand from a CS professor whose specialty was in compilers who had many former students working for Nvidia. AMD is not known for aggressive hiring. Thus, they have fewer software engineers to put on tasks.
* Nvidia has a unified driver, which reduces duplication of effort, such that their software engineers can focus on improving things. AMD maintains separate drivers for each platform. AMD tried doing partial unification with vulkan, but it took too long to develop, so the Linux community developed its own driver and almost nobody uses AMD’s unified Vulkan driver on Linux. Instead of killing their effort and adopting the community driver for both Linux and Windows, they continued developing their driver that is mostly only used on Windows.
* Nvidia has a unified architecture, which further deduplicates work. AMD split their architecture into RDNA and CDNA, and thus must implement the same things for each where the two overlap. They realized their mistake and are making UDNA, but the damage is done and they are behind because of their RDNA+CDNA misadventures. It will not be until 2026 that UDNA fixes this.
* Nvidia proactively uses static analysis tools on their driver, such as coverity. This became public when Nvidia open sourced the kernel part of their Linux driver. I recall a Linux kernel developer that works on static analysis begging the amdgpu kernel driver developers to use static analysis tools on their driver, since there were many obvious issues that were being caught by static analysis tools that were going unaddressed.
There are big differences between how Nvidia and AMD do engineering that make AMD’s chances of catching up slim. That is likely to be the case until they start behaving more like Nvidia in how they do engineering. They are slowly moving in that direction, but so far, it has been too little, too late.
By the way, AMD’s software development incompetence applies to the CPU side of their business too. They had numerous USB issues on the AM4 platform due to bugs in AGESA/UEFI. There were other glitches too, such and memory incompatibilities. End users generally had to put up with it, although some AMD in conjunction with some motherboard vendors managed to fix it the issues. I had an AM4 machine that would not boot reliably with 128GB of RAM and this persisted until I replaced the motherboard with one of the last AM4 motherboards made after suffering for years. Then there was this incompetence that even affected AM5:
AMD needs to change a great deal before they have any hope of competing with Nvidia GPUs in HPC. The only thing going for them in HPC for GPUs is that they have relatively competent GPU hardware design. Everything else about their GPUs have been a disaster. I would not be surprised if Intel manages to become a major player in the GPU market before AMD manages to write good drivers. Intel, unlike AMD, has a history of competent software development. The major black mark on their history would be the initial Windows ARC drivers, but the were able to fix a remarkable number of issues in the time since their discrete GPU launch, and have fairly good drivers on Windows now. Unlike AMD, they did not have a history of incompetence, so the idea that they fixed the vast majority of issues is not hard to believe. Intel will likely continue to have good drivers after they have made competitive hardware to pair with them, provided that they have not laid off their driver developers.
I have more hope in Intel than I have in AMD and I say that despite knowing how bad Intel is at doing anything other than CPUs. No matter how bad Intel is at branching into new areas, AMD is even worse at software development. On the bright side, Intel’s GPU IP has a dual role, since it is needed for their CPU’s iGPUs, so Intel must do the one thing they almost never do when branching into new areas, which is to iterate. The cost of R&D is thus mostly handled by their iGPUs and they can continue iterating on their discrete graphics until it is a real contender in the market. I hope that they merge Gaudi into their GPU development effort, since iterating on ARC is the right way forward. I think Intel having an “AMD moment” in GPUs is less of a longshot than AMD’s recovery from the AM3 fiasco and less of a long shot than AMD becoming competent at driver development before Intel either becomes good at GPGPU or goes out of business.
Trying to find fault over UDNA is hilarious, they literally can't win with you.
My business model is to support viable alternatives. If someone else comes along and develops something that looks viable and there is customer demand for it, I'll deploy it.
You totally lost me at having more hope with Intel. I'm not seeing it. Gaudi 3 release was a nothing burger and is only recently deployed on IBM Cloud. Software is the critical component and if developers can't get access to the hardware, nobody is going to write software for it.
I fixed some autocorrect typos that were in my comment. I do not find fault with UDNA and I have no idea why you think I do. I find fault with the CDNA/RDNA split. UDNA is what AMD should have done in the first place.
As for Gaudi 3, I think it needs to be scrapped and used as an organ donor for ARC. In particular, the interconnect should reused in ARC. That would be Intel’s best chance of becoming competitive with Nvidia.
As for AMD becoming competitive with Nvidia, their incompetence at software engineering makes me skeptical. They do not have enough people. They have the people that they do have divided into to many redundant things. They do not have their people doing good software engineering practices such as static analysis. They also work the people that they do have with long hours (or so I have read), which of course is going to result in more bugs. They need a complete culture change to have any chance of catching up to Nvidia on the software side of things.
As for Intel, they have a good software engineering culture. They just need to fix the hardware side of things and I consider that to be much less of a stretch than AMD becoming good at software engineering Their recent battlematrix announcement is a step in the right direction. They just need to keep improving their GPUs and add an interconnect to fulfill the role of nvlink.
Unfortunately, AMD and ATI before it have had driver quality issues for decades; and both they and their fans have claimed that they have solved the problems every year since.
Even if they have made progress, I doubt that they have reached parity with Nvidia. I have had enough false hope from them that I am convinced that the only way that they will ever improve their drivers if they let another group write the drivers for them.
Coincidentally, Valve has been developing the Vulkan driver used by SteamOS and other Linux distributions, which is how SteamOS is so much better than Windows. If AMD could get someone else to work on improving their GPGPU support, we would likely see it become quite good too. Until then, I have very low expectations.
As for being stuck in the past, I got fed up in 2006 after 8 years of nothing but ATI graphics. I spent years hoping that the issues would be fixed after the latest update, but they never were. I had a fairly problem free experience after switching to Nvidia. When issues did occur, Nvidia fixed them within months. While enjoying the relatively problem free experience on Nvidia, I would hear people claim everything was fixed on ATI (and later AMD), only to hear waves of people complaining about issues. Then Valve got involved with the driver development for AMD graphics and made the Steam deck. I brought one and it has been fantastic. I still hear about numerous problems involving drivers AMD wrote (especially their windows drivers), but I am using drivers that were in part authored by Valve, and Valve fixed the issues AMD was incapable of fixing themselves.
You claim that things are fine for HPC on AMD graphics hardware, but I have reason to be skeptical given that numerous people have reported severe problems just last year with no follow up that the various headaches have been fixed.
Also, you have repeatedly claimed that tinygrad’s software is not a driver, yet I see a userland driver here:
As I have said elsewhere: It loads various firmware blobs, manages part of the initialization process, manages memory, writes to registers, etcetera. These are all things a driver does.
I am going to listen to others and my own eyes over you on these matters.
So the MI300x has 8 different memory domains, and although you can treat it as one flat memory space, if you want to reach their advertised peak memory bandwidth you have to work with it like an 8-socket board.
The mains frequency is literally how fast the generators in power plants are turning. If the load on the grid increases, those generators slow down slightly and more natural gas/coal/heat needs to be added to increase the frequency again. This whole process is quite complicated as not every plant can react in the same time. Some plants are always at 100% capacity, while others are dedicated to governing the frequency.
So there are small fluctuations, often between 0.2 Hz around the base frequency, but the average is very close to the theoretical 50/60 Hz. And for an alarm clock that is good enough.
I have both DO and Hetzner VMs and I find them comparable, with Hetzner being a bit cheaper. If I look at the logs and fail2ban, it looks like DO does a bit more abusive traffic filtering, but that is basically the only difference.
However, the DO docs are on a different level and high quality. But those are also accessible if you are not a customer.
Yeah I just tried it with numbers I knew a bit and it seems totally made up. The generated chart showed a linear downwards trend while in reality there isn't one and the numbers seem way off.
Our university recently recommended the same thing, and I think this is a very bad idea for two reasons.
1. Not every sequence of words deserves to be cited. GPT-3 and ChatGPT are often confidently wrong about facts, why would you want to add a citation to this? When writing a paper, this needs to be fact-checked anyways, so why not add the original (actual) source?
2. It also breaks the citation graph. Imagine all papers now point to a catch-all reference from OpenAI (2023). Adding a citation is about saying where you got certain information from and this current format doesn't give enough to do that, it just points to the catch-all. With any other citation you can either look up the paper or—in the rare case of personal communication in a citation—ask the cited source directly. You can't ask chatGPT "hey, why did you say this in paper X" and expect a meaningful answer.
(Given that this is in my country and in my field of interest that's a pretty good indication of how big of a miss this is, so thank you for the correction.)
As a Belgian, I was looking in horror at that recipe. Why would people put sugar on their fries?!
So I'm glad you like the Belgian fries. What I've always learned to be key is that we fry our cut potatoes twice: once at 140°C, wait for them to cool down, and once at 175-180°C. Then you add salt, that's it.
And the mayo: I don't think many people like it like that when they eat fries at home, but it's easy ¯\_(ツ)_/¯
FWIW, they're brined in a solution containing sugar, it's not like there's sugar sprinkled on them. Very little of it makes it into the final product and in my experience brined fries aren't perceptibly sweet in an obvious way. It assists with browning and a crunchy exterior.
Some years ago a different “MacDonalds french fries” recipe was posted, I think it did the rounds on HN, and it was much more like the recipe you describe: fried twice and cooled between, no sugar, add salt.
There are some other variations, like blanching the potatoes with vinegar, or freezing the potatoes overnight between frying. The idea is to make them crispy on the outside and soft on the inside.
There is something very weird with mayo from a Belgian "frituur". I love it, I usually order mine with mayo on top, wrap them with paper and let it soak in on the drive home. Makes the fries delicous. And I order an extra packet of mayo on the side because they usually don't put enough on top.
But, in any other situation if mayo is involved I'm grossed out to the point of puking. Even when I just smell the mayo (from the same brand) someone is putting on their plate of vegetables or whatever.
Yes, you remove them completely from the oil and let them rest a bit. Mostly the fries are baked in 'fat' rather than oil though. Besides this, the type of potato that is used is pretty important too, as its consistency will impact a great deal too. The potatoes we use at home are called 'bintjes', they are a bit 'flowery'.
Yes, when double frying you remove fries from the oil and allow moisture near the surface to evaporate so it becomes tacky. For oil any common oil works but saturated fats get fries crispier. Even oven roasted potatoes in ghee and can almost get a french fry crunch. Duck fat is next level. Obviously much slower than frying.
As an European transplant I confirm that. And American sweets are absolutely loaded with sugar, it overwhelms all other flavors and tastes. My naive pet theory is that in order to cover up the use of lower quality ingredients or substitutes they’re flooded with sugar. That also creates an addictive effect so sales jump up
It's also cheap in the US (in the form of HFCS) and smashes that dopamine receptor with a fucking wrecking ball.
Europeans would absolutely eat sugar in the same volumes if it were as cheap there as it is in America. As with most of the cultural differences between the regions, a lot of it comes down to economics: if gas were cheap in Europe, they'd also drive big SUVs; if land were as cheap in Europe, they'd have McMansions; etc.
Sugar (in the form of sugar) is cheaper in Europe than sugar or HFCS in the USA.
£0.65/kg in Tesco in the UK, 7.5kr/kg ($1.20) in Føtex in Denmark, but $1.80/kg at Walmart. Corn syrup at Walmart is more expensive, $2 a pint, but presumably cheaper in bulk packaging. (Last time I was in the USA, I saw an entire freight train of tank cars full of HFCS.)
(You'll also note that plenty of rich Europeans prefer large "normal" cars, like Audis, BMWs, Mercedes etc, and a nice apartment in the middle of a city.)
That's super reductionist. Balancing salty and umami with sweet is so common that you should be surprised that it took so long to realize that adding it directly to the fries makes sense.
BBQ sauce is full of sugar. Ketchup too. People dip their fries in ice cream. Chocolate dipped pretzels work on the same principle. As does candied bacon and apple grilled cheese. Your Thanksgiving Honeybaked Ham is glazed in a sweet sauce. Turkey and cranberry sauce are a match made in heaven. Bacon and maple syrup were practically made for each other.
He's not incorrect about there being sugar in many things in the US, though. I've had to start reading labels a lot more after moving to the US, since some of the products I've used at home actually do have sugar added in the American version.
Some of the local foods in the US also have off the charts levels of sweetness, such as southern sweet tea and sweet potato casserole with marshmallows. Bread and pizzas also tend to be on the sweeter side in the US.
Those sauces are just different, though similarly named. The calories, sodium and fibre values are different, one contains cheese where the other doesn't.
Sugar is not uncommon in commercial pasta sauce, even in Canada.
> That's super reductionist... [list of American foods (or foods that aren't american but are extremely popular there, as in ketchup) with sugar added to it]
You didn't do much to dispell the idea that this is an American thing.
Maybe salted caramel pralines. But sour and sweet is a common flavour trope in much of the world. e.g. lemon + sugar e.g. in cannoli; tamarind and cane sugar in SE Asia and Mexican recipes; Chinotto.
Prosciutto e melone, brie en croute, fruit and cheese/cured meat in general, duck a l'orange, pad thai. Sweet + salty/savoury foods are definitely not uncommon throughout the world.
Sure, you can find this flavour combination anywhere, but I don't know of another country that uses sugar to the extent America does. I ordered a fried breakfast in an American hotel and they put icing sugar on the sausages! It's bizarre.
Did the breakfast come with pancakes or waffles? Maybe the sugar was intended for those but some missed. I don't doubt there are people who enjoy sugared sausages, but for a restaurant to sugar their sausages unasked is something I'd expect most Americans to find odd as well.
Most of it is pretty standard, since not that many different layers and primitives are used in LLM architectures, but once in a while something new comes along that needs more effort. MoEs are one example, they are sparse and allow for completely different inference patterns, which takes a while to figure out.
Last year I started a blogpost series about this topic (that I hope to update some time). I start from a minimal gpt implementation by Karpathy and build the engine around it, you might like it: https://pieter.ai/blog/2025/nanogpt-inference/
reply