This reminds of Magnus Carlsen saying that if he were to cheat, all he would need would be a signal to spend more time on the current move.

I think using a cotton bag 173 times to beat plastic bags and a mug 1,000 times to beat disposable cups sounds completely reasonable.

Can you use the cotton bag 173 times? Cotton wears out over time. They also didn't ask about the cost of washing the cotton bag, which really should be done once in a while since they collect dirt and food particles - this will increase the times you need to use the bag. (it isn't clear if it lasts longer with or without washing - dirt is often sharp enough to wear the bag, but washing also puts wear on them)

Similar for the cup - I've dropped and broke several ceramic cups in my life - and I don't drink coffee/tea. Those cups will never get another use since the shards are scrap.

> Can you use the cotton bag 173 times? Cotton wears out over time.

Yeah, comfortably. Shopping bags don't actually get much "wear" in each active use period. The main issue will be the handles but good stitching will solve that.

And it'll go in the washing machine with your clothes (though you might want to tie the handles together).

I have a Lidl bag that is already into the sixties of usage (it has been my food shopping bag for more than a year).

I have other two reusable bags made from recycled polyester (so recycled plastic bottles and plastic bags) that have seen probably 1800 uses each, maybe more, in their lifetime.

This material is probably a better choice for reusable shopping bags anyway; it can be made from plastic that can't be reused for food grade, it's incredibly tough, it folds down small etc.

    > Can you use the cotton bag 173 times?

I can confirm: Easily. While I don't like the feel as much (the scrunching plastic sound is awful compared to natural fibers), PET bottles that are recycled to make reusable bags are incredibly tough. I had one that I used for 15 years(!) for multiple shops per week.

    > a mug 1,000 times to beat disposable cups sounds completely reasonable.

Small nitpick here: A mug needs to be cleaned and washed after each use. (Certainly in commercial settings.) However, a reusable grocery bag probably gets washed at max: once per month. Still: I love it when a bar goes out of the way to use mugs/glasses instead of disposal cups!

thats assuming the plastic bag is used once, but i've always reused them, sometimes had the same bags for 10 years

however i have never re-used a paper bag because they are awfully loud and cannot be compressed. it is the worst kind of bag.

Turning a plastic grocery bag into a small bathroom trashcan liner is IMO the perfect reuse. it displaces a thicker plastic bag and ensures the grocery bag goes into a landfill.

I reused all my grocery store plastic bags for disposing of cat litter and other trash. When those bags went away I had to start buying new plastic bags for the first time.

That’s funny because when my locality switched from plastic to paper bags I was annoyed at having to buy trash bags for the bathroom for the first time but found the paper bags were excellent for litter cleanup since they stand up on their own and are easier to scoop into than a smaller fluttery plastic bag.

My grocery store now uses slightly thicker plastic bags, and I've found that you con reuse them dozens of times; I've had the same bags in my car for over a year.

Unfortunately, when they deliver, they use brand-new bags, and they don't accept them back for reuse, so a lot of them get thrown away after a single use.

I have a reusable cotton bag that has been in use for 30 years. That's an extreme example, perhaps, but all the same

I have been using the same mug for 50 years.

Absolutely.

Nice, I've always wanted my desktop PC to be coupled to the keyboard.

For the first 20 years or so of my career, mine was. The Apple II, the TRS-80 model 3 (even had an internal screen!) were super chunky blocks of computing.

This guy has an interesting publication history, programming books and what looks like evolutionary biology from a creationist perspective.

I don't think performance has got much to do with tinkering. IMO the real benefit of Zig is you get the flexibility of C with the ergonomics of a modern language.

I like Python as a tool language, and I am very impressed by projects like Micropython, but you always eventually run into a wall. I.e. you are never going to write a compute shader in python, but I assume someone is going to try.

I think the programmer should meet the hardware in the middle, and Python has a few too many layers of indirection to do this well.

> I don't think performance has got much to do with tinkering.

Yes, in general, but also there are cases when you realize you can, idk, parse a CSV file in 0.2 seconds instead of 200 seconds. That kind of improvement unlocks a new level of tinkering.

I found your previous comment chain on the matter, in which you conflated Catholicism and Conservatism, the second of which I'm sure you have your own personal definition, as there is no Conservative church that decides on doctrine.

Catholic social teaching, which this encyclical is grounded on, has it's roots in Rerum novarum (1891) by Leo XIII (the Pope's namesake) and it dealt with the changing conditions of people due to the industrial revolution. We are potentially in the midst of another revolution (I suspect it will be less significant the the IR), so it is prudent of the church to develop a house view.

I mean, the commenter you're responding to has assumed the position of sedevacantism, so I don't think you're going to be able to justify Vatican 2 to them from first principles. :)

A lot of this thinking still predates the Vatican 2 significantly, I think this person has made their mind up and is working backwards.

I'm slightly baffled it existed in the first place, considering they also used the small s, and it looks almost exactly like the f. To be fair, I am equally confused that so many modern typefaces don't distinguish I and l.

I ſuſpect the uſers of ye olde ſ were/are all liſping.

While less typographically interesting, this is a much clearer treatment of Elements.

Surprisingly, it hasn't seem much discussion here AFAICT.

I've only used Zulip when checking out the Lean Zulip a few years ago, and I thought it was an infinitely better interface than Discord for serious discussion, and also much easier for lurkers to find information. I wish more projects adopted it.

I just gave Zulip a try with a team of 3. I loved the UI because I spent years in pain finding things in big Slack and Mattermost installations (big = 30 users, nowhere near enterprise-level). For my two junior colleagues, instead, Zulip was too complicated. We evventually switched over to Discord.

I highly recommend Zulip to anybody who faces the problem that the concept of threads and channels is not a good fit to their mental model of tasks and groups in teams.

How is any chat app complicated with 3 people?

Well yes, I've used most of the main players. What exactly isn't user friendly about it? You can't find the conversation?

My only gripe is that on my phone sometimes it takes like 30 seconds to load, which doesn't seem to happen for almost anything else

There's new long-lived connection support in Zulip 12.0 that will enable the mobile app to do a lot better for startup in organizations with multiple 10ks of users.

I think it's expected to be enabled in the mobile apps in the next couple weeks.

Whenever somebody calls LLMs "non-deterministic", assume they meant "chaotic", in the informal sense of being a system where small changes of input can cause large changes to output, and the only way to find out if it will happen is by running the full calculation.

For many applications, this is equally troublesome as true non-determinism.

I don't think LLMs are that chaotic, you can replace words in an input at get a similar answer, and they are very good at dealing with typos.

They are definitely not interpretable, I was reading some stuff from mechanistic interpretability researchers saying they've given up trying to build a bottom up model of how they work.

> I don't think LLMs are that chaotic, you can replace words in an input at get a similar answer, and they are very good at dealing with typos.

Compare "You are a helpful assistant. Your task is to <100 lines of task description> <example problem>"

with

"you are a helpless assistant. Your task is to <100 lines of task description> <example problem>"

I've changed 3 or 4 CHARACTERS ("ful" to "less") out of a (by construction) 1000+ character prompt.

and the outputs are not at all similar.

Just realized I've never tried the "you are a helpless ass" prompt. Again a very minor change in wording, just dropping a few letters. The helpless assistant at least output text apologizing for being so bad at the task.

Sure. What did you expect? You changed the semantic of your prompt to the complete opposite. Of course it will attempt to make sense of it to its ability, and deliver what you requested. The input isn't formally specified, that's inherent for the domain, not the model or a human. GP, on the other hand, is talking about semantically negligible differences like typos.

That's not really true. If you turn a few knobs you can make them deterministic. Namely setting temperature to zero, and turning off all history. But none of the cloud providers do this. Because it's not a product as far as they are concerned. So in practice - not so much.

Can someone explain why this is? Do LLMs somehow contain a true random number generator? Why wouldn't they produce the same outputs given the same inputs (even temperature)?

edit: I'm not talking about an LLM as accessed through a provider. I'm just talking about using a model directly. Why wouldn't that be deterministic?

The model outputs a probability distribution for the next token, given the sequence of all previous tokens in the context window. It’s just a list of floats in the same order as the list of tokens that the tokenizer uses.

After that, a piece of software that is NOT the LLM chooses the next token. This is called the sampler. There are different sampling parameters and strategies available, but if you want repeatable* outputs, just take the token with the highest probability number.

* Perfect determinism in this sense is difficult to achieve because GPU calculations naturally have a minor bit of nondeterminism. But you can get very close.

I'm not so sold the LLM is an LLM without a sampler but it's not worth quibbling over. It's part of the statistical model anyways.

the llm is the trained part, the rest is the handwritten part. The sampler is handwritten, not learned.

Believe it or not in statistics and machine learning the hard coded parts of a model that impact the results are considered part of the model. But I understand that now days we don't care about these things because ai goes brrr.

There are A LOT of misconceptions about llms, biggest one is they are not deterministic. And they are 100% deterministic and temperature has nothing to do with it. You WILL get exactly same result every single time (at ANY temperature) as long as you use same sampling parameters and server config parameters. What causes variance in LLM's is server parameters like batch processing and caching among a few other things possibly. the batching being responsible for most of the issues. The reason that flag is used is because large providers serve multiple customers per one gpu, and breaking up the vram is tricky and causes drift. If you start llama.cpp for example with only one person per slot batching off, you will always get same results every time even at temperature 1.2 or whatever other parameters because you are using one gpu per inferance call so no fucky buseness there. Reason most people are unaware of this is because most people have experience only with api instead of working with the actual inferance enjine itself so this godd damned myth keeps spreading. my vide for referance here where you can download and try for yourself. https://www.youtube.com/watch?v=EyE5BrUut2o

Thanks so much for this! I still haven't got around to building my own language model yet, so I'm a bit fuzzy on the details, but if I imagined a thought experiment where I did all the math by hand on paper, I just couldn't see how I would end up with a different output each time given the same inputs. Finding out that the variance other people are seeing comes from the server/hardware stuff clears that up.

This is a surprisingly annoying question to Google. A lot of articles give the reason that softmax returns a probability distribution, as if the presence of the word "probability" means the tokens will be different every time.

An LLM model itself -- that is, the weights and the mathematical functions linking them -- does not tell you exactly how to train from data, nor how to generate an output. Instead, it describes a function providing relative likelihood(output | input).

Deciding how to pick a particular output given that likelihood function is left as an exercise for the user, which we call inference.

One obvious choice is to keep picking the highest likelihood token, feed it into the model, and get another -- on repeat. This is what most algorithms call "temperature=0". But doing this for token after token can lead boring output, or steer you into pathological low-probability sequences like a set of endless repeats.

So, the current SOTA is to intentionally introduce a random factor (temperature>0) to the sampling process -- along with other hacks, like explicit suppression of repeats.

Yea sure. So temperature is baked into these LLM models and when it isn't zero it increases the probability of taking a different path to decode the tokens. Whether it's at a provider or downloaded on your own machine.

Technically even when the temperature is 0 it's not deterministic but it's more likely to be... You can have ties in probabilities for generating the next words. And floating point noise is real.

All these models are doing is guesstimating the next token to say.

> Namely setting temperature to zero, and turning off all history

That's not nearly enough, though. The multi-node/GPU inference and specifically batching (and ordering in batching) have non-deterministic consequences for the current LLM services.

True but for small models it's pretty close. See my comment below about other cases leading to nondeterminism.

Eh, conceptually true, but in practice, it is rather hard to get any decent performance out of a GPU and still produce a deterministic answer.

And in any case, setting the temperature to zero will not produce a useful result, unless you don't mind your LLM constantly running into infinite loops.

Yes theres a good thinking machines lab blog about this

You're being downvoted, but you're right. Determinism is a different concept and doesn't characterise LLMs well. You can have deterministic random number generators for example.