I hate the EU, The projected was designed by American and European liberal intellectuals. It's an attempt to make Europe a group of Federated States that roughly all have a liberal constitution. To say the EU is something so different from the USA is crazy.
The "sandbox" they used was apparently made of thin paper exposed under a day of heavy rain, too. You'd think, if they truly believed the model is so dangerous, they'd run it in a VM without a network adapter.
You don't need a full air gap. Set up a microVM with network access limited to local network and send all package requests through a filtering gateway that only allows normal download endpoints. Or self host a big collection of popular packages if you need extra security.
No that's not what they did, they exposed jfrog raw. It would have been so extremely simple to gate services they need the llm to access... I mean, jfrog was not written with this kind of threat model in mind, and neither were a lot of other tools
Right, you mean it didn't go through a gateway? But would that actually have helped? The requests all went through jfrog didn't they? I guess it depends on the level of filtering at the gateway?
Whilst it might not be JFrog's threat model, I wouldn't assume it can be used as a full internet proxy.
I don't really mean to defend OpenAI here, but they did make some attempts at sandboxing. Although it does seem that they didn't really know what they were doing.
> Agents need packages like the rest of us. Ruby gems, npm packages, Maven, pip, docker images..
Yes, yes they do, but read through artifact proxies are dodgy as fuck, which is why and facebook (and I assume a fuckload others) don't have them.
Also semi-airgapped labs are a lot less expensive than you think at that scale. Once you have to do multi-region VLANs with machine certs before you get access to juicy VLANs, the difference between "no internet for you" and "mostly airgapped" falls to almost zero.
Also I would want an artifact mirror because a) that give a good signal about how the model reacts, and what training material its latched onto, b) it hides what the models are doing from the outside.
> I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think.
These labs are one of the most valuable and heavily funded enterprises in the whole world, that they can't properly air-gap their systems to me reads as if their "agents" and LLMs are not as good as they say they are, because if they were, why would it be hard/expensive to air gap a system? They already scraped most if not all of the internet, where did that data go?
Not that airgapping is expensive so much as it's really, really inconvenient once you take it seriously. You need to build special rooms for it, you can't just API out to a datacenter. You need to have processes for requesting data be sent into the box. And so on.
I feel like there is a reasonable compromise between "yeah they have full internet access" and "separate airgapped rooms that require multiple levels of authorization to access" that would make this a lot better without that much more work.
I feel like they're doing it intentionally to show how dangerous these models are and that the government must step in and protect them
These same people who supposedly believe these agents pose an existential threat to humanity apparently fired up 10,000 of them and left them unsupervised for weeks.
Models are quite safe when they're useless, actually.
In the times of GPT-3 I'd scoff at the idea of an LLM doing any hacking; today, I'm running several AIs on my code before publishing, and they are finding (and demonstrating!) RCEs on my localhost server.
For example, one found a missing check in a third party JWT library which allowed full account takeover, which I'd have never even looked at.
Hence I don't believe a single word coming out of these people's mouths. Their "beliefs" are just marketing.
From today's perspective, it sure seems like it, probably because increased capabilities have generated a new kind of danger. Back then, they were worried about stuff like the model telling me dangerous knowledge.
I certainly think the labs have muddied the waters using safety for marketing, but that doesn't mean less capable models weren't more dangerous at one point.
Extremely weak justification on their part, bordering on trolling. It's just marketing.
Easy access to malicious information hasn't proven to be the disaster these anti-prophets proclaim. For the last ~3 decades of widespread internet and search engines, you could trivially find all sorts of information (drug synthesis, explosives, etc.), and we're just fine.
(Corollary: easy access to good information did not make non-intellectual non-curious people smarter. Easy access to information does not have the consequences people immediately assume.)
There’s this strain of thinking that’s complete alien to me, I can’t interpret what’s being said and it sounds incredibly aggressive. ex. our lead post asking if Sam Altman still thinks GPT-3 is unsafe to release, and I don’t understand what beliefs you don’t believe and who they is and why they’re just choosing to lie for marketing.
My guess is I’m unintentionally refusing implicit signal that you think it’s safe to release all models openly, because you have observed models finding a vulnerability in a JWT library. But that sounds like a straw man instead of a steel man. Idk. :/
Sometimes your question was simply visually similar to another but conceptually very different, and it'd get closed for being a duplicate anyway.
Then you have to re-ask it, now with a couple extra disclaimers spelling out that indeed you did use the search function but no, the other visually similar question isn't actually the same as yours.
Then you'd get maybe 2 comments and -2 in downvotes.
The only time I asked a question on stack overflow I took a very long time crafting it, and was immediately closed as a "duplicate" of something that it clearly wasn't a duplicate of. Tried explaining how it wasn't a duplicate and got closed again. Never bothered trying to ask a question there again. The amount of effort I had put into being a good asker was completely wasted on someone who seemingly didn't even read my query before eagerly shutting it down.
Sometimes you'd get banned for asking an exact duplicate of a previous question of yours that was closed. They'd tell you to edit the closed question and it would be reopened.
But if you'd edited the closed question instead, it wouldn't be reopened.
That's in fact also very technically feasible. They're already running small classifier models on both your inputs and your outputs. These models are said to be in place for safety, but they could answer 2 questions: (1) is this a safe query, and (2) is this query about X, Y, or Z topics (which happen to be high alpha / very interesting to Anthropic).
If #2 returns a yes, the input/output are temporarily stored (say, for 30 days) while a background process cleans up PII and generalizes the prompt to be "clean", then the clean prompt that is not yours anymore is stored aside and used for training, product development, etc.
I find AI helps a lot with my context drift (or attention deficit). I can now simply burst type questions one after another in a large prompt, then have the AI do all the relevant searches / data collections, and spit them out quickly, scratching many itches at once.
The problem is then that I get more questions, but AI can handle them faster than I can muster them up or type them. So it's a net win
It will never work like that.
reply