Compaction is painful if you run just one local LLM, the best way to avoid it is to keep context as small as possible.
One trick I find useful is to have one model with two KV caches running and while first cache has produced tokens, second cache immediately summarizes them during input tokens are being generated (tools time), then harness switches to the second KV cache which takes newly produced input tokens while KV in first cache is getting replaced with compacted summary tokens. This is a kind of ping pong, so we trade more space for less time. Still experimenting but it looks it works, and nice bonus it improves GPU utilization. Btw I have my own harness and model serving code, but it can be easily implemented in any other harness and model server.
I tend to ask a line of questions to the LLM as first step to a point the context is enough for me. Then i ask for the work i need but then get back to previous context using /tree.
It's like asking the questions needed to get the answer you need from the LLM, can be either an answer for a plan/todo or a task to code/change something.
But so is German industry - every day in the news either insolvency or workforce reduction and moving production to Eastern Europe - main reason often stated "expensive Energy". I wonder what are the stats for industrial production in 2025 or total energy consumption in Germany?
Discussion about nuclear reactors aside, energy in germany became expensive when they cut off russian gas and they had no good alternative. There was no other source of cheap fossil fuels for electricity production. If anything, renewables are promoted as a solution to that energy problem.
The discussion about fossil fuels vs renewables is a bit different within countries which produce fossil fuels than within those that don't.
Gas became expensive, energy for private houeholds became expensive, despite that electric energy for industries is currently on the same level as 2020. Even though there was a short spike in 2022.
See: https://www.smard.de/page/en/topic-article/5892/216044/indus...
Political decisions that lead to that point and geopolitical situation. The russians cut the gas on their terms because it would have happened anyway and they were not stupid. Then somebody else blew the pipe up and the whole discussion conveniently ended. The divestment from russian economy and blocking imports was already established, but germany and others could not phase out the gas that fast.
Still, when did Germany cut off Russian gas back then? I don't think there is particular complexity in the sequence of events back then. Germany back then was not seeking an end to supplies then and there, if I recall correctly.
Russia did not completely cut gas exports to Europa, but reduced in substantially, but this was a reaction to sanctions and the decision in Europa to reduce imports from Russia (but more slowly). In fact, Russia still exports to countries in Europa (to be ended in 2027), but Germany stopped gas imports from Russia in September 2022. So the order of things was that Germany decided to reduce imports from Russia, Russian then gradually reduced and then stopped exporting via pipelines to Germany, and Germany then started to import LNT gas as a replacement but - in contrast to others - not from Russia.
It doesn't matter what they wanted to do. What matters is what happened. As a result of their actions they got cut off. If Germany had invaded Ukraine (via Poland?) the gas might still be on. Of course we don't know that because other countries would likely take offense to that action and I'm not willing to predict how the world would change if they had. (I can't even imagine change would cause them to be willing to invade in the first place, much less how everyone else would respond.)
Politics often are about the unintended consequences. Often they are also unforeseen.
Germany joining the Russian invasion was never anything even remotely in the option space at all - utterly inconceivable in 2022. (Not aware of Russia seeking any such support there either.)
Ukraine would have blown up the pipeline anyway. And the land pipelines would have been cut off to Germany by the Eastern Europeans. So no, the gas would not still be flowing with a high probability. That's so absurd.
If Germany had gone to war Ukraine would be split between Germany and Russia now and the pipeline would be running. It is absurd to think there was any possibility of Germany going to war on Russia's side, but the result of them doing so is far easier to predict.
Of course if they had done that other countries would likely object and so we can go on and on with what could have been (for starters the propaganda would be all different). There are no easy answers and we can never know what would have happened.
Probably not directly invading, but there was a strong possibility of Germany supporting the Russian invasion, just as they're doing with the invasion of Iran.
I think what grandparent is very very fucking poorly trying to say is, Germany/EU could've accepted the "annexation" of Ukraine like it did Crimea in 2014. "Bad Putin! But what can we do."
Germany is in no position to go against the USA's geopolitical interests. Sure, they do trade shenanigans sometimes, like with Russian gas, but the relationship with Russia was pretty good before Putin decided to hold power endlessly - he even gave a speech (in German) in the parliament. But in the end of the day, those times are over; only extremist parties are pro-Russian, and Germany still has the 2nd largest US Army presence, including 15-20 US nuclear bombs parked here. It's unthinkable that any government here would go against "Western" interests and support Russia.
But which USA are you talking about? Currently it is ruled by such an extremist party where some members declared love for Putin (and others are cowardly just "following orders" because they like their power and paycheck).
The former Warsaw Pact countries are aware capitulating to Putin would mean they're next, so they're vocal against EU surrender. Further, if East Europe gets threatened, West Europe knows it'll have a mother of a refugee crisis - even if Ukraine falls they'll have one.
Anyway, that's all to say currently USA's geopolitical interest seems to be "Trump Tower Moscow", and Germany is not going to go along with that.
Markets are now committed than a single pipeline. Germany was importing most gas through other channels - pipelines through Eastern and Southern Europe and shipping - as well as of course plenty of oil. While the NortStream cut had a significant short term impact it took much more effort to wean the country off all the other routes.
That is a half truth. A turbine was broken, repaired in Canada, which required a lot of bureaucracy. Then the turbine was shipped to Germany. Then the Russians started their own bureaucracy and came up with excuses why the turbine could not be installed.
During that time Nord Stream was blown up.
The point is moot, since Putin has offered multiple times to open the remaining intact pipe:
And the broken pipes can be repaired, as was determined in the LLoyds lawsuit recently.
I'm not saying that reopening would be reliable, but it would be one diversification away from US and Qatar dependencies. The US just cut the EU off from Qatar gas.
If you take a look at ng prices you will see that the prices went up more than half a year before Russias invasion of Ukraine: https://www.macrotrends.net/3235/eu-natural-gas-prices The reason for this was because Gazprom stopped supplying the spot market and also bought storage capacity and kept it empty. All of that only made sense after the invasion to better blackmail Europe. It was their bad luck that China allowed them to attack only after the Olympics which took most of the force out of that blow. Anyway there was no sane way to continue this business relationship.
And then Russia refused to install the turbine, proving it was a ploy all along. Like the maintenance in the first place. Which is why they acquired gas storage in Germany, then ran it low timed with their illegal war of aggression. This is a theme of course; Russia invented another gas crisis every few years with Ukraine. It should have been a warning sign when they settled them by taking payment in the form of Soviet-era cruise missiles.
The gas was never cheap, either; apart from the money spent now to restrain the military buildup and invasion it financed, even at the height of supplies it was much higher price than China is paying, a price sustained by Russia using military force and more to restrain other (Asian) countries from supplying to Europe.
(This was obvious to everyone, right? A single majority supplier nation state charging a cheap price out of the kindness of their heart? No, of course not; neither cheap nor kind. It was a monopoly, built on blood.)
They installed it and then they took the pipeline down for planned maintenance.
it's a little rich to accuse them of playing games with gas after the stunt Canada pulled and most especially after a western partner subsequently blew up the pipeline and blamed it on Russia.
Clearly certain western allies decided that Germany couldnt be trusted to make its own decisions about where Germany gets its energy.
If you want to see their approach to maintenance while prices and needs are high, watch them put oil refineries back in operation days after drone hits.
They reopened the pipeline after the planned maintenance. Then they said there was another issue and suspended supply indefinitely, and then the pipeline got destroyed.
I don't think the turbine that was serviced in Canada was causal there. The turbine thing was earlier, as far as I recall it. Gazprom announced its decision in early Sep 2022, btw.
> Discussion about nuclear reactors aside, energy in germany became expensive when they cut off russian gas and they had no good alternative.
Why aside? Germany had to burn gas and coal to compensate for the nuclear reactors they shut down, (and to import electricity from France which didn't shut down the reactors). Germany could've weathered the transition much better, and suffered without Russian gas much less had they not shut down their reactors.
You make it sound like there is a causality here. I would be careful. German GDP is declining for other reasons, not for their commitment to renwables. Also, by all measures, living in Germany still means you get one of the most comfortable, secure and free lives on earth. I wouldn't give up on Germany so fast. They still know how to work and often work, even if there is no reason to work (i.e looking at their current regulations, which rather motivate you to _not_ work).
Arguably Sweden (and possibly the other Nordic countries too) are even better places to live (or at least equally good). Well, as long as you can stand the climate and the lack of sun in the winter.
We have right to roam in nature here, which is quite rare across the world (and even compared to the few other countries that have something similar, the Swedish variant is stronger). I wouldn't trade that for anything.
I don't think the comment above meant to say there's nothing better than Germany. Instead the claim was that 1) Germany's decline is not due to renewables and 2) despite the decline it's still a pretty good place to live.
>They still know how to work and often work, even if there is no reason to work (i.e looking at their current regulations, which rather motivate you to _not_ work).
Ten weeks more, if you average over workers. You get a smaller difference if you average over working-age people. Germans are more likely to choose a part-time job voluntarily, while Americans are more likely to be completely outside the workforce.
More often that not I think this is a cheap excuse. When you look at struggling companies (take the car industry for example), I think the issue is more about missing innovation and building things that people don't want or can't afford anymore in Germany.
Given how European car companies, German in particular, are sticking their heads in the sand and lobbying the commission to delay the EV transition, I suspect the end game will be that the manufacturers will be bought by the Chinese car companies. The future VW, BMW, Mercedes etc will be Chinese cars with the traditional marque on the hood.
Maybe Porsche can survive by going independent, downsizing drastically, and selling 911's to the nostalgic faithful.
Hailing from Germany's eastern neighbor, where part of the industry went to and where energy is even more expensive, I call BS.
The main reason Germany was so hooked on gas wasn't because Hans Müller needed to run his Bosch washer-dryer and heat his home to a comfortable 24°C.
It was because natural gas is an great source of hydrogen, which in turn serves as feedstock for the production of all kinds of chemicals used in industry.
That was the engine of Germany's industry, not just energy. Energy has a whole spectrum of solutions - not the case with hydrogen.
> Energy has a whole spectrum of solutions - not the case with hydrogen.
Oh my, where ever else will we be able to find hydrogen, the (checks notes) tenth most common element on our planet ?
No, there's actually plenty of hydrogen, what people who say they "need" methane to make hydrogen actually mean is that they don't want to pay what this really costs.
> But so is German industry - every day in the news either insolvency or workforce reduction and moving production to Eastern Europe - main reason often stated "expensive Energy". I wonder what are the stats for industrial production in 2025 or total energy consumption in Germany?
Every First World country seems to be following the path from industrialisation, through deindustrialisation (which, when I was at school, was still called ‘structural change’), to a service-based society. It seems to be the price we pay for economic prosperity and cheap goods - and it is not the first time in Germany this happens.
People don't like factory work. Given options, they will choose to do some less physically wearing work that pays more, and pay somebody else to do it.
Robots are the answer to reindustrialize the developed world, but that necessarily keeps industry smaller than services.
Producing nothing only works as long as there's someone else to produce things for you who wants to produce things for you. It's been China, but only by accident. China is not far away from deindustrializing and then we all suffer.
> China is not far away from deindustrializing and then we all suffer.
How? China is actively moving up the value chain, into EVs, batteries, semiconductors, advanced manufacturing, etc. China is not abandoning industry.
> It's a political choice by our own politicians.
No, it's mostly how our economic system, how capitalism works. Production goes where production is cheap. You’ll never be able to produce goods in Germany at such low costs as in China or other third-world countries without eroding workers’ rights and thereby creating a new lower class.
Or you know, you could demand that the goods being imported are made to the same quality, environmental, health & safety standards as your home country.
But that would turn off the spigot of cheap goods.....
> Every First World country seems to be following the path from industrialisation, through deindustrialisation
Are you sure? Jobs are no longer in industrial companies, and that isn't where the growth is. However when I look at the numbers I see heavy industry output is still growing.
It also takes time. You've doubtless seen headlines about the Contracts for Difference Allocation Rounds 7 and 7a, or perhaps even Round 8 taking place now.
But those headlines are about the step where you find out who won these auctions because that's when that magic price number is known, wind power for £68.35 per MWh or whatever.
Going from the fun headline to actual contract signing in which there's a legal promise to actually try to make electricity and get paid, takes several months. After they've signed they begin the actual work, which still includes planning not just the mechanical aspects.
Just building a solar farm might take 12 months, and wind farms are often slower.
AR6 took place in 2024, you'd have read headlines back then about some excellent solar farms, getting paid about £50 per MWh. First, those are 2012 prices, for comparison reasons they choose a base year and then calculate inflation adjusted pricing in reality but I bet that £50 figure was what you saw in a headline, of course the solar farm didn't exist in 2012, so £50 per MWh was never the real price.
More importantly though, those solar farms weren't built in 2024. The plan when the bids went in, said 2026/27 for some and 2027/28 for the majority. A few of those AR6 solar farms (by no means the majority) were live in time for the July sunshine, in 2026. But you've already read headlines about AR7, and like I said maybe even AR8.
A big boost in practice is home install. A big business can take six years to do contract paperwork, get something signed off, build it and then commission it. A home owner is angry after six weeks without work done. Where are my fucking panels? You were going to install solar panels, make it happen! So lots of individuals and small businesses which own a building with a roof installed a few panels, they might add up to only 5kW each but if a million people do that it's 5GW of nameplate electricity generation, a sizeable fraction of UK usage for several hours per day.
I suspect plug in solar will make a massive difference. Plug in batteries would be even more but there’s too many people making too much money for that, at least this year.
i am pretty sure its not energy that domaines the move. eg skoda is now way more profitable than rest of VW. The cost of employment is way higher in DE..
Do you know of any major German industry for which energy costs amounts to even 10% of the labor costs? It's a rhetorical question intended to engage the common sense pathways of your cognitive system.
Realistically, without running the numbers, only few industries come to mind that may come close. Chemical manufacturing, steel production and maybe glass production as well.
For the remainder, the wage costs - more specifically - the overhead costs are the true killer. If you want to pay someone 80k a year in Germany, the actual costs are closer to 100k a year in purely salary overhead costs, not to mention the other bureaucratic overhead in Germany
it's less more around 94k - IMHO the overhead is the bureaucracy that basically forces you to employ a large amount of non-productive workforce in larger companies just for compliance topics.
Considering Germany wiffed on the tech boom, and still relies on the same industry as 50 years ago, I'm not sure if you are gesturing that 10% is way too small or way to large. Their three largest exports are all very energy intensive (cars, machinery, chemicals).
This code could appear many times in training data; but I wonder what decoding params did you actually used, because that effect appears likely with close-to-greedy way. Also, if it is a coding assistant model, chances are this skill was increased by additional sampling from that domain during training? Or maybe RL has teached model to provide verbatim answers for certain questions? So it was rewarded if that kind of answers is really helpful (I guess it was?).
Gemma 4 12B QAT is amazing - agents run very fast, and it's really very smart, at least in my agent's harness domain which is GNU software development - on par with frontiers like GPT Sol, DeepSeek, or Claude - Why to buy those expensive tokens if a local tiny model performs so well?
They’re exaggerating or have a very simple way of using these models. The Gemma 4 series, even at 31B, is nowhere near the frontier. They’re great models, but you will notice a huge difference for complex tasks.
The best local agentic coding experience I’ve had so far is Qwen3.6-27B with Pi.
> They’re great models, but you will notice a huge difference for complex tasks.
Yes and no. I think where frontier models really blow small models away is in how thorough they are in order to infer your intentions and how best to accomplish them. So you can tell Claude "change this code to make it do X", whereas a Qwen3.6-27B or Gemma4-31B can do the same, but you have to be a lot more thorough, i.e., "change this code to make it do X, but first, let me explain the concept of X as I see it and some notes about things to avoid or pay attention to while you're doing it." So for best success with small models you really need a big toolbelt of skills and MCPs.
>> At the moment, we are locked in an extremely intense, multilayered commercial and geopolitical race.
* Creating (and replicating) any software which can run in isolation locally is completely solved - we already see initial tries to create completely new operating systems, messengers, tools and platforms
* Replicating large Web-services like Netflix, Google Search, YouTube, or TikTok is more difficult but not impossible - national regulations may cause major and quick shift here. Tech is there, initiative is missing yet. Next 2-4 years we'll see new Amazons, Netflixes, and Spotifies.
* Remaining "hard" blocks are due to hardware production monopolies, but it's a matter of months or few years: we will see more and more new GPU types with new software frameworks, completely from scratch, covering all stages, incl. full physical layout - tapeout ready; new CPU and DRAM architectures; EDA tools - HPC-ready, GPU-aware
Exactly! I just vibe coded (with GPT Sol and Claude whatever-number) my own agent, it's trivial to add now any feature I want - simply ask more powerful model to do it for you. I am happy with end result, however it looks indeed these tools are trained to increase token count - they do quite stupid token-spending steps while making code, but the code itself is also a bit weird - it's like they intentionally do code which is hard to modify on your own without using exactly those authoring models. Interestingly, when I am using DeepSeek with OpenCode, I don't see that - it understands my intent well enough and overall code quality is not bad. I recently switched to local Gemma 4, and I often switch (in opencode) to just that less powerful model, because it understands my intent and has enough skills to provide good quality solution although it's rather for small size projects, and for not coding from scratch, but it's also free and private. It feels slower than any big cloud model, so my model switching is probably most quickest path to robust end result :)
It's probably a matter of short time until it's possible to disassemble any sophisticated software, rewrite it entirely with better features and usability, generate all needed artifacts, port to any platform. The only moat left is probably remote massive data storage. So if you want to replicate YouTube or TikTok, it's not impossible, but requires a lot more hardware assets than say anything that runs entirely locally (like operating systems or most video games).
How exactly do you envision this ? you will ask the ai to make youtube but better, people seperate the "what" from the "how" but they have a very complex relationship
Have you tried most powerful recent models? You may even don't understand at all "how" - what you get is end result you asked for, verified and tested. I am pretty sure if I can vibe code quite non trivial solution in just say 4-8 hours today, then why will it be impossible say in a year or two to vibe code something at the level of most sophisticated apps and services today? Pretty reasonable expectation taking into account what was possible just a year ago and what you can make today without even understanding "how".
> then why will it be impossible say in a year or two to vibe code something at the level of most sophisticated apps and services today?
Replace "vibe coding" with "outsource to India" and it's the same situation we were in 20 years ago: I remember around the time when YouTube had well-established itself (2009-ish?) and so a cottage-industry mushroomed all selling visual-lookalike YouTube clones, but implemented as monolithic single PHP project or WordPress plugin, using the local filesystem for video file storage (or worse: Base64-encoded text in MySQL, lovely).
...to someone with no prior experience or knowledge how highly-scalable, Google-tier web applications (like the real YouTube) work or are built, these "Me-Too-Tube" sites were indistinguishable from the real thing and as far as they were concerned they were happy with it.
Never mind that even if those $25 scripted websites were ever actually scalable, it isn't enough to simply have a video-sharing site; you need to get millions of people to decide to start using it - and that's something which will elude Claude's abilities... possibly forever, I feel.
One trick I find useful is to have one model with two KV caches running and while first cache has produced tokens, second cache immediately summarizes them during input tokens are being generated (tools time), then harness switches to the second KV cache which takes newly produced input tokens while KV in first cache is getting replaced with compacted summary tokens. This is a kind of ping pong, so we trade more space for less time. Still experimenting but it looks it works, and nice bonus it improves GPU utilization. Btw I have my own harness and model serving code, but it can be easily implemented in any other harness and model server.