Turning 100+ page SOP documents into executable policies using DeepClause and DML.
Building on the self-modifying harness concept we show here how to use DeepClause to build more reliable agents for outsourcing work defined via SOPs and policies. Instead of taking a big markdown file and putting it into the system prompt, we let a coding agent convert policies into small executable logic programs in a DSL called “DML”. The “leaves” of these programs can either be deterministic rules or LLM-driven agent loops. DML programs run safely inside the WASM build of SWI-Prolog.
Great work! If anyone is looking for a way to integrate something like this into their own harness or the pi coding agent, then you might be interested in DeepClause [0]. It comes with a Prolog-like language implemented on top of SWI-Prolog (WASM Version). The purpose of the project is to allow for broad experimentation around the intersection of LLMs/Agents and GOFAI. So you could use it to build memory systems like OP did, create executable specs, define graphs and loops for agents and subagents... It also comes with a pi extension that greatly simplifies getting started with it.
Opposed to OP, DeepClause uses Prolog semantics, so running some more complex queries on knowledgebases might cause some issues (which is the use case where a Datalog might be more useful). For smaller scales it should be fine though.
I'm not sure if this is the right direction, but it's certainly momentarily helpful. I think the right direction would be to enable the model itself do dynamic program analysis, deterministically and dynamically via runtime-inference.
btw. your comment is grayed out, not sure what it means. However, thanks for sharing, I'll look into it.
I’ve been working on DeepClause [0], an agentic harness, the core of which is based on Prolog. Every interaction (conversation turn, plan, execute plan, etc.) triggers a small logic program that orchestrates one or more agent loops or prompts. All these programs are completely hackable, so you can easily adjust the core logic and behavior of your agents in a reliable manner. Also, instead of using markdown specs, you can “compile” your markdown into Prolog code. My hope is that this might unlock new possibilities for spec driven development.
It’s been a lot of fun and I am somewhat proud of Prolog/Typescript integration layer built on SWI Prolog’s WASM version.
Other than that, I am not sure how and if I will continue with it. Feedback welcome!
Indeed, and maybe that's all there is to it. Still, I'd hope we will eventually better understand what's exactly happening in the wake of many repeated applications of f().
For my own (rather idiosyncratic) harness I've been experimenting [0] with "compiling" long markdown specifications into small executable logic programs. It's too early to tell for sure, but I believe that this approach does have its merits when you want some guarantees about how your agents behave for longer tasks.
Interesting idea. I’ve been noodling about something similar myself for a few months, but I haven’t moved forward with testing it. What sort of outcomes are you seeing with it? IMO, we’re never going to get to AGI without fusing “soft” AI decision making with “hard” logic and symbolic algorithmic reasoning. Humans don’t realize this most of the time, but we routinely use them all.
Sorry, did not notice your comment until just now.
So far I am observing two things:
1. For smaller models, performance on Benchmarks such as DeepPlanning does increase significantly.
2. Context hygiene for sub agents becomes much simpler, since that can be expressed relatively concise and the mechanics are handled by the runtime automatically.
Still looking for a good test cases to study possible advantages, but running reliable benchmarks does take time and money...
Are you the Deepclause author? I tried it yesterday and my first reaction was that it was slow. Perhaps I’m doing something wrong, however. Running against the same model in Pi was lightspeed in comparison. Second reaction is the prompt editor needs to handle more than a single line of text and it needs Emacs editing key bindings like Pi. I’m happy to do some testing on it and provide further feedback. What’s the best way to submit that? GitHub issues?
Yes, I am the author and thanks so much for trying! Please do submit a github issue. My first suspicion about the speed is that maybe an inner loop is taking too many turns until the model finally realizes that a task is finished (so that in turn the runtime knows whether the predicate failed or not and can continue execution accordingly). Happy to take a closer look!
The point about multiline prompts is very valid obviously, that's on the todo list.
As a European I have long given up on any meaningful change w.r.t AI. Imho the average European is much more risk averse than the average American or Chinese. That and a plethora of other factors that have been discussed over and over again, make it unlikely that we'll see things change within the next ten years or so. Only massive and immediate threats (e.g. he crisis in Ukraine) will make people and governments reconsider their fundamental beliefs (and even then the pace of change will be slow).
I am not sure whether an average European is more risk averse than an average American or Chinese. This is hard to measure. But in general, technological progress mostly does not depend on average people, and we have more than enough young warriors to try new things. Unfortunately the regulatory environment pushes them out to North America, where you don't have to jump through so many bureaucratic hoops.
Now the funny thing: everyone I talk to agrees that Europe is overregulated and that the amount of paperwork is unhealthy. But everyone also seems to have their own pet regulation that they will defend to death, and the sum of those pet regulations is basically the entire suffocating code. After all, those papers usually aren't brainchild of the bureaucrats themselves; they came into being as a result of lobbying of special interests, and those special interests usually still exist and are vigilant about protecting their achievements.
I am not sure if anyone, anywhere ever achieved an actual significant reduction of bureaucracy (say, by 30-40 per cent or more) by anything less than a societal collapse or a debellatio during a military defeat, Germany 1945 style. This seems to be the hardest, most intractable societal problem of entire humanity. Not cancer, not aging - we can probably conquer both. But an endless aggregation of more and more papers until the entire machine stops in its tracks.
I just had an across-fence talk with my neighbor, who is a retired teacher. She is not that old, and she told me that she retired voluntarily after the balance of her work shifted to "I spent more time filling out questionnaires and records than actually teaching kids".
I think this view stems mainly from the fact that we don’t have any big names like Anthropic or OpenAI, and that Europe is a bit slow off the mark. We do indeed have many promising tech start-ups, and plenty of talent, particularly in the field of AI. However, you don’t hear anything about many of these start-ups, as they often do not work on the flashy stuff. I would even go so far as to say that Europe is generally a continent where many influential companies fly under the radar. We simply don’t have any ‘rock stars’ like the US does. What we can actually be criticised for is squandering potential. Many promising and potentially disruptive developments remain confined to the universities. Unlike in the US, start-ups aren’t so easily set up here and showered with venture capital. However, we are not following the Chinese model either, where tech start-ups are bankrolled by the government - and I think we have atleast to choose for one of these models to accelerate development.
imho the biggest issue is structural - that investment and capital is spread out. Having tried to launch a startup in the EU and failed due to lack of access to capital, we could have moved to hamburg or berlin or london or paris, but not a place bigger than all of them at once the way a startup can in the us moving to san fran or NY.
Such concentration of wealth couldn't exist in the EU, its structurally designed to spread out wealth amongst its member nations. Probably a good thing for quality of life but it definately makes access to capital harder.
I have been invited in one of the UA/EU UAV cooperation programs. The largest government body is behind this program. The invitation was a 38-page PDF, out of which 36 pages were dedicated to regulatory compliance and <2 pages about actual requirements for the product.
Europe’s biggest problems are a cultural superiority complex (we’re better than brutish Americans who only care about money), combined with deep seated racism (we’re better than our former colonies, eg China), and a heavy dose of denialism (buying cheap Russian oil could never bite us in the ass).
Thinking you’re superior to everybody else and sticking your head in the sand isn’t exactly the recipe for success.
There’s also a huge talent drain: many Europeans who realize the superiority complex isn’t built on solid foundations - particularly in science, engineering, research, and development - leave to the US because they want to be with the best, which aren’t solely Americans, it’s people from around the world who aggregate in the US.
Europe has a brief window to be the new global talent hub as MAGA is attempting to destroy the US edge from within, but Europe is absolutely blowing it with mini-MAGA movements instead.
Europe has a lot to offer, but it needs to get real about where it stands in the world.
As a European, I see the word "change" used all the time as if it is a good thing. Change is good, bad or indifferent depending on circumstances. It is not automatically progress.
Irrelevant. The world changes for better and for worse, whether you want to or not, and you can only ignore it and piss against the wind so long before you get wet.
Ignoring, resisting or opposing global change only works if you're THE global hegemon and can bully everyone else to operate by your rules. Europe is not, therefore it can't dictate terms for others so it can't keep resisting global changes without loosing more of its prosperity, leverage and influence to the rest of the world.
EU is stuck in the middle of the race between two superpowers, and if it refuses to take part in the race and sit it out, it will simply become a colony of the winning power(s). It's why the US entered WW1 and WW2 even if it didn't have to. Because if it didn't and just decided to sit it out chilling away from the conflit, it would have missed out on all the major industrial and technical progress the war effort has led to and the USSR would have swallowed everything. That's how it became the world hegemon.
EU on the other hand keeps deciding to sit out most of the global battles and gets left behind with no leverage in key sectors, then complains that its trading partners are treating it "unfairly". Well yeah, when you have no leverage you will only get sloppy seconds, that's how the world works. If you want a piece of the action, you have to get in the fight, get a little bloody, and take what you want, not sit by and expect the victor will share a piece of the spoils with you for free.
> It's why the US entered WW1 and WW2 even if it didn't have to. Because if it didn't and just decided to sit it out chilling away from the conflit, it would have missed out on all the major industrial and technical progress the war effort has led to and the USSR would have swallowed everything.
The USSR was formed in 1922. Up until September 1917 (a consequence of the February revolution), Russia was ruled by the tsars. The US declared war on Germany on April 2, 1917.
> I like change for the better and not the other kinds so much.
I think you missed my point. I never said people should like change for the worst, my point was the change for the worst is often inevitable in the world, and the issue I was talking about is that the EU doesn't do anything when changes for the worst inevitably happen.
If the EU doesn't like changes for the worst happening, then it needs to get up its ass and implement the changes it wants to see in the world, by force if it has to, like the US and China are doing for their own interests. But the EU does nothing, it just sits on the sidelines, complains, talks down to, and moralizes its citizens and the rest of the word on how everyone should stop doing those bad things, which just gets applause from liberal activists, but doesn't actually fix anything, so EU's prosperity and influence declines leaving it poorer, more vulnerable and more and more at the mercy of external bad actors simply because it does nothing practical, only feel-good PR stunts and speech control on social media.
Edit: like the 3 Euro parcel tax the EU just added starting from today.
I just don't think we are able to do things. Language and countries split us apart.
Americans have one big internal market. Chinese as well. Indians - also.
In Europe? We just can't, you need to start in one country, you need to localise your products, then offer support in various languages, and I didn't even start to say anything about various tax laws or laws in general in other countries of the Union.
I believe that's the biggest issue we have and it will never be resolved.
Well India has a diverse set of languages spread across its regions.
As I understand it, the degree to which English thrives there is largely because choosing any other administrative language would be seen as oppressive to the other languages. Again, only stuff I picked up from the various Indian colleagues I have had.
Point being; I don’t think that is any different really from Europe.
If they can manage, so can we.
True, but they are one country in the end. We are not, and even if personally I do dream of European Federation, I know that this is an utopia that would have 0% chance to work in reality.
Building on the self-modifying harness concept we show here how to use DeepClause to build more reliable agents for outsourcing work defined via SOPs and policies. Instead of taking a big markdown file and putting it into the system prompt, we let a coding agent convert policies into small executable logic programs in a DSL called “DML”. The “leaves” of these programs can either be deterministic rules or LLM-driven agent loops. DML programs run safely inside the WASM build of SWI-Prolog.
reply