It's allegorical in the sense that this is general Lisp technique
useful not just in this case. The DSL is targeted at (non programmer)
end users and supposed to be fired through a REPL. Wouldn't want to
make them write C++ with lambdas, semi-columns and whatever syntax
traps of the latest standard.
That doesn't support the post's argument, that it's not about aesthetics, but about qualitatively simpler solutions.
> Often times I hear the claim that (programming language) syntax doesn't matter or if it does, it's only to provide some subjective readability aesthetics. It's somewhat surprising to me how large majority of supposedly rational people like programmers/computer scientists would throw out objectivity with such confidence. So let me provide a simple real world case where uniform syntax enables out of the box solution which is qualitatively simpler.
That making non-programmers use complex syntax and type semicolons is bad is fair, but it's a rather different claim than the post's.
Well, dynamic queries by end users is the main goal here. Your static
helper functions are completely unusable in that context. Analysing a
query and generating code at runtime is easy and idiomatic with
uniform syntax (and accompanying language support) and the claim is
that solution is not only speedier but implementation in CL is
qualitatively simpler than the alternatives.
The point my first comment made is that you don't need to analyse a query at runtime, you just need to provide functions. I will agree again that Lisp makes REPLs easier to write than does C++, but a CL with Lua syntax could still, just as easily, provide functions and expose a REPL. It's the same solution, just without the unjustified AST transformations.
How can you provide a REPL language without analyzing it at runtime?
Write Lua random syntax in the REPL? Not a great improvement over
C++. Not to mention that you'll probably use something like `eval`
which is not compilation thus inferior.
By the way, even your original example - I can't see how it can work
honestly. How can you identify fields through (lambda) parameter names
only (no mention of types either)? Probably the least
boilerplate-heavy solution would be stringly typed.
> something like `eval` which is not compilation thus inferior
I honestly don't know what that means. Turning text into code is compilation; there is no difference between the two in that regard, except perhaps that in the Lisp DSL case it's more manual.
> How can you identify fields through (lambda) parameter names only (no mention of types either)?
Not familiar enough with Lua, but in Python you just use keyword arguments.
Implementing a "simple virtual machine" for a particular task is
greenspunning[1]. Introducing a JIT library is work, complexity, debug
and portability issues. LLVM compiles much slower to native code as
the Clasp[2] guys notice comparing to SBCL. And in the end, any of
these would be at most "competitive" to the simple Common Lisp
implementation in speed (which is even portable across the different
implementations).
"we should always use the least powerful mechanism that will accomplish our goal"
I like this when implementing something for non proficient users. But when it comes to providing tools for (supposedly) advanced users, like programmers... There's late-"socialism" joke in Bulgaria: "thrift is mother of misery". A designer doesn't know ahead of time what problems "creative" users will face long term. Providing a set of simplest mechanisms for today's challenges would possibly constrain them in the future - combination of multiple mechanisms in ways not foreseen may add large incidental complexity (like OO design patterns). Which could be avoided if less by count but more powerful mechanisms were used in first place. Macros have main role in keeping Common Lisp relevant to the latest paradigm hypes despite the standard being set in stone. Opposite to this, for example, C++ must keep introducing piles of new least-powerful mechanisms to keep pace.
Sounds very related to "shadow languages" ( https://gbracha.blogspot.com/2014/09/a-domain-of-shadows.htm... ) where we add a feature to our language in a very limited form, e.g. imports; then we end up wanting that feature to be a little more powerful, so we add some special case for that, e.g. conditional imports; then we want to use that feature some other way, so we add support for that, e.g. renaming imports; etc.
We end up with a language that has a complicated, limited, special-purpose second-language built in just to handle that feature.
The alternative is to try implementing the feature using existing facilities right from the start, e.g. making imports first-class values that can use the language's own conditionals, variable names, etc. for imports.
We can also go one step further and rather than just trying to re-use the existing language features as they are (e.g. conditionals, variables, etc.), we can ask what new feature could we use to build both the new functionality and the old functionality. That way, rather than e.g. using the built-in conditionals to implement conditional imports, we might decide to something more powerful than both, like macros, and use macros to implement conditional imports and replace the built-in conditionals :)
> The question is whether you want that sort of power in day-to-day programming
It's good to use the least powerful mechanism, no doubt. But it seems you are trying to sneak the usual "macros are too powerful for everyday use" so better be left out of a language altogether? I think when the storm comes - you'd better be equipped. Having varied ways to tackle problems (and macros are sort of linguistic abstraction orthogonal to lambda calculus/Turing machine derived toolboxes) allows for less complex solutions.
> sneak the usual "macros are too powerful for everyday use"
Not trying to "sneak" anything, I openly say that language design, which is what macro usage is, is not something you should have to engage in everyday. In fact, I would turn it around: if you have to (repeatedly) resort to language design in your everyday programming, your programming language is (woefully) inadequate. Most are.
Which is why the reasons for hitting that boundary interest me: where do I have to resort to metaprogramming, why, and what can I do about it? What non-metaprongramming facilities are missing here so that I don't have to resort to metaprogramming? And if I don't want to just add those facilities to the base, which I don't, what mechanisms can I add to the language so that users of the language can use plain, non-meta mechanisms to provide those facilities themselves?
This is a bit tricky, but I am making good progress using a software architectural approach[1], with frequent surprises as to how much simpler things can be.
> left out of a language altogether?
Quite the contrary. I think "escape hatches" (metaprogramming) are fundamental and your everyday language(s) should be built 100% on top of those mechanisms. Heck, I named my company "metaobject"[2] 20 years ago, after The Art of Metaobject Protocol.
So where's Haskell's advanced object system or the restart/condition system that Common Lisp has? Even its performance is hardly on par with CL for all the flexibility of the latter.
Static languages are still complete blubs compared to lisps as far as practical metaprogramming goes. And Common Lisp's type system is much less of a blub in that regard.
> Static languages are still complete blubs compared to lisps as far as practical metaprogramming goes.
Not convinced; many modern statically-typed languages offer macros or equivalents (and also offer alternative ways to achieve most or all of the headline use cases). Metaprogramming in those languages is substantially more code-level work than in lisp, certainly, but even lisp users tend to treat macros as something expensive (because even though custom macros are cheap in terms of code cost, they're expensive in terms of reader (and tool) comprehension): the standard advice is not to write a macro unless there is no alternative, and using lots of specific custom macros in each code area is regarded as poor style. So in practice developers in modern static languages use macros in much the same way as lisp users.
> And Common Lisp's type system is much less of a blub in that regard.
Disagree; if you don't have types that are reliably accurate and enforced at compile-time then you gain very few of the advantages, counterintuitive as it is.
I'm yet to see a "modern" (statically-typed) language with random syntax whose macro-like facilities are actually usable by mere mortals. My observation is that in practice lisp programmers, even with the good rule of thumb of not using macros when functions suffice, still use way way more macrology. Good macros are just the opposite of reader burden - they tame complexity.
And (compile-time) macros are one thing, but what happens when you need metaprogramming at runtime (for example dynamic code generation)? With the Common Lisp compiler being always present one can easily generate and compile arbitrary code at any point in time. Here's simple real world example (shameless plug):
> And (compile-time) macros are one thing, but what happens when you need metaprogramming at runtime (for example dynamic code generation)?
I'd agree that stage polymorphism is rarely used or appreciated in languages that don't have it (though I see no reason it should be incompatible with typing; as far as I can see your example is much the same as e.g. Scala's LMS; as with macros of course it's substantially more cumbersome to do this in a language with much more syntax than lisp[1]). I'm not sure that it qualifies as a "blub paradox" since to my mind it's a performance optimization rather than something that fundamentally changes language expressiveness - having AST-like datastructures that are gradually transformed/interpreted at runtime is very much a standard technique in ML-family languages, wider adoption of stage polymorphism would mostly lead to programming the same way and having it run faster, I think.
[1] I'm not convinced that it would be impossible to make a language with ML-style types but a very lightweight syntax that made metaprogramming easier. Personally even looking at e.g. Haskell I find myself wishing for a more visible syntax more often than I'm wishing for better interpreter tower performance, shrug.
OT Emacs clunkiness/outdated architecture...give me a break, minimal
core with lisp interpreter on top giving programmable and extensible
platform next to none is still what...30 years ahead of anything called
"modern".
I have yet to find anything vim can't do that I'd like it to do. Like most critics you only make vague statements. Anyway, the source is out there, if there really was a need for new features somebody would implement them. Hint, hint!
