Software engineer with 30+ years building production-ready business applications and 3+ years as a core database engineer building high-performance RDBMs.
Founder and principal engineer of small ISV with a long track record of owning complex systems end-to-end, from low-level design to production reliability.
Driven to learn new domains quickly and choose the simplest effective architecture, tools and paradigms for each problem.
I also work on this area (https://tablam.org) and have used languages where this weird, poorly developed language SQL was not the main interface (FoxPro).
Think on this: You imagine yourself writing a regular website with ONLy sql? no, because SQL is not a "programming language" for developers.
Is possible you could think in various ideas about why is "nonsensical" to make an app with a relational language (that SQL clearly is not) but is the same as with OOP or functional: there is not reason to be a problem, and there is a lot of things that will be far easier if a proper relational language is used, like for example, is unnecessary and ORM and/or is not complicated and confusing to make one.
It has good points, but has 2 major problems, IMHO, that we observed when I was part of a team building a RDBMS:
* It again ignore the best solution for nulls: Algebraic types. Once you have this much get solved for free
* The argument against bags:
The best idea of the paper is that a RDBMS in fact internally do different data structure and temporal representations that are not of concern to the user. Correct.
THEN it jumps to the conclusion that bags should not be presented to the user, despite the fact acknowledge it need to display them.
This is wrong, and the major reason that is always overlooked is that assume that 2 identical values are wrong to exist.
I can have "Jhon, Jhon", be 2 separate persons, not know at this time more to disambiguating, adding a Id do nothing to help me, and yet, this data is correct as is.
The language MUST allow me to deal with this. It will absurd to claim an array language, procedural, functional, imperative, declarative, etc can and a relational can't.
It say the lang is intentionally limited for purity, that true, is ideal in some cases, but that purity is what the engine/compiler should track, but not distort MY DATA for this.
And the part where the perf of having bags dismissed is easy to debunk: implement a dbms and profile.
P.D: Is true that set unlocks lots of benefits, and in some cases are ideal. But very fast once you hit the real world you see that need both, similar how btrees alone is not enough, and then there is hash-based indexes and many others.
it sucks, but reality is unfriendly to pure pure languages
P.D.2: And to clear misunderstanding, I agree with the points in the same way Rust say the borrow checker is ON by default, but I argue we CANT make a strict rule, in special because data is not ideological pure
this is an unfortunate take. btrees do not imply bags. in fact extreme bagginess can be a problem when all the keys in your page are the same. lets take this from the point of view of the user, we can certainly understand that some domain models really want to be set like, and other ones baggy.
if the base domain is sets we do get a lot of nice constraints, particularly with semantics around projections and if we want to support bags, its trivial to add a counter for the number of identical records as an additional column.
if we want to support sets in bag-land, we could imagine an extension which support 'row unique' on insertion. or we can use a 'unique results' post-filter. so there are paths, but its a lot less clear in a SQL context.
you insist that your data demands bags. I really prefer to work in sets and bagginess is always just a source of crappy errors for me. shouldn't we really support usable models for both regardless of the underlying storage?
No, I insist that real data needs both. You can bias the language/engine in one way or other.
The underlying problem is that once you say:
> if we want to support bags, its trivial to add a counter for the number of identical records as an additional column.
Then it means that this is second-class.
The main gripe I have, is that the relational model has this beautiful promise of "make the storage invisible" but need to have a coherent semantics about the data.
If you say "2 identical values/rows can't exist" then we adding the same problem that all the rdbms has: None implement the relational model, and none can deal with "relation on relations".
Imagine how crazy if I tell you a language can't do `[[1,2], [1, b10000]]` and to get into `b10000[:1] where ...` is necessary to switch to another paradigm.
P.D: note how "2 identical values can't exist" break when you have 2 columns, of numerical values, where the slice of both are the same value pair-wise: Are the 2 columns wrong?. And if both are called the same? same type? from different or same table? etc. "Duplicated" values are everywhere and is trivial to get there.
And if you "add a column" then it breaks the soundness of the language:
It blows my mind that Algebraic types are only just breaking into the mainstream. Someone told me Rust invented them, which is funny because Rust wasn't around in 1961!
They probably just don't know the name. STT is what TypeScript uses, and that's pretty well-known, though a lot of the STT power is considered "advanced typing."
I write a lot in other language that uses ADTs, and I really miss STTs for the ease by which I can construct a "Partial" data type with significantly less boilerplate.
Of course, it also enables laziness, as instead of considerig whether it's proper to have a specific Partial product type, I just create it and move on. Easier to break loosey-goosey on modeling the domain.
P.D: I implement parts of the SQL support for a RDBMS. I think the teams expect >5 years to fully do it.
I bet is possible to implement Rust way faster than that (ie: I mean here for both: Parsing and correctly executing, not all optimization and other concerns)
And harder than that? Report the error, in a way that make some sense.
This is compounded by the fact that you need the semantics involved, the environment (ie: everything on scope), the source (that means you need to keep carrying big strings around).
And what is efficient means to be destructive, but you need instead the opposite for semantics, error messages, optimizations and the like.
Software engineer with 30+ years building production-ready business applications and 3+ years as a core database engineer building high-performance RDBMs.
Founder and principal engineer of small ISV with a long track record of owning complex systems end-to-end, from low-level design to production reliability.
Driven to learn new domains quickly and choose the simplest effective architecture, tools and paradigms for each problem.
Maybe only an OS can be close to how MUCH deep you can go with a DB engine.
And I bet DB engines can go far than normal OS.
You don't know how much is still waiting for somebody to try, and how much is not applied. And how many of that DBs are not even doing, because all are constrained be being "apps" with so poor interface (sql).
Fun fact: Not exist a viable true relational DBs implemented, neither exist one with a viable programming language AND apis that is for developers.
At the risk of arguing about semantics (and not interesting database semantics) it seems half the commenters here think "deep tech" just means "like, technology you go really super DEEP into, like really dig into that DEEP DEEP HARDCORE TECHNOLOGY"
> Deep tech innovations are often radical and may create new markets or disrupt existing ones. Deep tech companies often address big societal and environmental challenges and have potential to impact everyday life. Silicon chips are an example of innovation that enabled calculation at previously unimaginable speed and scale.
Database research is good, important, critical, even! But it's not creating something new that has never existed before. It's not inventing the transistor or the integrated circuit.
>> It's not inventing the transistor or the integrated circuit.
Integrated circuit on its own isn't doing anything unless used for something. Better chips enable more use-cases, more electronics etc etc. Databases in similar way are enabling more applications for users. None of the apps or services we reliably use everyday at the scale (e.g. Google Search, Online shopping, Messaging) would not exist without solid storage software.
> Integrated circuit on its own isn't doing anything unless used for something
YES! YES EXACTLY!
That's part of what makes things like ICs "deep tech" - they are transformative innovations that have no immediate application so they need government or a very well-funded far-thinking private org (e.g. Bell Labs) to bring them into being. They are not like making a new, better database.
You have just reiterated my point very concisely, so thank you.
I don't think I did reiterate your point. You were saying unlike ICs, databases -- "But it's not creating something new that has never existed before".
The point I was making was like newer/better ICs pave way for new electronics/use-cases; so do better databases/storage software pave way for more new applications that wouldn't have existed before.
I guess I get what you are saying. Creating the first of its kind is "deep tech".
I really don't care about the definition or semantics here personally. I think creating first one is great, but then depending what it is creating 10th generation can also be quite a breakthrough. First IC had like 10 transistors, modern ones do billions and getting there probably require way more work, ideas and research than the original IC did. Things just don't improve in silos.
The good reason for worry about it is the same for data grid, list, scrolls and any other UI component that loads arbitrary data.
All UI, honestly, is only meaningful in what the screen size and our vision permit. END.
UNFORTUNATELY, you can't avoid that a user is writing "a___" and the source data has millions of things that start with `a` and all the others are dozens.
So, you can end with a massive influx of data, and sure the user see that big mess and wanna dial in, but in the meantime is nice if the UI not die in the process.
Also open to do custom software development!
Location: Medellín, Colombia Remote: Yes! Willing to relocate: No
Technologies: * System engineer: Rust · query engines · programming languages · VMs · transactions · storage · performance optimization · data modeling · ERPs · Business Apps
* Databases: RDBMs · PostgreSQL · SQL Server · SQLite · SpacetimeDB
* Backend: Business logic · APIs creation · integration · orchestration · ETLs
* Additional: Python, F#, Swift, Web assembly, Git, Jujutsu, macOS, Linux (NixOS, Debian), Windows
* ERP/eCommerce building from scratch
Résumé/CV: https://www.linkedin.com/in/mario-alejandro-montoya-cortés-6... Email: mamcx@elmalabarista.com https://www.elmalabarista.com
Software engineer with 30+ years building production-ready business applications and 3+ years as a core database engineer building high-performance RDBMs.
Founder and principal engineer of small ISV with a long track record of owning complex systems end-to-end, from low-level design to production reliability.
Driven to learn new domains quickly and choose the simplest effective architecture, tools and paradigms for each problem.
reply