Here's the problem though: people have used the absence of tooling to justify the creation of new, supposedly-superior schemas and codecs that by definition have strictly less tooling available on day zero and which invariably turn out to be worse than ASN.1/DER were in 1984 because the authors also refused to study the literature to see what good ideas they could pick up. That's how we end up with:
- PB being a TLV encoding, just like DER, with all the same problems
(Instead PB should have been inspired by XDR or OER, but not DER.)
- PB's IDL requiring explicitly tagging every field of every data structure(!) even though ASN.1 never required tagging every field, and even though ASN.1 eventually adopted automatic tagging.
- PB's very naive approach to extensibility that is just like 1984 ASN.1's.
It's a mistake.
Some people, when faced with a dearth of tooling, will write said tooling. Other people will say that the technology in question is a nightmare, and some of those people will then go on to invent a worse wheel.
I'd be ecstatic to use something other than ASN.1 if it wasn't a poor reinvention of it.
Protobuf ended up having more tooling in the end though, and it didn't take very long to get there. This is like how JSON replaced XML for many use cases.
If they had put the same energy towards building tooling for an existing IDL/codec then they would have had strictly less work to do. Besides being inefficient in the use of their resources they also saddled us with 15th system (probably more like a 25th system, but you get the reference), and a poor one at that. There is really nothing much good to say about PB.
Yes, it's about a lot more than that. It's about automatically and recursively encoding/decoding through "typed hole". A typed hole is where you have a struct with one field that denotes the type of the other, and the other is basically a byte string whose value is an encoding of a value of a type identified by the other field. Typed holes are surprisingly common in protocols. Typically you first decode the outer value, then you inspect the typed hole's type ID field, then you decode the typed hole's value accordingly, and this is code you have to write by hand. Whereas with automatic handling of typed holes just one invocation of the codec is sufficient (as opposed to one codec invocation for the outermost value plus one invocation for every typed hole).
Why isn't the other value just a oneof? I get if your holed value is passthru data encoded in some special way that isn't standard asn1 or proto, but at that point it's heavily application-dependent and not really the outer protocol's job to support.
You can do CHOICE in ASN.1, yes, and you can even make it an extensible CHOICE. In that case the tag is the type determinant, and it looks a lot like a typed hole. But! sometimes you want a typed hole where the type determinant is something like a URN, or a URI, or some other type where the value space is a) large, and b) structured so you can avoid needing a registry. And sometimes the protocol you're writing inherently can't have a type registry -- think of an RPC layer where you have headers that provide things like authentication and negotiation of things, session-like things, while the application provides the procedures (the 'P' in RPC) and so you need to identify the application without a registry of oneof tags.
- PB being a TLV encoding, just like DER, with all the same problems
It's a mistake.Some people, when faced with a dearth of tooling, will write said tooling. Other people will say that the technology in question is a nightmare, and some of those people will then go on to invent a worse wheel.
I'd be ecstatic to use something other than ASN.1 if it wasn't a poor reinvention of it.