The step by step is the right approach... but you'll discover that many 'components' are _INSANE_ and that includes their SDK.
Bare LFS is not enough (I run my own): you need a kind of userland (above glibc) multi-version system which some kind of "atomic-ish" switching (always have a stable SSH running in case something goes wrong, better than unplugging the system disk and fix it on another computer).
For linux, same thing, with a kind of flip-flop-ing for updates/fix/etc.
The insight that gives you will probably scare you: the current "open source" stack is an abomination (the worst is the SDKs I think).
That's why we need _LEAN_ open source, and that includes the SDKs.
If you think that's an abomination, try corporate billed-by-the-hour enterprise software where nothing ever changes unless it's approved by several layers of management.
I promised myself not to work for any of those anymore :)
Well, in this very case, it is more complicated than that: they have to protect their software against planned obsolescence and developer tantrum (it would be the same for any "in production software").
Modifications and changes must be weighted very carefully. For instance "removing" (including in the SDK) is usually a less worse modifications than
the others and it decreases the global technical size and complexity/number of layers of the stack/SDK (which is top priority in "security"), namely most of the time but not everytime a good thing.
Because, on most complex software (open source or not), a "believed" benign modification can be disastrous. It is even worse if close to bare metal involving complex hardware (even with massive QA[testing]).
Free software doesn't typically have an SDK. A compiler is just one of the available programs. So is a text editor, a shell and script interpreters, some incarnation of make, etc. There's nothing really special about the compiler and you can write perfectly fine programs without ever touching it.
That said, port trees usually have devel-* sections. Does that count?
Ohohoh! The complexity and cost of the various "open source SDKs" is on stellar scale. I build my own distro, and I usually remove SDKs to switch to linear orthogonal brutal build shell scripts... and I am going to warn you, you are going to see a lot of ugly close to insane.
You'll see also a good amount of more than questionable code generators. And when you will have a look at some _GNU_ makefiles of big corner-stone projects, you will question the sanity of their authors (the "makefile" itself is often, but not all the time, very questionable too nowdays). Everything was with a good intent, turned sour and painful on the long run.
On our modern computers, we should have "one-compilation-units" for most, if not all binary products (exes or shared libs) anyways. "Same language" binary static linking in the open source world should not be a thing anymore (have a look at those 'header-only' projects).
The invisible backdoor injectors (namely compilers) are a problem, for many reasons that most know here (barrier to implementation of real-life alternatives, [open source] developer lock-in, etc). The more complex is the syntax of a computer language, the worst it is (on c++ syntax complexity scale, or with a heavy/intrusive required runtime like many "new" languages do shove down your throat).
If you don't see any issue, get cproc/qbe C compiler (and the binutils gas assembler), then you compile linux (with a brutal and linear shell script, let's say for a real life x86_64 desktop) and we'll talk again after, ok? And once you succeed, we'll see how it holds in 7-10 years (the time ISO takes to break in a non pertinent way many things).
I reckon what you call SDK is what I'd call the build tools. C++ projects tend to use CMake and that's what I dread most. There have been projects where I brute forced my way on the command line towards succesfully compiling a binary. It's rare, but I kinda know the pain.
If I'd want to roll my own distro again, I'll have a look at an old dir with lots of scripts and patches where I tried doing just that. (I bet almost none of it works with current tarball versions.)
I think I'll retire with NetBSD. Keep up the good work though, someone ought to highlight the insanity of build tools building build tools.
In the SDK, I do include the "compiler": the computer language syntax complexity is critical here since it is a barrier of entry of any new "real-life" alternative compiler.
NetBSD won't protect you here.
For this major reason, computer languages with a "simple" syntax and "alternative" real-life compilers should be favored (or you know it is reasonable to build one from scratch). The current _LESS WORSE_ alternative is C and similar. But this is far from a silver bullet. I think our only way out, namely getting rid of all this toxicity is assembly without abuse of any macro-preprocessor, that's why worldwide non-IP locked ISA standard like RISC-V are so much important.
NetBSD can be compiled from another OS and a compiler can be compiled to shell script. You'd take a minimal compiler (eg lcc), compile it to shell script, run it on the target, have it compile itself, older gcc and work your way up to one that builds the rest of the OS.
If you really want to start over from scratch, some form of assembly is indeed required :) And the first assembler would have to be assembled by hand. I'm not sure if riscv is useful for that, I'd try finding the oldest m68k still serviceable. And the sooner you get to C, the better. Starting from scratch is one thing, doing it all in asm takes more than a lifetime. And from C to various interpreters, of course. Most programs can be written in a scripting language and have fewer bugs than if they were written in more verbose languages.
It's a very strange algorithm completely different from most compression methods, that's what makes it interesting IMO. But it's probably not realistically competitive, since one needs to do more after the burrows-wheeler transform, and all that "more" has been ridiculously more optimized in zstd and other modern compression methods. Compressing bwt-transformed data is easier, but that doesn't mean it's easy to further than what's easy.
I saw bellard neural net compression benchmarks. If you have time and performant system, based on the semantic of the data to compress, neural net seems tough to beat.
It is a matter of good compromises: speed and efficacy. I even wonder if LZMA2 is worth replacing bzip2.
They have to keep in mind: it will be more expensive (significantly, and in competent people, not only $$$) as you cannot beat big tech with their thousands of billions of $ and the fact they are scaled on a worldwide scope. And with a long term perspective, it is going to be even harder and more 'expensive': being independent and small has a price.
Just do not go on the ground of their "complexity", if you do, you are done for, going to be hell (specs subset only of file formats/protocols, and/or very simple alternative file formats/protocols, think docx vs utf8 text file).
And you better work on those technical regulations, and it is hard since it must not imped real and pertinent "progress".
Keep in mind that those front-ends are, in general, much more friendly towards noscript/basic HTML browsers than the official front-ends broke a few year ago.
(well, nitter was at the time)
Don't think the 'whatwg cartel' will let some web not require their web engines.
Expect shadowpaid hackers to ruin their infrastructure. Don't be a fool, expect the worst (more so when the whatwg cartel is involved).
And if you, yeah you! If you are running an "AI scrapper", have the decency to preserve those front-ends (or get the code and run it yourself).
Real BOTs are more and more using whatwg cartel web engines, namely this has an end. If not all, namely the end was already reached. The ones not using them are more likely to be shadowpaid hackers by the whatwg cartel to ruin anything noscript/basic HTML (or they could be beyond idiotic hackers, unlikely).
Not to mention, many "maths challenges" out there are nothing more than a 'wait loop' in the end. Namely, you could replace them with a HTTP hard refresh timer with human scale wait times in order to get the same pertinent result. Those are actually even worse because they are pleasing the whatwg cartel since those are made to work only in their web engines.
The least they could do is to standardize those 'maths challenges' and make them HTTP header friendly.
Human scale wait timers aren't a big issue for bots. More parallel requests. Or more websites being scraped in parallel, if the website doesn't allow multiple timers going at once.
Whereas number crunching actually ties up some CPU time.
Ofc a timer will be set per connection and has to be only one part of a whole global and probably fine-grained connection policy !
Their only purpose compared to such timer, is, in their current state to create a hard dependency on exactly the web engines of the whatwg cartel and do exclude low end hardware (this does create a barrier of entry for new hardware or small hardware).
At least, they could have had the decency to design some "HTTP" specs to de-couple those maths challenges from those abominations which are the web engines of the whatwg cartel (maybe that's the case).
Yep, I could bet that a lot of BOTs are now based on those abominations (and probably using neural nets). You also have click farms with real humans.
Because it has been hostile to noscript/basic HTML browsers.
I could use a web API with some token, but it is impossible to create an account and generates such API tokens without a web engine from the whatwg cartel.
The total cost of this experiment may be a lot less than you expect. I'd encourage you to make a guess as to what you think it could cost the US taxpayer and then check what the Department of Energy contributed [1].
The LZ collaboration is going to run their machine until at least 2030. They have almost no hope of making a meaningfully larger detector, no will you hear them suggest one. Their detector is already designed at such a scale that it contains a substantial fraction of all the needed xenon isotope available on Earth.
But you could collect this data point by chance in a bigger or smaller detector.
What I was addressing was whether this detector is big enough for a reliable discovery. In that case what you really want to constrain is the rate, such-and-such events per kilogram of xenon per year (the per-kilogram-of-xenon can be traded for a per-liter rate given the density of the xenon).
Bare LFS is not enough (I run my own): you need a kind of userland (above glibc) multi-version system which some kind of "atomic-ish" switching (always have a stable SSH running in case something goes wrong, better than unplugging the system disk and fix it on another computer).
For linux, same thing, with a kind of flip-flop-ing for updates/fix/etc.
The insight that gives you will probably scare you: the current "open source" stack is an abomination (the worst is the SDKs I think).
That's why we need _LEAN_ open source, and that includes the SDKs.
reply