It’s occured to me that even someone technical may see all the movement here and just not really understand what this project is, so, I thought I’d write a brief to catch folks up to speed.
So, what’s the Dark Horse Linux Project?
The Dark Horse Linux Project (DHLP) is a new Linux Distribution.
You’ve likely heard of Linux, already. What you may not know is that the term “Linux” actually includes many different kinds of operation systems.
If you’re not sure what an Operating System is, you may have heard of Microsoft Windows. You get a little compact disc and install it on the machine, and it gives you things to click on and do your work, and, for the more nerdly, to write your own software or host your own services.
In the Linux space, the term “Linux” is really just referring to the kernel itself and in some crowds includes the gnu toolchain as well. What you might not know is that there are many providers of unique operating sytems that fall under the category of “Linux”. Each of those providers gives you a different variation of what Linux is and comes with different pieces that make up the operating system. Here are some famous examples:
- Fedora Linux
- RedHat Linux
- Ubuntu
- Debian
- Slackware
- CentOS
- Mint
- Arch Linux
You’ve likely already heard of these, and there are lots to choose from, each with unique properties. RedHat Linux and Ubuntu, for example, have attributes that make them prime selection candidates for an enterprise computing environment (think of servers, cloud, etc).
Indeed, the various communities surrounding Linux are myriad, and this diversity is critical to the continued existence of the Free and Open Source Systems movement.
However, there are some complex problems, and always have been in those communities. Some can’t be addressed simply by the introduction of a new contender, but some issues can.
Diversity in the Linux Ecosystem
If those communities were to homogenize it would become vulnerable to interference from entities that sell proprietary systems — all of which stand to gain a great deal financially from the elimination or control of a sizable portion of Linux communities and their internal development pipelines. The best thing that can happen for those cooperative adversaries is the unification of all Linux distribution communities into 3-5 distributions, as this would allow them to steer those projects into directions that are financially beneficial for those companies, but, not so great for us.
Check out the latest upcoming Linux distributions for this year. If you go through the list, you’ll see an exemplification of one of the issues this project exists to address. Every single one of these new Linux distributions is, in fact, not a new Linux distribution at all. They are rebranded releases of already existing Linux distributions.
In fact, if you look closer at the “famous” list I provided earlier, you’ll notice a few oddities that parallel this: CentOS is a rebranded RedHat. Ubuntu is a rebranded Debian. Mint is a rebranded Ubuntu (twice over on that one!).
In the list I linked to for this year’s new contenders, you’ll see that about half of them are based on Arch Linux, while the rest are based on others on the list I typed out above.
Almost all “truly new” or “parent leaf” distributions die out pretty soon after release.
Of those that don’t, there are serious issues that serve as barriers to widespread adoption, such as a non-standard libc variant, or using busybox as the userland toolchain, or missing multilib support, preventing a desktop market, etc. They have serious missing pieces that Linux communities expect to see.
That is a large part of why this project exists.
Most Established Distro Communities are Toxic
The other part of it is that, as someone who’s been participating in the Linux community since the mid-90s, I’ve found that the non-commercial communities have categorically toxic cultures. I had been involved in and even led numerous campaigns to try to address that over the years, and was seeing some progress, but I found that after all that work, one kind of toxicity was simply being exchanged for another. I wanted to create something better. So I decided to.
While researching this issue quite a few years ago, I came across a project called “Linux From Scratch”. This is such an impressive project! They tediously list out, step by step, a process to create a bespoke installation to their own specifications of a new Linux system. It’s not quite a distribution, as there is no downloadable OS, but a set of instructions that can easily serve a large part in the genesis of a fully equipped new parent leaf Linux distribution with all of the pieces those users look for.
When I said that the process is tedious, I mean that in every sense of the word. It’s alot of work and takes a lot of skill to do it by hand. So, I got to thinking:
What if I unitized all the steps that create a new distribution into separate automations represented in an object notation that allowed me to automate the entire process with selectable components?
The thinking behind this is that this would allow more people to have access to the tools they need to create fully capable Linux distributions, and that if this took on momentum it could help to facilitate the diversity of the Linux ecosystem, either directly, or by inspiring a competitor project who can do it better.
There was the ALFS project (Automated Linux From Scratch) but it had gone stale and didn’t, in my impression, fully unitize the steps — though it did automate them. Even if it had, it did not appear to be terribly receptive to customization, and the project appeared to be abandoned. I believe it still is.
So, I probed around for a few years, knowing I was about to start building a new Linux distribution.
Initially I was going to call it SURRO Linux, as I was trying to figure out how to create a new “parent leaf” linux distribution anyway, and, there was initially some interest. I created an IRC channel for the project, and a couple of the LFS crew even helped out. Until…I joined their channel to ask some questions about making some changes to the compile options.
It turned out, someone from the Ubuntu Linux community had figured out a way to gain rank in the LFS project and my access to LFS resources was immediately terminated by that person when I explained the scale of what I was attempting to do, despite being a year into the process already with another LFS developer.
As a result of Matt’s deliberate efforts to starve the project of critical resources, though, it inspired me to create an entirely new thing: A distro that helps other distros be created, as an example of how this should work — both technically and at a human level.
So I guess you could say, in a way, Matt played an important role in the creation of SURRO Linux by demonstrating that the problems I was concerned about on many levels were not just further along than I believed, but closing the gaps on one of the last bastion projects in the F/OSS world that gauranteed the perpetuity of the Linux operating system by facilitating diversity– by infecting the LFS project with his presence and compromising the decision authority in it in order to prevent the genesis of a product intended to compete with Canonical’s (Ubuntu).
I recall being profoundly disappointed at this turn of events, as, I had been learning so much from the LFS project’s other members, quite quickly, and am immensely grateful of the feedback they were providing in the early stages before that. I will always be appreciative of that learning period.
Galvanized in determination, over the years since, and in between other projects, I picked at SURRO Linux, the hard way, and had to develop some custom tools to implement what I could extract from the LFS documentation.
You may have seen some mentions of these. Rex. Examplar.
Examplar is the original name of the tool. It was recently rebranded as Rex. Rex is an automation tool that reads a libary of json objects, each representing the necessary parts of a largely idempotent automation. It then reads another json file whose contents represent steps in a plan to carry out, which reference the automations in the library. This method allows me to do large chains of automations with each step in a fully controlled context, including environment, whether a pty is used, what user the automation runs as, as it captures STDOUT and STDERR of the automation to log files for review to ensure the build is clean without having to stop.
After some time, I ran into a technical barrier with Rex that prevented the continuation of the development of SURRO Linux until it was unkinked. I worked on other projects and campaigns for a while to get my head fresh on it.
Once that issue was finally solved, enough time had passed to where I needed to refresh the codebase used as the solution involved a full rewrite of Rex.
So, I renamed it to Dark Horse Linux and started over. With the new tools in place I was able to do what had previously taken 2 years to develop in just a few days, and now there seems to be some momentum behind it.
I’m hoping to have it generate a bootable image by July of this year finally, though, it won’t really be the release of Dark Horse Linux, so much as a snapshot of a working system for others to borrow from before I move on to building out an installer image. That’ll just be when the real work starts to build out the installer and package management infrastructure. Honestly, with my workload at my dayjobs, I think July is an ambitious target. Walking through LFS is only of moderate difficulty, automating the steps and extracting enough to create a new Distribution is another matter entirely. This is all just one person currently.
You’ve probably seen some mention of a repo called “Pyrois”. Pyrois is a Rex project that will compile the install images related to Dark Horse Linux and is where the majority of the work takes place.
I’m always receptive to people participating, and if you’d like to participate, feel free to email me at “““`dhlp@silogroup.org
Along the way, many things happened since I initially announced SURRO Linux was going to be a thing. My Dad died early on. I ran into access issues researching what I was trying to do. I have other projects. I have a whole other career besides this and the project is self-funded currently. I was involved into two activist campaigns in the open source community (RMS, Freenode). My personal blog has more details about those, though, I just wanted to make it clear that this is a slowly moving project for now — though I do have high hopes for it.