Just over a year ago, we launched our first open-source software project for Bitcoin mining: Braiins OS. The necessity of an open-source operating system for ASIC miners became apparent with the covert AsicBoost fiasco of 2017 (which we talked about here), but it was going to be important regardless of that incident. With so few competitive HW manufacturers building ASIC machines for Bitcoin, providing a transparent alternative to the factory firmware increases decentralization in what is perhaps the most centralized part of the entire industry.

However, Braiins OS is only one component of the full Bitcoin mining stack. Our greater ambition when we began this project was to make the full stack open-source, standardized, efficient, and secure. That means addressing the other components of the stack: CGminer and stratum protocol. We’ll share an update about the latter in a future post, but for now we’ll focus on the alternative we’ve developed to replace CGminer, which we call BOSminer.

Background on CGminer

CGminer started out as an open-source CPU miner that anybody could run. With the introduction of GPU mining in late 2010, we began to see some of the open-source parts of the miner disappearing. Each GPU variant had special pieces of the GPU kernel being developed, and only a fraction of them were made open-source right away. The general community was stuck using whatever happened to be available.

FPGAs were the next evolution of mining hardware, but not much changed when it came to this CGminer component. Some people made their code open-source, while others did not.

Then, in late 2012, ASICs entered the market and quickly began to dominate SHA-256. ASICs are embedded devices, which means that they contain a special (rather than general) purpose computing system. The typical ASIC architecture consists of the following:

Typically, you have a control board with an FPGA and a CPU running some form of Linux on it, and then you have some hashing boards with mining chips that perform the actual Bitcoin mining work. The role of the control board in this architecture is to continuously feed the chips with the right amount of new mining work. Originally, CGminer was utilized for this purpose. Over time, however, manufacturers shifted logic away from CGminer into the FPGAs on the control board.

The best-known example of this is what we mentioned above: the AsicBoost incident with Antminer S9s in which the FPGAs were between the CPU and the hashboards, enabling AsicBoost functionality covertly. The manufacturer essentially prevented the use of AsicBoost because the code in the controller was intentionally wrong and it wasn’t documented. Some CGminer source code was available, but it would just generate bad solutions and was practically useless.

In the present, we need an FPGA and a microcontroller to drive the hashing boards. The CGminer has become essentially just a disorganized and severely limited front-end for the FPGAs. In fact, it reached a point where even the engineer who created it, Con Kolivas, had to stop supporting it.

At the same time, ASIC architecture is such now that CGminer itself isn’t enough to run the whole system because it’s running some Linux. So you need drivers, bootloaders, etc. to actually run it — which are also typically closed-source. Even when manufacturers say that they’re complying with the General Public License (GPL), it’s still up to individuals to try to collect all the pieces of software and somehow patch them together to get a full system image running. Simply put, it’s a mess.

Lack of Open-Source Today

There are so many exciting open-source projects being developed throughout the Bitcoin ecosystem these days, but mining has unfortunately been lagging behind. Given incidents like the Antbleed miner backdoors and covert AsicBoost, we decided that it was about time to do something about this. Obviously it’s complicated because there are various types of mining devices and the different manufacturers often wouldn’t publish their modifications to the CGminer codebase (violating the GPL).

That’s why we started developing the Braiins firmware for mining devices. And one of the more interesting things about that is that we decided to write our software in Rust programming language, which is turning out to be a challenging but rewarding decision.

Why We Chose Rust

Having extensive experience with embedded development, we knew that this was going to be a serious challenge no matter which language we choose. With that being said, Rust is a modern programming language with a handful of properties that make it ideal for this kind of application:

1) Memory safe

Rust performs the majority of its safety checks and memory management decisions when it’s compiling. As a result, it’s better protected from software bugs and other security vulnerabilities relative to non-memory safe languages when it comes to memory access.

2) Suitable for embedded devices

There are no other prominent programming languages that are compiled and strongly typed and that have no runtime. Rust has no runtime, no virtual machine handling memory management — everything is statically compiled. And it’s much more lightweight than, say, C++. All of those things make it very suitable for embedded devices like ASICs.

3) Code reusability

Unlike most other languages, with Rust we know that if something compiles for a device, then chances are that the same code component can be reused on the server side. That’s great, because it allows use to share once code base for the hosts and embedded devices, which simplifies testing as well as production.

4) Packaging system

Rust is pretty unique in that it allows you to use different versions of the same code package simultaneously. In other words, we can run two versions of the same library in a compiled image, which would be practically impossible if, for example, we were using C or C++.

5) Built-in test harness

Generally speaking, developers don’t like to write tests. You see that with CGminer, where none of the manufacturers are contributing test cases. Rust comes with a nice test harness, so you can simply run “test” and check that there are no regressions in the code..

6) Bright future

There’s something to be said for the fact that companies like Microsoft, Amazon, and many others are heavily investing in the further development of Rust. It makes sense to choose a technology with a bright future ahead of it. Plus, that means there should be a more reusable code, which can save us a lot of time in future development.

The Difficulty of Hiring Rust Developers

The first challenge we faced after deciding to use Rust was simply the scarcity of engineers who could work with it. Braiins is based in the Czech Republic, and there simply aren’t that many Rust developers walking around looking for work. (Take note, computer science students 😉). There’s certainly a growing interest in Rust and we had several people on our team who were ready to take on the challenge of learning it, but it took many months to reach the point where we were productive enough in the codebase to start generating original code in-house. (And by the way, we’re still hiring engineers.)

Choosing Between Multithreading vs. Async Programming

Once the decision to use Rust was made and we had a competent team in place, it was time to start designing the software. This is the fun part, of course, but it comes with more difficult decisions and challenges.

One of the first decisions we had to make was how we were going to divide our computation into tasks. There are two options:

Multithreading

This is a well-established method where you have libraries and you create threads to synchronize them. The codebase for this is very solid in Rust. However, multithreading doesn’t scale if you’re receiving many requests on your network (as is the case with Bitcoin mining), and having a lot of threads slows you down because it’s a full task in the OS sharing the same address space.

Async

The async approach requires more thought up-front, but it can scale amazingly well on a single thread. We already had some engineers who have worked with node.js, and it’s easier for us to find developers that think in terms of async programming. The drawback of async is that it’s invasive to your software design — it contaminates everything and seeps through the layers. So you have to constantly be aware of that as you design the software.

Ultimately, we saw that the async approach was going to pay off in the long-term. At the time, however, the async framework for Rust wasn’t quite where we would have liked it to be. (If you look at this website, you can see the current state of things. Depending on when you’re reading this, it may actually be async … but as of writing there are still a couple things missing. Still, the standard library has matured enough and all of the async keywords are stable, so we felt comfortable to take the async path.

This was just one of dozens of decisions and challenges we’ve faced so far in writing this software in Rust, but we can’t concisely cover it all in this post. If you’re curious to hear more about the design process, stay tuned for a video of Jan’s full talk at Dev++ in Tel Aviv. (It should be posted around mid-October.)

Timeline for Moving from CGminer to BOSminer & Stratum V2

We’ve been working on two important pieces of the Bitcoin mining stack in parallel, which are BOSminer and Stratum V2. A nice side effect of this is that we’ve been able to include a Stratum V2 simulator in the BOSminer prototype which we released for developer testing. Next up on the agenda is to add a V2 proxy (written in Rust) as well so that we can simulate various possible deployment scenarios.

Meanwhile, we’ll be taking developer feedback and improving on BOSminer for the following months in order to release an alpha version in December, ultimately followed by a fully-fledged release around March 2020. You can see more details in the timeline below.

Once the fully-fledged BOSminer is released, it will make it significantly easier for us to add support for different ASIC machines in the future, such as the latest Whatsminer M20S and Antminer S17. Considering that, we are currently committing the majority of our effort and resources towards making BOSminer a reality within the timeframe shown above.

‍