To all of the users of the OpenWrt One device, I need to start by saying "I'm sorry". My intention was not to break your firmware. 😅
Of course, I'm talking about commit 6c8cc862 where one of the onboard storage drivers was removed. This all comes down to a difference of opinion in the theory about the job of an operating system.
In my (humble) opinion, an operating system should include the drivers needed to bootstrap and install said operating system. This is pretty important in the idea of "self hosting" in computer science (not to be confused with the "homelab" idea of self hosting):
In computer programming, self-hosting is the use of a program as part of the toolchain or operating system that produces new versions of that same program—for example, a compiler that can compile its own source code. Self-hosting software is commonplace on personal computers and larger systems. Other programs that are typically self-hosting include kernels, assemblers, command-line interpreters and revision control software.
It seems pretty simple. In the same way that you want your storage drivers in your initramfs, you want them in your main system as well. Personally, I'm a fan of having network drivers in my initramfs as well (specifically for handling cases like remote key escrow for disk encryption), but that's a divergence from what we're talking about.
Now coming back to the topic at hand, openwrt#16565 lays out the topic.
Recently I swapped out my core router for a Banana Pi R4. This is a 10Gb router with an two SFP+ port (one WAN, one LAN) and a 1+4 set of gigabit ethernet ports (one WAN, four LAN). At just over $100 it's a great device. It's got 4GB of RAM, 128MB of onboard SPI flash, an 8GB eMMC, a bunch of GPIO pins 2 mini PCIe ports specifically intended for a janky WiFi 7 network card, and 2 M.2 slots: one with the "B key" and one "M key".
The "B key" slot is specifically for the following device types:
- PCIe x2
- SATA
- USB 2.0
- USB 3.0
- HSIC
- SSIC
- Audio
- UIM
- I2C
If you notice a trend here, it's all about throughput. Even at the 480Mb/s of USB 2.0, B key devices are intended to not need a bunch of PCIe lanes. They (compared to the "M key") sip at the bandwidth they have available. Side note: this is part of why you'll see most of these devices hanging off of your "Southbridge" or "Platform Controller Hub" if you're on an older x86_64/AMT64 machine.
The "M Key" slot has one key purpose: expose 4 lanes of PCIe.
This isn't something I'm just pulling out of my ass. Go read the slides from the "Solid State Storage Initiative" from the Storage Networking Industry Association (SNIA) describing the use of "M.2" slots.
Clearly, the folks at SinoVOIP (the OEM behind the BananaPi and a number of the RISC-V single board computers coming out of China) got the message. In the Wiki they are pretty clear:
M.2 interface 1x M.2 KEY-B slot with USB3.2/PCIe3.0 interface for 5G,1x M.2 KEY-M slot with PCIe3.0 1lane interface for NVME SSD
as are they with the OpenWRT One:
M.2 2242/2230 socket for NVMe SSD (PCIe gen 2 x1)
From this it's pretty clear that their intentions were not ethereal. The slot is for storage. Stick an NVMe drive in it and go (which, by the way, increases the performance significantly).
Back to the issue at hand (pun intended), with this knowledge in mind I thought it was going to be pretty straightforward to add the missing driver to the default OpenWrt build for the BananaPi R4 and not have to think about it again. Oh, the joy! Not being forever stuck with custom firmware builds! The ability to flash, login, and start configuring the device. It sounded totally dreamy.
The position of the OpenWrt developers is ~~~clear~~~:
The rule here is simple: If a module is not required for any of the onboard devices, it should not be installed by default.
By my understanding, it seems like it's a slam dunk
Of course, their clarification makes the concern fuzzier:
The PCIe host controller is part of the SoC and hence, even if it is not used, an onboard device. And so are the SFP cages, for example.
If you want to create a custom image which has all sorts of drivers installed, you are free to do so, by using the buildroot, or the ImageBuilder or the online service at firmware-selector.openwrt.org.
So at the end of the day this really just comes down to whether or not an NVMe device is supplied *with the device:
Imho, as the board doesn't come with an NVMe installed from factory, I would leave it up to the user to install those packages (or create a build including them, or use the "Customize installed packages and/or first boot script" option of the OpenWrt Firmware Selector). Including every possible driver for every possible PCIe device by default is certainly out of scope.
Because, after all:
The user may install kmod-nvme. Or kmod-ata-ahci. Or maybe the user isnt even using the M.2 slot for an SSD at all, but for another XHCI USB host, or a bunch of 16650 UARTs, ...
I was a bit miffed about this, but you know what... a policy is a policy... as long as it's enforced evenly. I don't need to like it, I just need to live with it.
Fast forward five months...
I notice the commit mentioned above... For some reason the OpenWrt One (also manufactured by SinoVOIP & largely based on the BPi R4 platform) HAS THE NVMe DRIVER IN IT!!!
Well, turns out this was an oversight and tout de suite it was removed. Now,
what you have is a situation where the build on many of those devices which
"worked out of the box" is now going to break. OpenWrt users will need to
understand more of the intricacies of how builds work (turning "users" into
"developers") to fix a regression in the operating system of a device that was
previously working. In the best case scenario users will just lose access to
everything in /dev/nvme?n?p?. A little bit annoying, but hey, as was pointed
out in a comment in that issue the users can always re-install the packages by
hand... on their router... which hopefully is booting correctly (and if it isn't
they won't have internet access to use to troubleshoot the issue).
But hey, good luck with that M.2 UART card hanging off your "M Key" slot.