We continue to participate in the promising RISC-V ecosystem. Already in 2021, Ben Dooks fixed a challenging kernel bug, Robin Heywood prototyped an Arduino Mini clone, and we completed the initial bootstrap of Freedesktop SDK to the RISC-V architecture.
Since our last post, James Thomas and Valentin David worked on stabilising the remaining parts of Freedesktop SDK. We are delighted that the new 21.08 release of Freedesktop SDK comes with RISC-V support out of the box, as this opens up the whole Flatpak app ecosystem on RISC-V platforms. All this was first demo'd in a talk at Linux App Summit 2021.
GNOME's testing-only OS was our next goal and we recently demo-ed this running on RISC-V, first on virtual hardware and then on a SiFive Unmatched board. This lets us validate the whole GNOME desktop software stack on RISC-V, which is exciting for everyone - many of the same software components are also used in the commercial Linux space.
So how close are we to a full desktop experience? Will desktop Linux be ready when the first RISC-V laptops arrive? Read on for more about what we've done and what challenges remain.
Building for RISC-V
RISC-V support is now available in stable releases of the various toolchains we need (with a little help from Codethink in the case of Rust).
Valentin summarized the status of toolchain support in his GUADEC 2021 talk:
Freedesktop SDK has a clean bootstrap process designed to support cross-compilation and to avoid any circular dependencies, so the initial RISC-V bootstrap was quick. The majority of its components must be native-compiled and this is more tricky. Due to IO and CPU constraints, compiling on today's RISC-V hardware takes a long time - it's faster to emulate a RISC-V system on your x86_64 laptops and build there.
QEMU user mode
QEMU is a powerful, open source machine emulator.
In "user mode", it allows a single running Linux kernel to run binaries built for other architectures, without needing a full virtual machine. The binaries can be executed transparently using the kernel's
binfmt_misc feature. Following our instructions, the whole BuildStream build process can run on an x86_64 machine and produce working RISC-V binaries and images.
While running our first builds with this method, we encountered a bug that caused some programs to hang. We first noticed the issue in
cargo, the build tool for Rust.
The hang can be triggered when a program uses
fork() and threads, and when we investigated we saw it was inside GLib, a dependency used in a lot of Linux desktop software. This library is used by QEMU itself, but it might also be used in the process being emulated. When a
fork() call happens, GLib's memory allocator hits a situation that it isn't designed to handle, and its thread synchronization locks up completely.
We discussed this upstream in a GLib issue and a QEMU issue, and we hope that QEMU can eventually work around the problem.
In the meantime, you can work around it manually by telling GLib to use
malloc() instead of its own allocator -
malloc() isn't necessarily safe to use in a forked process either, but the GNU C Library guarantees that it is.
To enable this workaround you set a special environment variable:
GNOME on virtual hardware
There are a lot of components involved in today's graphical desktops. GNOME defines all of these in the gnome-build-meta repository, and uses BuildStream and Gitlab CI to automate building them. Over the last 12 months we worked to enable RISC-V builds across the stack, including configuration changes, component upgrades, and lots of testing. This is now all merged to gnome-build-meta (in MR 1112) and its dependency Freedesktop SDK (issue 1165), enabling anyone to build and test mainline GNOME for RISC-V using the instructions in Merge Request 1112. Try it out!
The end result of this process is shown here: a happy Valentin playing a game of SuperTuxCart in a RISC-V virtual machine:
SuperTuxKart RISC-V running as @FlatpakApps on a bootable image of @FreedesktopSdk on qemu-system-riscv64. pic.twitter.com/XHIuYIKPiI— Valentin David (@valentindavid) February 18, 2021
GNOME on bare metal: SiFive Unmatched
SiFive are a big player in the RISC-V hardware space. In early 2021 we ordered several HiFive Unmatched devboards and waited excitedly for them to arrive. As soon as they did, we set it up in a case and we got to work preparing GNOME OS for the new platform. There is no built-in graphics hardware on the Unmatched board, so we added a Sapphire PCIe graphics card, shown here:
The two main challenges going from virtual to real hardware are getting the initial boot process to work, and ensuring the specific device drivers and other kernel tweaks are all available. Codethink has already done this for some ARM devices - you can already test GNOME OS on Pinebook Pro, Raspberry Pi 4 and more. The SiFive board isn't fully supported in mainline Linux yet, so we apply the necessary patches within Freedesktop SDK until they land upstream, along with the SiFive specific firmware and device tree configuration. You can see most of the changes along with basic instructions in MR 5046 - "Add support for SiFive Unmatched". The end result was demoed during Valentin's talk at GUADEC 2021:
Let's talk about bootloaders
The boot process is often intricate and it was one of the most complex aspects of the Unmatched bringup. I asked Valentin to help me understand OpenBSI, a component specific to the RISC-V boot process:
Sifive Unmatched firmware will load an image of U-Boot with OpenSBI from specific partitions. U-Boot will then find systemd-boot in the EFI boot partition and execute it. Through the EFI API of U-Boot, systemd-boot will load the correct kernel EFI-compatible image file. Note that the disk where U-Boot+OpenSBI is loaded from does not have to be the same as the EFI boot disk since U-Boot will scan for EFI Boot partitions in all detected disks.
The nice thing about this approach is we can use the same installer ISO for QEMU and for the Unmatched board. We prepare an SD card with U-Boot, OpenSBI and a device tree. This loads the installer ISO from a USB stick, and the GNOME OS image is installed to the attached hard drive. That's the idea, but it took some work to make this work on the Unmatched board.
Most of the boot software that we use, like dracut, and systemd-boot, required only minimal patches. But they depend on GNU EFI, a bridge between the usual Linux executable format and the COFF/PE32+ format used by EFI. Mainline GNU EFI provides support for x86 and ARM families of processors. As part of the "Add EFI boot for RISC-V" merge request, we added a large patch which enables support for building EFI binaries on RISC-V, which works well but has some remaining issues to solve before it can be merged upstream.
In the short term, we have some work to make sure the various patches are reviewed and merged by the relevant upstream projects.
Long term, we want to see RISC-V hardware with the same capabilities as other platforms. The work we've done here shows what is possible, but we can't yet enable RISC-V builds by default in GNOME and Freedesktop CI pipelines - limited processor and IO speeds mean that it would slow down the developer workflows if we ran a RISC-V build and test cycle for every merge requests, so some of this work depends on further iterations by hardware manufacturers.
Once we are able to source fast RISC-V build machines, we'll look at linking them up to and the GNOME and Freedesktop Gitlab CI instances. This will enable continuous testing to ensure first-class support, and by drawing on our knowledge of OpenQA and LAVA we could even start ongoing hardware testing. We can also add builders to the Flathub build service for apps, and at that point we'll have everything in place to bring the full Flatpak ecosystem of apps to RISC-V.
Related blog posts:
- Our start with RISC-V: Codethink's first research about the open instruction set >>
- Making a breakout board for a RISC-V microcontroller: RISC-V: A Small Hardware Project >>
- Using STPA with software-intensive systems
- Codethink achieves ISO 26262 ASIL D Tool Certification
- Automated Linux kernel testing
- Native compilation on Arm servers is so much faster now
- Higher quality of FOSS: How we are helping GNOME to improve their test pipeline
- RISC-V: A Small Hardware Project
- Why aligning with open source mainline is the way to go
- Build Meetup 2021: The BuildTeam Community Event
- A new approach to software safety
- Does the "Hypocrite Commits" incident prove that Linux is unsafe?
- ABI Stability in freedesktop-sdk
- Why your organisation needs to embrace working in the open-source ecosystem
- RISC-V User space access Oops
- Tracking Players at the Edge: An Overview
- What is Remote Asset API?
- Running a devroom at FOSDEM: Safety and Open Source
- Meet the codethings: Understanding BuildGrid and BuildBox with Beth White
- Streamlining Terraform configuration with Jsonnet
- Bloodlight: Designing a Heart Rate Sensor with STM32, LEDs and Photodiode
- Making the tech industry more inclusive for women
- Bloodlight Case Design: Lessons Learned
- Safety is a system property, not a software property
- RISC-V: Codethink's first research about the open instruction set
- Meet the Codethings: Safety-critical systems and the benefits of STPA with Shaun Mooney
- Why Project Managers are essential in an effective software consultancy
- FOSDEM 2021: Devroom for Safety and Open Source
- Meet the Codethings: Ben Dooks talks about Linux kernel and RISC-V
- Here we go 2021: 4 open source events for software engineers and project leaders
- Xmas Greetings from Codethink
- Call for Papers: FOSDEM 2021 Dev Room Safety and Open Source Software
- Building the abseil-hello Bazel project for a different architecture using a dynamically generated toolchain
- Advent of Code: programming puzzle challenges
- Improving performance on Interrogizer with the stm32
- Introducing Interrogizer: providing affordable troubleshooting
- Improving software security through input validation
- More time on top: My latest work improving Topplot
- Cycling around the world
- Orchestrating applications by (ab)using Ansible's Network XML Parser
- My experience of the MIT STAMP workshop 2020
- Red Hat announces new Flatpak Runtime for RHEL
- How to keep your staff healthy in lockdown
- Bloodlight: A Medical PPG Testbed
- Bringing Lorry into the 2020s
- How to use Tracecompass to analyse kernel traces from LTTng
- Fixing Rust's test suite on RISC-V
- The challenges behind electric vehicle infrastructure
- Investigating kernel user-space access
- Consuming BuildStream projects in Bazel: the bazelize plugin
- Improving RISC-V Linux support in Rust
- Creating a Build toolkit using the Remote Execution API
- Trusting software in a pandemic
- The Case For Open Source Software In The Medical Industry
- My experiences moving to remote working
- Impact of COVID-19 on the Medical Devices Industry
- COVID-19 (Coronavirus) and Codethink
- Codethink develops Open Source drivers for Microsoft Azure Sphere MediaTek MT3620
- Codethink partners with Wirepas
- Testing Bazel's Remote Execution API
- Passing the age of retirement: our work with Fortran and its compilers
- Sharing technical knowledge at Codethink
- Using the REAPI for Distributed Builds
- An Introduction to Remote Execution and Distributed Builds
- Gluing hardware and software: Board Support Packages (BSPs)
- Engineering's jack of all trades: an intro to FPGAs
- Bust out your pendrives: Debian 10 is out!
- Why you should attend local open source meet-ups
- Acceptance, strife, and progress in the LGBTIQ+ and open source communities
- Codethink helps York Instruments to deliver world-beating medical brain-scanner
- Codethink open sources part of staff onboarding - 'How To Git Going In FOSS'
- Getting into open source
- How to put GitOps to work for your software delivery
- Open Source Safety Requirements Analysis for Autonomous Vehicles based on STPA
- Codethink engineers develop custom debug solution for customer project
- Codethink contributes to CIP Super Long Term Kernel maintenance
- Codethink creates custom USB 3 switch to support customer's CI/CD pipeline requirements
- Codethink unlocks data analysis potential for British Cycling
- MIT Doctor delivers Manchester masterclass on innovative safety methodology
- Balance for Better: Women in Technology Codethink Interviews
- Introducing BuildGrid
- Full archive