We continue to participate in the promising RISC-V ecosystem. Already in 2021, Ben Dooks fixed a challenging kernel bug, Robin Heywood prototyped an Arduino Mini clone, and we completed the initial bootstrap of Freedesktop SDK to the RISC-V architecture.
Since our last post, James Thomas and Valentin David worked on stabilising the remaining parts of Freedesktop SDK. We are delighted that the new 21.08 release of Freedesktop SDK comes with RISC-V support out of the box, as this opens up the whole Flatpak app ecosystem on RISC-V platforms. All this was first demo'd in a talk at Linux App Summit 2021.
GNOME's testing-only OS was our next goal and we recently demo-ed this running on RISC-V, first on virtual hardware and then on a SiFive Unmatched board. This lets us validate the whole GNOME desktop software stack on RISC-V, which is exciting for everyone - many of the same software components are also used in the commercial Linux space.
So how close are we to a full desktop experience? Will desktop Linux be ready when the first RISC-V laptops arrive? Read on for more about what we've done and what challenges remain.
Building for RISC-V
RISC-V support is now available in stable releases of the various toolchains we need (with a little help from Codethink in the case of Rust).
Valentin summarized the status of toolchain support in his GUADEC 2021 talk:
Freedesktop SDK has a clean bootstrap process designed to support cross-compilation and to avoid any circular dependencies, so the initial RISC-V bootstrap was quick. The majority of its components must be native-compiled and this is more tricky. Due to IO and CPU constraints, compiling on today's RISC-V hardware takes a long time - it's faster to emulate a RISC-V system on your x86_64 laptops and build there.
QEMU user mode
QEMU is a powerful, open source machine emulator.
In "user mode", it allows a single running Linux kernel to run binaries built for other architectures, without needing a full virtual machine. The binaries can be executed transparently using the kernel's
binfmt_misc feature. Following our instructions, the whole BuildStream build process can run on an x86_64 machine and produce working RISC-V binaries and images.
While running our first builds with this method, we encountered a bug that caused some programs to hang. We first noticed the issue in
cargo, the build tool for Rust.
The hang can be triggered when a program uses
fork() and threads, and when we investigated we saw it was inside GLib, a dependency used in a lot of Linux desktop software. This library is used by QEMU itself, but it might also be used in the process being emulated. When a
fork() call happens, GLib's memory allocator hits a situation that it isn't designed to handle, and its thread synchronization locks up completely.
We discussed this upstream in a GLib issue and a QEMU issue, and we hope that QEMU can eventually work around the problem.
In the meantime, you can work around it manually by telling GLib to use
malloc() instead of its own allocator -
malloc() isn't necessarily safe to use in a forked process either, but the GNU C Library guarantees that it is.
To enable this workaround you set a special environment variable:
GNOME on virtual hardware
There are a lot of components involved in today's graphical desktops. GNOME defines all of these in the gnome-build-meta repository, and uses BuildStream and Gitlab CI to automate building them. Over the last 12 months we worked to enable RISC-V builds across the stack, including configuration changes, component upgrades, and lots of testing. This is now all merged to gnome-build-meta (in MR 1112) and its dependency Freedesktop SDK (issue 1165), enabling anyone to build and test mainline GNOME for RISC-V using the instructions in Merge Request 1112. Try it out!
The end result of this process is shown here: a happy Valentin playing a game of SuperTuxCart in a RISC-V virtual machine:
SuperTuxKart RISC-V running as @FlatpakApps on a bootable image of @FreedesktopSdk on qemu-system-riscv64. pic.twitter.com/XHIuYIKPiI— Valentin David (@valentindavid) February 18, 2021
GNOME on bare metal: SiFive Unmatched
SiFive are a big player in the RISC-V hardware space. In early 2021 we ordered several HiFive Unmatched devboards and waited excitedly for them to arrive. As soon as they did, we set it up in a case and we got to work preparing GNOME OS for the new platform. There is no built-in graphics hardware on the Unmatched board, so we added a Sapphire PCIe graphics card, shown here:
The two main challenges going from virtual to real hardware are getting the initial boot process to work, and ensuring the specific device drivers and other kernel tweaks are all available. Codethink has already done this for some ARM devices - you can already test GNOME OS on Pinebook Pro, Raspberry Pi 4 and more. The SiFive board isn't fully supported in mainline Linux yet, so we apply the necessary patches within Freedesktop SDK until they land upstream, along with the SiFive specific firmware and device tree configuration. You can see most of the changes along with basic instructions in MR 5046 - "Add support for SiFive Unmatched". The end result was demoed during Valentin's talk at GUADEC 2021:
Let's talk about bootloaders
The boot process is often intricate and it was one of the most complex aspects of the Unmatched bringup. I asked Valentin to help me understand OpenBSI, a component specific to the RISC-V boot process:
Sifive Unmatched firmware will load an image of U-Boot with OpenSBI from specific partitions. U-Boot will then find systemd-boot in the EFI boot partition and execute it. Through the EFI API of U-Boot, systemd-boot will load the correct kernel EFI-compatible image file. Note that the disk where U-Boot+OpenSBI is loaded from does not have to be the same as the EFI boot disk since U-Boot will scan for EFI Boot partitions in all detected disks.
The nice thing about this approach is we can use the same installer ISO for QEMU and for the Unmatched board. We prepare an SD card with U-Boot, OpenSBI and a device tree. This loads the installer ISO from a USB stick, and the GNOME OS image is installed to the attached hard drive. That's the idea, but it took some work to make this work on the Unmatched board.
Most of the boot software that we use, like dracut, and systemd-boot, required only minimal patches. But they depend on GNU EFI, a bridge between the usual Linux executable format and the COFF/PE32+ format used by EFI. Mainline GNU EFI provides support for x86 and ARM families of processors. As part of the "Add EFI boot for RISC-V" merge request, we added a large patch which enables support for building EFI binaries on RISC-V, which works well but has some remaining issues to solve before it can be merged upstream.
In the short term, we have some work to make sure the various patches are reviewed and merged by the relevant upstream projects.
Long term, we want to see RISC-V hardware with the same capabilities as other platforms. The work we've done here shows what is possible, but we can't yet enable RISC-V builds by default in GNOME and Freedesktop CI pipelines - limited processor and IO speeds mean that it would slow down the developer workflows if we ran a RISC-V build and test cycle for every merge requests, so some of this work depends on further iterations by hardware manufacturers.
Once we are able to source fast RISC-V build machines, we'll look at linking them up to and the GNOME and Freedesktop Gitlab CI instances. This will enable continuous testing to ensure first-class support, and by drawing on our knowledge of OpenQA and LAVA we could even start ongoing hardware testing. We can also add builders to the Flathub build service for apps, and at that point we'll have everything in place to bring the full Flatpak ecosystem of apps to RISC-V.
Keep up with our RISC-V news
Fill out the form to receive our latest updates on the open CPU architecture in your inbox.
Related blog posts:
- Our start with RISC-V: Codethink's first research about the open instruction set >>
- Making a breakout board for a RISC-V microcontroller: RISC-V: A Small Hardware Project >>
- Using Git LFS and fast-import together
- Testing in a Box: Streamlining Embedded Systems Testing
- SDV Europe: What Codethink has planned
- How do Hardware Security Modules impact the automotive sector? The final blog in a three part discussion
- How do Hardware Security Modules impact the automotive sector? Part two of a three part discussion
- How do Hardware Security Modules impact the automotive sector? Part one of a three part discussion
- Automated Kernel Testing on RISC-V Hardware
- Automated end-to-end testing for Android Automotive on Hardware
- GUADEC 2023
- Embedded Open Source Summit 2023
- RISC-V: exploring a bug in stack unwinding
- Adding RISC-V Vector Cryptography Extension support to QEMU
- Introducing Our New Open-Source Tool: Quality Assurance Daemon
- Long Term Maintainability
- FOSDEM 2023
- Think before you Pip
- BuildStream 2.0 is here, just in time for the holidays!
- A Valuable & Comprehensive Firmware Code Review by Codethink
- GNOME OS & Atomic Upgrades on the PinePhone
- Flathub-Codethink Collaboration
- Codethink proudly sponsors GUADEC 2022
- Tracking Down an Obscure Reproducibility Bug in glibc
- Web app test automation with `cdt`
- FOSDEM Testing and Automation talk
- Protecting your project from dependency access problems
- Porting GNOME OS to Microchip's PolarFire Icicle Kit
- YAML Schemas: Validating Data without Writing Code
- Deterministic Construction Service
- Codethink becomes a Microchip Design Partner
- Hamsa: Using an NVIDIA Jetson Development Kit to create a fully open-source Robot Nano Hand
- Using STPA with software-intensive systems
- Codethink achieves ISO 26262 ASIL D Tool Certification
- Automated Linux kernel testing
- Native compilation on Arm servers is so much faster now
- Higher quality of FOSS: How we are helping GNOME to improve their test pipeline
- RISC-V: A Small Hardware Project
- Why aligning with open source mainline is the way to go
- Build Meetup 2021: The BuildTeam Community Event
- A new approach to software safety
- Does the "Hypocrite Commits" incident prove that Linux is unsafe?
- ABI Stability in freedesktop-sdk
- Why your organisation needs to embrace working in the open-source ecosystem
- RISC-V User space access Oops
- Tracking Players at the Edge: An Overview
- What is Remote Asset API?
- Running a devroom at FOSDEM: Safety and Open Source
- Meet the codethings: Understanding BuildGrid and BuildBox with Beth White
- Streamlining Terraform configuration with Jsonnet
- Bloodlight: Designing a Heart Rate Sensor with STM32, LEDs and Photodiode
- Making the tech industry more inclusive for women
- Bloodlight Case Design: Lessons Learned
- Safety is a system property, not a software property
- RISC-V: Codethink's first research about the open instruction set
- Meet the Codethings: Safety-critical systems and the benefits of STPA with Shaun Mooney
- Why Project Managers are essential in an effective software consultancy
- FOSDEM 2021: Devroom for Safety and Open Source
- Meet the Codethings: Ben Dooks talks about Linux kernel and RISC-V
- Here we go 2021: 4 open source events for software engineers and project leaders
- Xmas Greetings from Codethink
- Call for Papers: FOSDEM 2021 Dev Room Safety and Open Source Software
- Building the abseil-hello Bazel project for a different architecture using a dynamically generated toolchain
- Advent of Code: programming puzzle challenges
- Full archive