Watch: Real-time Scheduling Fault Simulation

At Open Source Summit 2024, Codethink Fellow Ben Dooks presented ‘Real-time Scheduling Fault Simulation’. In the talk, Ben gives a brief description of Linux scheduling, a short history of real-time Linux, methods for disruption, and an outline for future work on the project (including investigating scheduler patches).

If you’d like to discuss how Codethink will help you achieve results with real-time Linux, please contact us here.

Talk synopsis:

There is a lot of work around achieving consistent real-time performance on Linux, but not as much on simulating faults such as system jitter, deadline misses, or other faults. Without this, it is difficult to test how your application or entire system copes with these problems.

As part of our work with several clients, especially in the safety sphere, questions have come up about how to test processes that rely on real-time scheduling. If we have a way of injecting faults, we can reliably test error handling and other mitigations, including throttling, restarting, or some measured shutdown of services.

This talk will review some methods we evaluated for fault injection via user and kernel space. We will discuss how existing kernel features can be used and what needs to be done to configure or extend kernel features. We will also discuss how each method works and the comparative merits where overlaps exist.

We hope that this can help promote thinking and improvements in how the scheduler, particularly real-time scheduling, is tested under Linux.

About the speaker

Ben Dooks is a Senior Engineer at Codethink and an experienced open source contributor. He is a longtime contributor to various projects, such as the Linux kernel.