For any modern technology company, consuming open-source software isn’t optional anymore: it’s a mandatory requirement for your organisation.
Consuming open-source is only the start of the journey for many companies. Whether by design or necessity, organisations find themselves adapting from being simply open-source consumers to being full-fledged producers, contributors and maintainers, eventually working in public, following ‘open by default and ‘upstream first’ principles.
But organisational transformation is never easy. Codethink has worked with many clients as they embark on this journey of change, and based on our experience, we believe the most viable option is to fully embrace open-source culture and operate as part of the open-source ecosystem.
This article intends to outline some of the things we’ve learnt from this experience, about the benefits of working in the open, and some mistakes to avoid.
The open development model is simply better
Firstly, let’s touch on why open-source is so ubiquitous in modern technology. In my heart, I’d like to believe that it’s down to the fact that companies care deeply about the utility of free software, viewing it as a shared public resource and a force for a common, greater good. But sadly, that’s probably not the case.
Open-source is so wildly popular because the development model is so successful. Nine times out of ten, it’s simply the better way to develop software in all measurable aspects: faster development cycles, fewer defects, more robust and versatile result, to name but a few. Lots have been written on this already, and I won’t go into it too much here. Still, these merits essentially stem from the collective intellect of an unlimited number of developers able to contribute and review the source code (consider Linus’s law: ‘given enough eyeballs, all bugs are shallow’), in combination with progressive strategies for controlling changes
Prior to open-source, most proprietary software followed a traditional design and development approach: a team of experts worked behind entirely closed doors to develop and eventually release their product. The result of this in today’s market would be a severe duplication of effort and re-inventing the wheel due to the ‘Not Invented Here’ mentality. In contrast, let’s consider the Automotive industry, where the world’s vehicle manufacturers work collaboratively in the open via the AGL project. The project provides an infotainment based platform for all auto-makers to build on. It gives them 70-80% of the starting point for a production system and allows them to focus on innovating and customising the last 20-30% to differentiate themselves, rather than investing wasted effort to develop the same core each time in silo.
This is not to say that open-source is an ‘all-or-nothing’ proposition. Organisations do well to identify which of their software is close to their strategic core and perhaps cannot be shared; the rest can be considered as supporting services and will likely benefit from the open-source approach.
The lesson: gain an appreciation of why open-source is so popular and consider the ethos behind the development model as part of your culture.
Don’t wait until it starts raining to fix a hole in the roof
At the beginning, consuming open-source is expensive, as organisations have to accept changes: learning how to integrate new dependencies, modify features for specific use cases and apply bug fixes. After this, organisations can realise the medium-term benefits: reduced costs from removing licence fees and greater freedom to control inputs.
For the time being, consuming open-source seems like a solid business decision. Actively contributing to open-source and operating within the ecosystem may not seem necessary. It can certainly be hard to appreciate the return on investment doing so. After all, it takes time and expertise to be done correctly.
Eventually, though, the market demands innovation in the form of new features, and new security patches are likely also needed. Some backporting of the latest developments in open-source is possible, but again it feels expensive. The kernel and toolchain now feel a little dated. So, organisations realise that they must upgrade. Again this feels expensive but is absolutely necessary, and the benefits are clear and obvious.
But here is where the specific changes made by organisations matter. All the differentiation factors that provide the USP for the product have been applied locally: the code modifications for specific use-cases and bug-fixes mentioned earlier. They’re critical to your business, and they now must be forward-ported to the newly upgraded software. Because the organisation has carried these changes locally, only they can re-apply them, and the cost is immense. It’s a complex task and requires a specific expertise, often requiring outside help [side note: often, it’s in these exact situations where Codethink has been contracted to assist customers].
As a result, leaders at the organisation realise that they have less control over the inputs of their product than they thought and decide they must contribute features upstream in order to avoid this cost when the next upgrade is required.
The lesson: wherever possible, upstream features sooner rather than later, and avoid local patch carrying at all costs.
Make upstreaming part of your development process, not an afterthought
By this point, it is clear that contributing code in the open to the projects that are relied on is necessary. The action required now seems relatively straightforward: simply submit the required patches to those projects. But without proper planning, this activity will almost certainly fail. Many organisations fall into the trap of considering this as something to be done when time permits or assigning it to the wrong people who cannot focus on it due to other priorities.
Instead, upstreaming needs to be embedded as part of the development culture: with proper planning of which release cycles are the target for features and active engagement upstream. Teams should be built with the express intention of becoming key members of the upstream community, working towards eventually being able to drive the focus of the project. This will also lead to faster resolution of issues and shorter development cycles for the features that matter to your organisation. Only then will you be able to fully benefit from the innovation brought by the intellectual horsepower of the community.
This will also help with your hiring process: encouraging engineers to work in the open as part of their day job is a major point of desirability when hiring.
The lesson: plan properly how to invest in open-source projects that develop the key software you consume, and feel the benefit of the innovation from the community by being a driving force in the direction of the projects.
Related to the blog post:
- Automated Linux kernel testing
- Native compilation on Arm servers is so much faster now
- Higher quality of FOSS: How we are helping GNOME to improve their test pipeline
- RISC-V: A Small Hardware Project
- Why aligning with open source mainline is the way to go
- Build Meetup 2021: The BuildTeam Community Event
- A new approach to software safety
- Does the "Hypocrite Commits" incident prove that Linux is unsafe?
- ABI Stability in freedesktop-sdk
- RISC-V User space access Oops
- Tracking Players at the Edge: An Overview
- What is Remote Asset API?
- Running a devroom: FOSDEM 2021 Safety and Open Source
- Meet the codethings: Understanding BuildGrid and BuildBox with Beth White
- Streamlining Terraform configuration with Jsonnet
- Bloodlight: Designing a Heart Rate Sensor with STM32, LEDs and Photodiode
- Making the tech industry more inclusive for women
- Bloodlight Case Design: Lessons Learned
- Safety is a system property, not a software property
- RISC-V: Codethink's first research about the open instruction set
- Meet the Codethings: Safety-critical systems and the benefits of STPA with Shaun Mooney
- Why Project Managers are essential in an effective software consultancy
- FOSDEM 2021: Devroom for Safety and Open Source
- Meet the Codethings: Ben Dooks talks about Linux kernel and RISC-V
- Here we go 2021: 4 open source events for software engineers and project leaders
- Xmas Greetings from Codethink
- Call for Papers: FOSDEM 2021 Dev Room Safety and Open Source Software
- Building the abseil-hello Bazel project for a different architecture using a dynamically generated toolchain
- Advent of Code: programming puzzle challenges
- Improving performance on Interrogizer with the stm32
- Introducing Interrogizer: providing affordable troubleshooting
- Improving software security through input validation
- More time on top: My latest work improving Topplot
- Cycling around the world
- Orchestrating applications by (ab)using Ansible's Network XML Parser
- My experience of the MIT STAMP workshop 2020
- Red Hat announces new Flatpak Runtime for RHEL
- How to keep your staff healthy in lockdown
- Bloodlight: A Medical PPG Testbed
- Bringing Lorry into the 2020s
- How to use Tracecompass to analyse kernel traces from LTTng
- Fixing Rust's test suite on RISC-V
- The challenges behind electric vehicle infrastructure
- Investigating kernel user-space access
- Consuming BuildStream projects in Bazel: the bazelize plugin
- Improving RISC-V Linux support in Rust
- Creating a Build toolkit using the Remote Execution API
- Trusting software in a pandemic
- The Case For Open Source Software In The Medical Industry
- My experiences moving to remote working
- Impact of COVID-19 on the Medical Devices Industry
- COVID-19 (Coronavirus) and Codethink
- Codethink develops Open Source drivers for Microsoft Azure Sphere MediaTek MT3620
- Codethink partners with Wirepas
- Testing Bazel's Remote Execution API
- Passing the age of retirement: our work with Fortran and its compilers
- Sharing technical knowledge at Codethink
- Using the REAPI for Distributed Builds
- An Introduction to Remote Execution and Distributed Builds
- Gluing hardware and software: Board Support Packages (BSPs)
- Engineering's jack of all trades: an intro to FPGAs
- Bust out your pendrives: Debian 10 is out!
- Why you should attend local open source meet-ups
- Acceptance, strife, and progress in the LGBTIQ+ and open source communities
- Codethink helps York Instruments to deliver world-beating medical brain-scanner
- Codethink open sources part of staff onboarding - 'How To Git Going In FOSS'
- Getting into open source
- How to put GitOps to work for your software delivery
- Open Source Safety Requirements Analysis for Autonomous Vehicles based on STPA
- Codethink engineers develop custom debug solution for customer project
- Codethink contributes to CIP Super Long Term Kernel maintenance
- Codethink creates custom USB 3 switch to support customer's CI/CD pipeline requirements
- Codethink unlocks data analysis potential for British Cycling
- MIT Doctor delivers Manchester masterclass on innovative safety methodology
- Balance for Better: Women in Technology Codethink Interviews
- Introducing BuildGrid
- Configuring Linux to stabilise latency
- Full archive