GitOps is increasingly popular in the cloud-native world, allowing developers to deliver software to production using their native tooling—a pull request in Git. The underlying principle is that of infrastructure as code](https://techbeacon.com/enterprise-it/infrastructure-code-engine-heart-devops). Namely, any change in operations can be affected by a change in code.
This is also the origin of the term "GitOps": operations via Git.
GitOps is typically thought of as a cloud-native enabler, given its close association with Kubernetes, where an operational environment can be expressed declaratively in code. Kubernetes will seek to ensure that the desired state (the code) matches the observed state (the live instance).
A Fortune 500 organization I consulted with recently transformed its deployment of a legacy three-tier application to a global footprint. Existing deployments were slow, error-prone, fragile, and lacking in audit controls.
Here's how we solved some key enterprise software delivery pain points using GitOps and the key benefits for your team.
Enterprise software delivery challenges
Unfortunately, at many enterprises the adoption of DevOps is nascent, and software is still deployed by manual or—at best—semi-automated methods. Central to such deployments is the infamous change advisory board, which is required (by ITIL et al.) to rubber-stamp any change request.
Ubiquitous in this process is the run book, providing instructions on how to perform the deployment. Typically, these are complex, lengthy documents describing a sequence of manual steps that must be performed. Often, steps are skipped or performed incorrectly, resulting in deployments that differ from the expected state.
This lack of reproducibility requires the intervention of operators to access production systems to correct observed discrepancies, typically via SSH access as the root user. More often than not, such interventions create as many problems as they resolve. And more important, there is a loss of control, since it is necessary to provide operational staff with privileged access to production systems.
An additional problem arises due to manually intensive processes: The system cannot be audited, since there is no record of exactly what changes were performed and by whom. The ultimate downside of the lack of this audit capability is the frustrating poor incident resolution encountered in much enterprise software.
Frustrated users create service tickets that are passed from one team to the next, with no one taking responsibility for resolution and resorting to blaming other colleagues or vendors.
How GitOps delivered
Codethink, where I am a consultant, was tasked with transforming an organization's legacy three-tier application to one that included a global footprint. The solution comprised application deployment automation using PowerShell scripting, Azure ARM templates, Azure DevOps Pipelines, and Git as the repository. First, all of the application deployment run books were transformed into PowerShell scripts to ensure that the deployment was fully automatable and repeatable.
To make sure that the computation and database resources were built in a consistent manner, standard Azure ARM templates were used to build instances, and PowerShell Desired State Configuration (DSC) was used to ensure that the end states matched the desired states.
To utilize Git as the system of record, a cluster of independent repositories was used as follows:
- The config repository stored the desired configuration of the deployment.
- The validation repository performed validation of the configuration against the enterprise IT policies.
- The deployment repository contained the scripts responsible for taking a given configuration (received, for example, from a service desk ticket) and using automation scripts to deploy the configuration to a live instance.
A core design principle of this approach was to partition responsibilities, to ensure that individual roles and duties were rigidly enforced. This was done via Azure identity and access management, using finely granular access controls to the individual repositories. As an example, an operator doing a deployment could not access the configuration repository, since this was not the operator's responsibility.
Upon a successful deployment, a repository representing the deployment would be created from the configuration, validation, and deployment repositories. This repository would be assigned to the end user requesting the resource. This ensured a hard enforcement of the principle of separation of control—only the resource owner had access to the resource.
GitOps is all about operations via Git. Here's how an operational change could be made using this solution. Each deployed instance had an associated repository containing full details of the instance. To modify an instance—for example, to add a new database—a change could be made to the configuration code, which could be checked in and reviewed in a pull request.
Upon successful merge, a continuous integration (CI) process could execute to use automation to bring the end state in accordance with the new desired state. Precisely the promise of GitOps!
Key benefits for your team
This relatively novel approach to solving a traditional enterprise pain point using GitOps brings with it a number of key benefits. First, all changes are fully declarative based on code or configuration; gone are the run books of yore. Probably the main benefit is that Git is the underlying system of record, so there is a very strong and auditable record of change.
All changes to the target deployment are made via changes to Git and, as such, every change is held in a Git changeset (underpinned by the mathematics of a Merkle Tree). Other interesting areas for enhancement include the ability to automate pull request reviews using algorithms to determine the impact of the change. Think of this as an automated change advisory board!
Since every stage of the deployment is performed by a step in a CI process, one can perform a post-mortem on a failed deployment to determine at which stage of the deployment the failure occurred to determine remedial action. This eliminates the all too familiar finger pointing that occurs in a manual process by enforcing a perfect partition of responsibility.
Most importantly for the enterprise, however, is the rigid enforcement of separation of control by virtue of identity and access controls using Git repositories as the enforcement boundaries. It is no longer possible for operators to directly access deployments; rather such access must be attained via a relevant changeset in a repository.
This article was originally published on TechBeacon
- Automated Linux kernel testing
- Native compilation on Arm servers is so much faster now
- Higher quality of FOSS: How we are helping GNOME to improve their test pipeline
- RISC-V: A Small Hardware Project
- Why aligning with open source mainline is the way to go
- Build Meetup 2021: The BuildTeam Community Event
- A new approach to software safety
- Does the "Hypocrite Commits" incident prove that Linux is unsafe?
- ABI Stability in freedesktop-sdk
- Why your organisation needs to embrace working in the open-source ecosystem
- RISC-V User space access Oops
- Tracking Players at the Edge: An Overview
- What is Remote Asset API?
- Running a devroom: FOSDEM 2021 Safety and Open Source
- Meet the codethings: Understanding BuildGrid and BuildBox with Beth White
- Streamlining Terraform configuration with Jsonnet
- Bloodlight: Designing a Heart Rate Sensor with STM32, LEDs and Photodiode
- Making the tech industry more inclusive for women
- Bloodlight Case Design: Lessons Learned
- Safety is a system property, not a software property
- RISC-V: Codethink's first research about the open instruction set
- Meet the Codethings: Safety-critical systems and the benefits of STPA with Shaun Mooney
- Why Project Managers are essential in an effective software consultancy
- FOSDEM 2021: Devroom for Safety and Open Source
- Meet the Codethings: Ben Dooks talks about Linux kernel and RISC-V
- Here we go 2021: 4 open source events for software engineers and project leaders
- Xmas Greetings from Codethink
- Call for Papers: FOSDEM 2021 Dev Room Safety and Open Source Software
- Building the abseil-hello Bazel project for a different architecture using a dynamically generated toolchain
- Advent of Code: programming puzzle challenges
- Improving performance on Interrogizer with the stm32
- Introducing Interrogizer: providing affordable troubleshooting
- Improving software security through input validation
- More time on top: My latest work improving Topplot
- Cycling around the world
- Orchestrating applications by (ab)using Ansible's Network XML Parser
- My experience of the MIT STAMP workshop 2020
- Red Hat announces new Flatpak Runtime for RHEL
- How to keep your staff healthy in lockdown
- Bloodlight: A Medical PPG Testbed
- Bringing Lorry into the 2020s
- How to use Tracecompass to analyse kernel traces from LTTng
- Fixing Rust's test suite on RISC-V
- The challenges behind electric vehicle infrastructure
- Investigating kernel user-space access
- Consuming BuildStream projects in Bazel: the bazelize plugin
- Improving RISC-V Linux support in Rust
- Creating a Build toolkit using the Remote Execution API
- Trusting software in a pandemic
- The Case For Open Source Software In The Medical Industry
- My experiences moving to remote working
- Impact of COVID-19 on the Medical Devices Industry
- COVID-19 (Coronavirus) and Codethink
- Codethink develops Open Source drivers for Microsoft Azure Sphere MediaTek MT3620
- Codethink partners with Wirepas
- Testing Bazel's Remote Execution API
- Passing the age of retirement: our work with Fortran and its compilers
- Sharing technical knowledge at Codethink
- Using the REAPI for Distributed Builds
- An Introduction to Remote Execution and Distributed Builds
- Gluing hardware and software: Board Support Packages (BSPs)
- Engineering's jack of all trades: an intro to FPGAs
- Bust out your pendrives: Debian 10 is out!
- Why you should attend local open source meet-ups
- Acceptance, strife, and progress in the LGBTIQ+ and open source communities
- Codethink helps York Instruments to deliver world-beating medical brain-scanner
- Codethink open sources part of staff onboarding - 'How To Git Going In FOSS'
- Getting into open source
- Open Source Safety Requirements Analysis for Autonomous Vehicles based on STPA
- Codethink engineers develop custom debug solution for customer project
- Codethink contributes to CIP Super Long Term Kernel maintenance
- Codethink creates custom USB 3 switch to support customer's CI/CD pipeline requirements
- Codethink unlocks data analysis potential for British Cycling
- MIT Doctor delivers Manchester masterclass on innovative safety methodology
- Balance for Better: Women in Technology Codethink Interviews
- Introducing BuildGrid
- Configuring Linux to stabilise latency
- Full archive