Dilaver DEMİREL
What is GitOps?
At its core, GitOps consists of code-driven infrastructure and operational procedures that utilize Git.
I wanted to share an article I found particularly insightful on understanding GitOps.
GitOps is an evolution of the Infrastructure as Code (IaC) approach. It is the best DevOps practice for creating, updating, and deleting system architecture by using Git as the single source of truth for the actual/current infrastructure/system state. In simpler terms, it enables controlled and automated implementation of system infrastructure changes using the Git Pull Request feature.
GitOps enables the system’s cloud infrastructure to be rebuilt according to the latest state in the Git repository. The state of the Git repository is modified via pull requests. Once a pull request is approved and merged, the current state of the running infrastructure/system is synchronized with the latest state in the Git repository. The real-time synchronization of the pull request with the production environment constitutes the core feature of GitOps.
System administrators, who were once hesitant to make changes to the system, are now warming up to development practices like Agile and DevOps. System administration has a somewhat checkered history. In the past, system administrators used to manage hardware manually.
The History of GitOps
Git plays a critical role in software development by providing features such as pull requests and code reviews. Pull requests ensure visibility into changes made to the codebase and facilitate communication, discussion, and review of changes within the team.
Pull requests are a key feature in collaborative software development. The pull request feature has transformed how teams and companies approach software development. It has also helped facilitate the application of GitOps principles to software development.
System administrators, who were once hesitant to make changes to the system, are now warming up to development practices like Agile and DevOps. System administration has a checkered history. System administrators used to manage hardware manually. Admins would store a series of imperative scripts and configurations as a routine in many different places. These scripts could break or get lost at any time. Since the scripts were not regularly documented or shared, they made collaboration difficult.
The DevOps movement emerged from this rudimentary environment of system administration. DevOps took the best ideas from software engineering and applied them to system administration; at this point, integrated tools became code subject to version control.
The Infrastructure as Code (IaC) approach is one of DevOps’s greatest innovations. Previously, system administrators preferred imperative scripts to configure systems. Imperative software follows a series of steps to achieve a goal;
Imperative software is prone to errors and can easily break due to changes in the order of steps. Modern software development has shifted from imperative patterns to declarative patterns.
Declarative software follows a description or definition of the desired state rather than a series of commands. Below, we see examples of imperative and declarative DevOps statements.
Imperative approach:
- Install an operating system on this machine
- Install these packages
- Download the code from this URL
- Move the code to this directory
- Do the same for the other 3 machines
Declarative approach:
The software from this URL is installed on all 4 machines and is located in this directory.
IaC has encouraged and supported declarative system management tools over custom imperative solutions. This has led to the emergence of tools like Docker Containers, Ansible, Terraform, and Kubernetes, which use static declarative configuration files. Human-readable and consistently reproducible state are among the beneficial outcomes of this approach. These config files are naturally committed to Git for tracking and review. This indicates that we have reached a point that is close to GitOps but not quite GitOps itself.
Many traditional system administration problems have been solved at this stage. Configuration files are now documented and accessible to team members at a central location. Commits and pull requests are used to track configuration changes and to encourage collaboration, discussion, and review. The remaining issue at this stage is that the configuration and the live system are not yet fully synchronized. In other words, a manual change made in the live system may not be reflected in the configuration. When a configuration change is approved via a pull request and merged into the repository, the production system is manually updated to match the state of the static repository. This means that certain commands must be run manually to apply the configurations to the production environment. This is precisely the problem that GitOps solves.
The concept of GitOps was first introduced and shared by WeaveWorks, an enterprise Kubernetes management company. Since then, it has quickly spread throughout the DevOps community. GitOps is an extension of Infrastructure as Code (IaC) and declarative configuration. By synchronizing the live system state with a static configuration repository, GitOps adds a new dimension to the pull request workflow.
The Benefits of GitOps
GitOps shares many of the advantages of the “feature branch software development” workflow. Its first major benefit is that it is easy to adopt due to the use of common tools like Git. Git is the de facto standard version control system and is a tool commonly used by many developers and development teams. This makes it easier for developers familiar with Git to become cross-functional and participate in GitOps, contributing to the process.
Using a version control system allows the team to track all changes made to the system’s configuration. If something breaks or unexpected issues arise, this provides us with the “source of truth” (the source of the current state). Teams can review the GitOps history and see when an unintended issue occurred. Additionally, this audit trail can serve as a reference for organizational policies or security audits. The GitOps history can be used as evidence, for example, when encryption certificates are updated.
GitOps brings transparency and clarity to an organization’s infrastructure needs under a central repository. Having all system configurations in a central repository helps measure team contributions. Pull requests create passive communication channels that allow all engineering teams to review and track infrastructure changes.
GitOps can significantly boost a DevOps team’s productivity. It gives the DevOps team the opportunity to quickly experiment with new infrastructure configurations. If the new changes do not behave as expected, the team can roll back the changes to a known good state using the Git history. This is an incredible asset for environments with complex infrastructure.
How Does GitOps Work?
GitOps procedures are executed using core orchestration tools. GitOps itself is an independent application pattern. Today, many GitOps solutions primarily use Kubernetes as their orchestration tool. Some alternative GitOps tools that directly support Terraform changes are entering the market.
A pipeline platform is required to achieve a full GitOps setup—such as Jenkins or CircleCI. By automating pipelines, they bridge the gap between Git pull requests and the orchestration system. Once a pipeline hook is created and triggered by pull requests, the commands are executed on the orchestration side.
The service that sits between the pipeline and the orchestration tool has been introduced as a new pattern or tool under the name “GitOps operator.” After a pull request, it triggers a pipeline that initiates the operator. The operator checks the repository’s status, checks the orchestrator’s status, and synchronizes the two. The operator is GitOps’s magic tool. One of the most well-known operators is ArgoCD.
GitOps Examples
Imagine a team working on a system experiencing performance bottlenecks or high traffic; the team notices that the load balancer isn’t functioning as intended. They look at the GitOps repository containing the infrastructure configurations and find the config file that configures and deploys the load balancer. After some review and discussion, they determine that certain config values for the load balancer aren’t optimal and need to be adjusted.
A team member creates a pull request to optimize the load balancer values. This pull request is reviewed, approved, and merged into the repository by another team member. The merge triggers a GitOps pipeline that activates the GitOps operator. The GitOps operator detects that the load balancer configuration has changed. It verifies through the orchestration tool that the change does not match the production environment.
The operator sends a signal to the orchestration tool to update the load balancer configuration. The orchestration tool handles the rest; it redeploys the load balancer with the new configuration. The team monitors the newly updated system and confirms that it has returned to a healthy state. This is an ideal GitOps scenario.
Let’s expand on the use of GitOps a bit further;
Instead of optimizing the current parameters, the team makes a more aggressive decision to completely overhaul the configurations and deploy a new type of load balancer. Believing the current load balancer is fundamentally problematic, they want to try an alternative. The workflow here is the same as optimizing parameters. The team creates a pull request that provides a completely new load balancer configuration and deletes the old one. This is approved and deployed via the pipeline.
Unfortunately, they discover that the new load balancer is incompatible with some services in the cluster. The new load balancer causes serious traffic issues and halts user operations. Fortunately, since the team has a fully established GitOps pipeline, they can quickly roll back these load balancer changes. The team creates a pull request to restore the old load balancer by reverting the changes in the history. This will be automatically deployed again via the pipeline. This quickly improves the infrastructure and boosts the team’s reliability score.
In Summary
GitOps is an incredibly powerful workflow pattern for managing modern cloud infrastructures. Although it is primarily focused on Kubernetes cluster management, the DevOps community is adapting and deploying GitOps solutions to other non-Kubernetes systems. GitOps can provide engineering teams with numerous benefits, including improved communication, visibility, stability, and system reliability. One of the fundamental requirements for a GitOps experience is a modern Git platform, such as Bitbucket.