When approaching to GitOps workflow you will soon have to choose between push and pull based approach. In this blog-post I will explain the high-level differences of each approach with pros and cons.
What is GitOps
GitOps stands for Git Operation: in this workflow all the infrastructure configurations are stored in a Git repository, which represents the source of truth of the configuration. When you need to modify something you will commit changes to your repository, once changes have been pushed you need to apply them to the target environment and here the difference between push and pull based approach enters in the room.
A common misunderstanding is that push and pull are related to the Git actions but git pull and git push have nothing to do with the deployment and it is not relevant whether you directly push directly on the main branch, open a pull request waiting for human confirmation, use a test branch or whatever will do in the deployment workflow.
Let’s suppose to have an Openshift cluster to manage in GitOps way.
Push-based approach
In a push-based approach there is an external entity, usually a CI/CD job, which reacts to the changes and “push” them in the production environment. For example we can have a Jenkins job which:
- Receive a webhook from Bitbucket when changes are pushed
- Pull the proper branch
- Execute a command (for example an
oc apply) to deploy the changes
This will achieve the goal to reflect the changes on the Git repository to the target system, but has several drawbacks:
- Security: an external entity (i.e. the CI) is required to deploy changes and it needs privileged or even superadmin access to your cluster
- Self-Healing: If something goes wrong during the deployment the system does not automatically rolls back the the previous stable configuration, unless something specific is implemented in the pipeline. Furthermore, if some change is applied automatically, it will not be reverted to the Git state.
- Scalability: if you have more clusters to manage you have to extend your CI accordingly.
Pull-based approach
In a pull-based approach is the target system itself which detect the changes on Git repository and deploy them. As usual there is no there is no free lunch: you need something to perform this actions like Flux or ArgoCD. In our cluster we are using Argo.
This brings several advantages with respect to the CI/CD job:
- Security: You don’t have to store privileged credentials outside the cluster.
- Self-Healing: Argo can detect issues and automatically roll-back to the previous working version, furthermore also manual changes are no longer possible (without disabling self healing) so that you are sure that the repository is really the only source of truth.
- Scalability: ArgoCD is installed within the cluster and it can manage everything automatically after the initial setup, including its own updates. This means also that the system is decentralized and, for example, your testing environment is completely independent from your productive environment.
Of course there are still limitations, for example you may not notice immediately misconfiguration, and you may not have full control over timings: ArgoCD periodically checks the changes and apply them, but the process may take some minutes and in the meanwhile you don’t have any feedback. So that you need to introduce ArgoCD monitoring to promptly detect issues and out of sync configurations to be sure that changes deployed properly.
Conclusion
In conclusion a pull-based approach seems preferable, and i general it is. Unfortunately this is doable only when you have tools like ArgoCD which takes care of the whole deployment cycle. Unfortunately there are several cases in which such tools are not available: for example RPM packages installation or, more in general, handling environment different than a Kubernetes. In those cases you still need a CI/CD or an equivalent tool like Ansible Automation Platform to apply changes and still rely on a push-based approach. In those environments DevOps engineers responsibility to implement check and rollback automation.
Alessandro Valentini
Author
