Business Development Director CROC Cloud Services
To organize the process of modern IT development in Russia, it is customary to use the concepts and practices of continuous integration and delivery (CI/CD). There is a lot of talk about it, but, as often happens in practice, rarely anyone applies it correctly. The reason lies in the lack of specialists who could methodologically build a pipeline and automate it. Let’s talk about what common errors in CI/CD we see in projects.
But first, a little educational program
CI/CD is a whole culture that includes a set of principles and implements a sequence of stages of software delivery from the moment it is written by the developer to deployment in a productive loop. Actually, this abbreviation is based on individual steps that are considered in a complex:
- Continuous integration — continuous integration;
- Continuous delivery — continuous delivery;
- Continuous deployment — continuous deployment.
Continuous integration (CI) – this is the packaging of the application by the developer and testing. At the planning stage, the business or the product owner has an idea how to improve the customer service and what features are needed for this. At the same stage, the analyst collects business requirements and passes them to the developer.
That one (or rather those, since several people are employed in development in a large company) writes code and uploads the so—called build — a modified application – to a separate branch. The build is tested – manually and/or using autotests — and moves to the next stage of the pipeline.
Continuous delivery – this is the delivery of the application, that is, ensuring its operation in industrial operation. At least one manual action is important here – confirmation of the person responsible for the output in production. As a rule, the team leader of the development team is responsible for this. The direct deployment of the application in the client’s infrastructure should be automated.
Continuous deployment – this is the process of continuous code delivery to the production without any additional approvals and manual operations. Monitoring of such withdrawals should also be carried out continuously and automatically. Its goal is to quickly identify the problem with the build and enable the developer to roll back the application version and fix the errors in parallel.
In one company, they did quite creatively to solve this problem — they installed a traffic light that is integrated with a monitoring system. If errors appear in it, this traffic light flashes red, requiring employees to actually react instantly.
Common mistakes when implementing CI/CD
Automation is poorly applied
An idle conveyor is a frequent pattern. In most cases, this is due to incorrectly constructed processes within the organization.
For example, even in large companies, often when developing automation is very poorly applied or it is not applied optimally. This leads to the fact that each operation of the developer, tester, administrators is accompanied by routine manual actions: write about the status, transfer information, delegate “roll out” the infrastructure, and so on.
As a result, the release period of the application increases 2-3 times, and at the same time the number of conflicts within the team increases, since it is difficult to understand at what stage the process stops.
There are no formally prescribed areas of responsibility
Often we see a different situation — in general, the CI/CD process works well, but sometimes it fails. The IT director of a Russian bank once complained that when rolling out a new version of the release, some packages and scripts “do not reach”, and feedback about non-working services does not come from DevOps engineers, but directly from users.
The problem in this case may be related to the absence of formally prescribed areas of responsibility and the presence of a large number manual operations in the pipeline, which increases the chance of an error. Also, due to the presence of such a problem, it can be concluded that functions within the team are duplicated and coordination between several participants in the process is lost.
To fix this, it is recommended to use a RACI matrix. By the way, it is often used by providers. They “on the shore” determine who will be responsible for what in the project. For example, we prescribe in the document our obligation to deploy and maintain virtual machines and infrastructure components where we are suppliers of infrastructure and Kubernetes clusters. And the software itself and the availability of CI remain on the conscience of the client.
It happens that the client entrusts us with the entire assembly and delivery cycle – in this case, we roll out the application into a productive environment, and the customer focuses exclusively on the development of his product.
Failure to follow IaC
The efficiency of the entire IT development is also affected by the availability of the infrastructure. It is justified in this case to follow the concept of IaC (infrastructure as code), when scripts and templates are written for deployment and management of the environment. They help to deploy a typical infrastructure faster, make changes without logging into the management console or to a specific virtual machine.
Administrators who do not script every action with the infrastructure are forced to apply the same efforts every time this infrastructure is required by developers for a new release. Failure to follow the IaC approach increases the cost of IT maintenance.
So, the initial deployment of the infrastructure may cost conditionally 100,000 rubles, but a repeat, based on a ready-made template, will require only 35,000 rubles, since the labor costs will not go to any comparison with what it was before automation.