Automating deployment workflow with GitLab CI/CD

In previous posts, we have provisioned an Oracle Compute Linux VM and deployed a simple helloworld Django app to it. We then ran the app as a Docker container . To streamline our workflow, we also identified an integrated platform to store both our code and container image.

Now let’s build on top of this progress to migrate our existing containerized application workflow to a fully automated CI/CD pipeline using GitLab. In this post, we will

switch pushing code from Github to Gitlab
use Gitlab CI/CD Pipeline (similar to Githab Actions) and shared Runners to build Docker images and store them in the GitLab Container Registry
use self-hosted Runners to deploy freshly built images to our Linux VM

By doing so, a developer only needs to push changes to a Gitlab project repo, and the Pipeline will take care of building and deploying. This involves just 1 step and beats our previous setup of pushing changes to Github, building image manually in dev box and pushing to Docker Hub, then pulling the image from VM and manually running the container.

Let’s go!

Prerequisites

Before we dive into GitLab CI/CD, ensure you have the following:

GitLab account
Linux VM access: SSH access to your Linux VM with a user that has sudo permissions to manage Docker containers (i.e., belongs to the docker group), and install and configure the Gitlab runner.
Docker installed on VM: Docker Engine must be installed and running on Linux VM.
Application Dockerfile: A Dockerfile in the project’s root directory that correctly builds the application’s Docker image.

Project setup

new

Create a new group to invite team collaborators.
Create/Import Project
In the same interface, choose Create new project/repository. We can either Create blank project or Import project from our existing Github repo. When creating the new project, make it under the group namespace we created in #1.
Sync to Gitlab repo
If we choose to create a blank project, we will push our local code to the new repo manually. We need to update the Git remote to point the local repo to the new GitLab repo instead of the old GitHub one. Here are the steps:
- Get your new GitLab repo URL in its project page, it should look something like https://gitlab.com/your-namespace/your-project.git
- In the local project folder, see what branch we’re on with git branch. If it is not main, rename it with git branch -M main
- Rename the existing GitHub remote as a secondary with git remote rename origin github.
- Create a new remote entry to the new Gitlab repo with git remote add origin <url>
  If you want to replace the Github remote with Gitlab, just run git remote set-url origin <url> to update it.
- Check that the remotes are correctly setup with git remote -v
- Push your existing code to GitLab with git push -u origin main. Git will then redirect to Gitlab login screen to authenticate for the first time. It then pushes the current local branch main to the remote called origin, and sets the upstream or tracking branch so future git pull, fetch or push commands will target origin/main automatically.

GitLab CI/CD components

GitLab CI/CD uses a file named .gitlab-ci.yml in the root of the repository to define a pipeline. This file tells GitLab Runner what to do when changes are pushed to the repository. Here are the components in a pipeline:

Stages: A pipeline is composed of stages (e.g., build, test, deploy). Stages run sequentially. If you don’t define a stages: array at all, GitLab falls back to its default stage list in this order:
```
.pre
build
test
deploy
.post
```
.pre and .post are special hidden stages for setup/cleanup and run before/after everything else.
You can define your own stages: array to customize an ordered list of phases your pipeline will run through. This will replace GitLab’s default stage ordering (except .pre and .post, which still work without being listed).
Jobs: Each stage contains one or more jobs. A job defines a set of commands to execute. Jobs without a stage: key automatically land in the default test stage. If you do set stage: on a job, GitLab will accept any stage name that appears in its internal default list (above), even if you didn’t explicitly declare it.
Jobs in the same stage run in parallel, as long as there are enough runners available and no needs: or dependencies: to force a specific order. needs: lets you run jobs earlier than their stage would normally allow. For example, you could run a deploy job while test is still running if it only depends on build.
Runners: GitLab uses Runners to execute jobs. These are machines (virtual or physical) that pick up jobs from GitLab. GitLab provides shared runners, or we can register our own private runners in our VM. In this post, we will employ BOTH: We will leverage shared runners in the GitLab cloud to build, and a self-hosted runner on the local VM to deploy.

Building the Docker Image

Create .gitlab-ci.yml
In the root of your GitLab project, create a new file named .gitlab-ci.yml. You can do it in 3 different ways
- click + above the repo directory and select new file
- on the right Project Information pane, select Setup CI/CD -> Configure Pipeline. The Pipeline Editor opens with default build/test/deploy stages that only echo some messages.
- the best way: on the left Project pane, select Build/Pipelines. Under Ready to set up CI/CD for your project?, select the right template. Since we are planning to build and deploy Docker containers, scroll to Docker and click Use Template. The Pipeline Editor opens with working scripts for logging in, building and pushing to the registry.
Define stages
Assuming that we start from scratch, we will define our stages array. For this workflow, build and deploy will suffice. This tells GitLab:
1. Run all jobs in build stage first.
2. If they pass, run all jobs in deploy stage.
```
stages:
  - build
  - deploy
```
Define Build job in .gitlab-ci.yml
Add a job named build_image to the build stage. This job will login to the project’s Container Registry, build an image with 2 tags, and push them to Container Registry. It will only run if certain files are changed and checked in to specific branches.
```
# Define the stages of your CI/CD pipeline
stages:
  - build
  - deploy

# Job to build and push the Docker image to the GitLab Container Registry
build_image:
  stage: build
  image: docker:git
  services:
  - docker:dind
  script:
    - echo "Logging into GitLab Container Registry..."
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - echo "Building Docker image, tag with comit SHA and latest..."
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -t $CI_REGISTRY_IMAGE:latest .
    - echo "Pushing both tagged versions of image to Container Registry..."
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  rules:
      # job runs if commit to main or master branch
      - if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "master"'
      # if any of these paths changed
        changes:
        - Dockerfile
        - requirements.txt
        - helloworld/**  
        - web/**  
      # allow Manual trigger fallback 
      - when: manual
```
Let’s go over each line:
- stage: build: tells GitLab which phase from stages: array this job runs in.
- image: docker:git : runs the CI job with docker CLI
- services: docker:dind uses the Docker-in-Docker service with the Docker daemon and runtime to build images within the GitLab Runner environment.
- docker login logs in to the GitLab Container Registry using predefined environment variables (CI_REGISTRY, CI_REGISTRY_USER, CI_REGISTRY_PASSWORD).
  - CI_REGISTRY: The URL of our GitLab Container Registry.
  - CI_REGISTRY_USER: GitLab username for the registry.
  - CI_REGISTRY_PASSWORD: GitLab password or a Personal Access Token with read_registry and write_registry scopes.
    GitLab injects CI_REGISTRY_USER and CI_REGISTRY_PASSWORD automatically for projects with a Container Registry enabled, and if we are using shared runners and building within GitLab.
- docker build builds our Docker image. It’s common to publish two tags for the same image:
  - Immutable, unique tag: $CI_COMMIT_SHORT_SHA for traceability. Every commit gets a different SHA (shortened to 8 characters), so our image tag is guaranteed to be unique. We can use this tag to pull exactly the build that came from a given commit. Unlike latest which constantly moves, a SHA tag will never be overwritten so we can roll back to it reliably.
  - Mutable, convenience tag: latest (or branch name) so team members and automation can easily pull the newest build for quick test ,
    Again, Gitlab will automatically inject CI_REGISTRY_IMAGE with the full path to our Docker image in the registry (e.g., registry.gitlab.com/your-group/your-project).
- docker push publishes the built image to the GitLab Container Registry.
- rules lists conditions under which this job will execute.
  - if: limits the job run on main or master branch
  - changes: means it only runs if any of those files or folders changed. For example, if we only edit the .gitlab-ci.yml and commit, the pipeline will not execute the build_image job.
  - when: tells Gitlab if none of the above rules match, schedules the job in a manual state. The whole pipeline pauses until someone triggers it. Go to the Pipelines page and click your pipeline. You’ll see your job with a ▶ Run button. Click that, and the job will start immediately without needing a new commit.
  manual jobs
  The reason for skipping unnecessary builds is to save both our time and cost, as GitLab’s free tier provides only 400 CI/CD minutes per month per namespace on GitLab’s shared runners. Using rules save build minutes for only app code or the Docker build context changes, not pure CI config edits.

Pushing to Container Registry

The docker push commands in the build_image job handle this automatically. Once pushed, you can view your images in GitLab under Deployments > Container Registry.

When the pipeline is run for the first time, Gitlab will verify your account with phone and a tedious CAPCHA program. This is enforced per account, not per project.

Deploying to Linux VM

Deploying to the Linux VM requires a way for GitLab CI/CD to connect to our VM and execute commands. The most common and secure method is a GitLab Runner self-hosted on the Linux VM.

In this setup, GitLab sends the job definition to the VM runner, the runner executes commands locally with direct access to Docker daemon and system tools. This eliminates network latency of SSH connections from Gitlab. It also provides safer secrets handling as secrets stay on the VM (via local env vars or config), not in GitLab. Jobs on our own runners also consume zero of our free 400 CI/CD minutes.

Here are the steps:

Install GitLab Runner on VM

In Ubuntu, install dependencies

sudo apt update
sudo apt install -y curl

Add the official GitLab Runner GPG key and repo and install

curl -fsSL https://packages.gitlab.com/gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/gitlab-runner-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/gitlab-runner-archive-keyring.gpg] https://packages.gitlab.com/runner/gitlab-runner/ubuntu/ noble main" \
  | sudo tee /etc/apt/sources.list.d/gitlab-runner.list

sudo apt install -y gitlab-runner

Once done, verify installation: gitlab-runner --version

Register runner with GitLab project
Register on the VM with sudo gitlab-runner register
Answer the prompts:
- GitLab instance URL: https://gitlab.com/
- Registration token: This binds it to a scope (project or group):
  - Use a Project token if you want this runner only for this project. We will use this to keep the runner private to our repo. In GitLab.com project’s left pane: Settings → CI/CD → Runners → New project runner → copy the registration token and paste here.
  - Use a Group token (Group → Settings → CI/CD → Runners) if you want one runner for all projects in that group.
- Description: e.g. oci-vm-runner. A friendly name so you recognize the machine.
- Tags: oci,deploy. These define the labels the scheduler uses to match jobs to this runner. We’ll add the same tags to jobs in .gitlab-ci.yml so only this runner picks them up.
- Executor: Defines how jobs run on the machine.
  - shell: Runs directly on the VM. Best for local deploys that need Docker, systemd, files, Nginx, etc. We will pick this as it’s simplest for deployments to our VM using local Docker.
  - docker: Runs each job inside an isolated container. Good for clean, reproducible build/test environments. For Docker‑in‑Docker builds, you’ll need privileged mode and to pick a Default Docker image, which is the image used when a job doesn’t specify one. Example: alpine:3.20 for light jobs, or docker:24 for Docker CLI out of the box.
Setup permission for runner
The runner will need appropriate permissions on the VM to perform its tasks (e.g., access to the Docker socket). We will add the runner user to docker group. Then we’ll restart the runner service so group membership takes effect.
```
sudo usermod -aG docker gitlab-runner
sudo systemctl restart gitlab-runner
```
Verify runner status
sudo gitlab-runner list
You should see your new runner.
Check the service status on the VM with sudo systemctl status gitlab-runner
If it’s inactive or failed, run sudo systemctl enable --now gitlab-runner. This ensures it runs now and starts on every boot.
The new runner will also show up as “online” in Gitlab’s Project Settings → CI/CD → Runners page. (Note: you need to refresh the Runner page to see it updated).
Note: The runner must be able to make outbound HTTPS requests to gitlab.com:443. On OCI Portal, make sure your instance’s public subnet has enabled outbound HTTPS in the security list. Do the same for your VM’s firewall rules.

Define Deploy job in .gitlab-ci.yml

Add a deploy_to_vm job to the deploy stage. This job will target our local runner using at least one tags that matches one of the runner’s tags. The script section would directly execute the deployment commands, which are

login to the registry, similar to the build job
pull the image, similar to the push in build
if the local docker daemon is running a container with the same name, stop and remove it.
run the container with port mapping using the pulled image

# ... (previous stages and build_image job) ...
deploy_to_vm:
  stage: deploy
  tags: [ "oci", "deploy" ]   # Assign this job to your VM runner with these tags
  script:
    - echo "Deploying directly on the VM via GitLab Runner..."
    # Log in to the registry directly from the VM's runner environment
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    - APP_NAME="django_hello"
    - docker pull "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - docker stop APP_NAME || true
    - docker rm APP_NAME || true
    - docker run -d --name APP_NAME -p 7777:8000 "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - echo "Deployment complete for $APP_NAME!"

Complete `.gitlab-ci.yml` Pipeline:

stages:
  - build
  - deploy

build_image:
  stage: build
  image: docker:git
  services:
  - docker:dind
  script:
    - echo "Logging into GitLab Container Registry..."
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - echo "Building Docker image with commit SHA and latest tags..."
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -t $CI_REGISTRY_IMAGE:latest .
    - echo "Pushing Docker images to Container Registry..."
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  rules:
      # job runs if commit to main or master branch
      - if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "master"'
      # if any of these paths changed
        changes:
        - Dockerfile
        - requirements.txt
        - helloworld/**  
        - web/**  
      # allow Manual trigger fallback 
      - when: manual

# Job to deploy the Docker image to the Linux VM

deploy_to_vm:
     stage: deploy
     tags: [ "oci", "deploy" ]   # Assign this job to your VM runner with these tags
     script:
       - echo "Deploying directly on the VM via GitLab Runner..."
       # Log in to the registry directly from the VM's runner environment
       - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
       - APP_NAME="django_hello"
       - docker pull "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
       - docker stop APP_NAME || true
       - docker rm APP_NAME || true
       - docker run -d --name APP_NAME -p 7777:8000 "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
       - echo "Deployment complete for $APP_NAME!"

Appendix: Common tasks

Rerun a past pipeline

If you just want to repeat a failed run, go to the Pipelines list, click the pipeline ID, and hit Retry to re-execute the pipeline. Alternatively, hit the Refresh icon of a job to run a single job only.

rerun

Edit a private runner’s setting

You can edit the runner’s settings in 2 ways

in the VM, by sudo nano /etc/gitlab-runner/config.toml
in Gitlab’s Project Settings → CI/CD → Runners page, by clicking the pencil edit button next to the runner
- Can edit tags and descriptions
- Lock to current project: Yes, so the runner can’t be used elsewhere.
- Run untagged jobs: Off, if you only want tagged jobs (safer).
- Protected: On, if you only want it to run on protected branches/tags (e.g., main).
- Privileged (docker executor only): Required for Docker‑in‑Docker builds.