Automating Kubernetes Deployment from GitLab CI/CD to a K3s Cluster: A Complete Guide

In this comprehensive tutorial, we walk through the complete process of building a scalable and reusable GitLab CI/CD pipeline for deploying containerized applications to a K3s Kubernetes cluster. This article covers server roles, setup procedures, file configurations, and YAML template generalization to enable seamless deployment across multiple projects.

Overview

Our goal is to automate the following:

• Build container images using Podman
• Push images to GitLab’s built-in Container Registry
• Pull the images from a K3s cluster
• Deploy the application via Kubernetes manifests

1. Server Infrastructure Involved

1.1 GitLab Server (VM1)

• Domain: src.yellowgnu.net
• Role: Hosts GitLab, GitLab Container Registry, manages CI pipelines

1.2 GitLab Runner Host (VM2)

• Hostname: git-runners.yellowgnu.net
• Role: Executes CI/CD pipelines using Podman as the container runtime

1.3 K3s Kubernetes Node (VM3)

• Hostname: k3s-node
• Role: Lightweight Kubernetes cluster that runs deployed applications

2. Step-by-Step Server Setup

2.1 GitLab Server (VM1)

• Ensure GitLab is installed and running with Container Registry enabled.
• Create a personal access token (glpat-...) for GitLab Registry access.

2.2 GitLab Runner (VM2)

• Install GitLab Runner and register a runner:

  gitlab-runner register
  

• Install Podman:

  sudo dnf install -y podman
  

• Ensure the runner can authenticate to GitLab Registry via CI/CD variables.

2.3 K3s Node (VM3)

• Install K3s:

  curl -sfL https://get.k3s.io | sh -
  

• Create image pull secret:

  kubectl create secret docker-registry regcred \
    --docker-server=registry.src.yellowgnu.net \
    --docker-username=gitlab-ci-token \
    --docker-password=<glpat-token> \
    --docker-email=admin@example.com
  

• Create a service account and bind cluster-admin role:

  kubectl create serviceaccount gitlab-deployer
  kubectl create clusterrolebinding gitlab-deployer-binding \
    --clusterrole=cluster-admin \
    --serviceaccount=default:gitlab-deployer
  

• Extract KUBE_TOKEN and set KUBE_API_URL:

  TOKEN=$(kubectl get secret $(kubectl get sa gitlab-deployer -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 -d)
  echo $TOKEN
  kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'
  

• Store both in GitLab project variables.

3. Files Involved

3.1 .gitlab-ci.yml

Manages the CI/CD pipeline.

3.2 Dockerfile

Defines the image build configuration.

3.3 k8s/deployment-template.yaml

Deployment template with placeholders for dynamic substitution.

3.4 k8s/service-template.yaml

Service template to expose the application.

4. File Explanations

4.1 .gitlab-ci.yml

stages:
  - build
  - push
  - deploy

variables:
  IMAGE_TAG: "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
  REGISTRY: "$CI_REGISTRY"

before_script:
  - echo "$CI_JOB_TOKEN" | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY

build:
  stage: build
  script:
    - podman build -t $IMAGE_TAG .

push:
  stage: push
  script:
    - podman push $IMAGE_TAG

create-regcred:
  stage: deploy
  script:
    - echo "Creating imagePullSecret on K8s"
    - kubectl delete secret regcred || true
    - kubectl create secret docker-registry regcred \
        --docker-server=$CI_REGISTRY \
        --docker-username=gitlab-ci-token \
        --docker-password=$CI_JOB_TOKEN \
        --docker-email=admin@example.com

kube-deploy:
  stage: deploy
  script:
    - echo "$KUBE_TOKEN" > token.txt
    - kubectl config set-cluster k3s-cluster --server="$KUBE_API_URL" --insecure-skip-tls-verify=true
    - kubectl config set-credentials gitlab-deployer --token=$(cat token.txt)
    - kubectl config set-context default --cluster=k3s-cluster --user=gitlab-deployer
    - kubectl config use-context default
    - export APP_NAME="$CI_PROJECT_NAME"
    - export CI_REGISTRY_IMAGE="$CI_REGISTRY/$CI_PROJECT_PATH"
    - export REGISTRY="$CI_REGISTRY_IMAGE"
    - export CI_COMMIT_SHORT_SHA="$CI_COMMIT_SHORT_SHA"
    - envsubst < k8s/deployment-template.yaml > k8s/deployment.yaml
    - envsubst < k8s/service-template.yaml > k8s/service.yaml
    - kubectl apply -f k8s/deployment.yaml
    - kubectl apply -f k8s/service.yaml

4.2 Dockerfile

FROM eclipse-temurin:17-jre
COPY target/app.jar /app/app.jar
ENTRYPOINT ["java", "-jar", "/app/app.jar"]

4.3 k8s/deployment-template.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${APP_NAME}
  labels:
    app: ${APP_NAME}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ${APP_NAME}
  template:
    metadata:
      labels:
        app: ${APP_NAME}
    spec:
      containers:
        - name: ${APP_NAME}
          image: ${REGISTRY}:${CI_COMMIT_SHORT_SHA}
          ports:
            - containerPort: 8080
      imagePullSecrets:
        - name: regcred

4.4 k8s/service-template.yaml

apiVersion: v1
kind: Service
metadata:
  name: ${APP_NAME}
spec:
  type: NodePort
  selector:
    app: ${APP_NAME}
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      nodePort: 30080

✅ Result: Seamless Multi-Project Kubernetes Deployments

Once this setup is complete:

• You can reuse the same .gitlab-ci.yml, Dockerfile, and Kubernetes templates across multiple projects.
• Each project only needs its own application logic and CI/CD variables like KUBE_API_URL and KUBE_TOKEN.
• A new image is built and deployed to the K3s cluster automatically on each push to GitLab.

🔚 Conclusion

By setting up a general-purpose GitLab CI/CD pipeline, you’ve eliminated project-specific friction and centralized your deployment process. This approach is especially valuable for teams managing multiple microservices or wanting a standardized deployment pipeline without rework.

For enterprise scalability, consider extending the solution with:

• Namespaces per project
• TLS via Ingress and cert-manager
• Helm charts for abstraction
• GitOps via ArgoCD or Flux