Did you know about the Opslogix DevOps Upskill Program? Through the program, skillful IT consultants improve their DevOps knowledge through a combination of theoretical and practical training.
In this new blog series, our DevOps consultant Dennis will share some of the insights he has gained through the program.
Keep on reading this first part of the blog series to learn more about Dennis' insights on Kubernetes cluster orchestration with Ansible and Terraform on Proxmox VE.
In the fast-paced world of DevOps and container orchestration, deploying a Kubernetes cluster efficiently is a crucial step toward building scalable and resilient applications. In this blog post, we'll explore a project that leverages the power of Ansible and Terraform to automate the deployment of a Kubernetes cluster on Proxmox VE. This combination of tools not only simplifies and speeds up the process but also ensures consistency and repeatability in the deployment.
The resulting GitHub-project containing Ansible and Terraform code can be found here.
As a DevOps practitioner, Kubernetes is a central part of my toolbox. I have previously been using k3s as my primary home lab Kubernetes cluster, which is a lightweight version of Kubernetes that comes with some required components preinstalled, that you otherwise have to chose and install manually. Until recently the cluster worked fine, but during an upgrade I got an unexpected behavior which made me look for the control of software that standard Kubernetes provides.
Because this is a home lab environment, things get broken all the time and a quick way to restore things is essential in order to keep things going. As in production, a way to keep track of infrastructural changes is also vital in order to maintain a understanding of the environment as a whole, in order to protect it from things like malicious actors or bad configurations. While I do keep rigorous backups, I wanted a way to spin up, or down, my cluster (or several) with the push of a button in order to reset it - i.e. automation! That's why I looked to the tools Ansible and Terraform solve my problem.
Ansible and Terraform are two popular open-source tools known for their infrastructure-as-code (IaC) capabilities. Ansible excels in configuration management, allowing for seamless automation of tasks such as package installation, configuration file management, and service orchestration. Terraform, on the other hand, specializes in provisioning and managing infrastructure resources across various cloud providers and virtualization platforms.
By combining Ansible and Terraform, developers and operations teams can create a powerful synergy that addresses both configuration and provisioning aspects of infrastructure management.
Proxmox Virtual Environment (Proxmox VE) is an open-source virtualization platform that combines two virtualization technologies: KVM (Kernel-based Virtual Machine) for virtual machines and LXC (Linux Containers) for lightweight container-based virtualization. Proxmox provides a user-friendly web interface and powerful command-line tools for managing virtualized environments.
For this project, I am using Opslogix's test environment consisting of 3 Proxmox nodes that are ready to go. If you want to get started with Proxmox, take a look here.
Terraform scripts are used to define the infrastructure requirements for the Kubernetes cluster. This includes virtual machines, networks, storage, and any other resources necessary for the cluster. The Terraform configuration files, written in HashiCorp Configuration Language (HCL), provide a declarative syntax for specifying the desired state of the infrastructure.
In the below code snippet from the main Terraform file, we declare how our Virtual Machines should be provisioned - e.g. how much resources to give, the host name, IP addresses, and which template to use.
resource "proxmox_vm_qemu" "proxmox_kubernetes_vms" {
count = var.x_number_of
name = "${var.vm_name}-${count.index + 1}"
ipconfig0 = "gw=${var.ip_gateway},ip=${var.ip_addresses[count.index]}"
target_node = "proxmox_hostname_or_ip"
onboot = true
# Same CPU as the Physical host, possible to add cpu flags
# Ex: "host,flags=+md-clear;+pcid;+spec-ctrl;+ssbd;+pdpe1gb"
cpu = "host"
numa = false
clone = "${var.template_vm_name}"
full_clone = true
os_type = "cloud-init"
agent = 1
ciuser = var.user
memory = var.memory
...Once the infrastructure is declared in Terraform, Ansible takes over for actual deployment. Ansible playbooks use YAML (Yet Another Markup Language) to define tasks to be performed and in which sequence.
In the snippet below from the Ansible host provisioning file, we can see how Ansible will create the Terraform plan and then apply it in sequence.
- name: Creating k8s VMs Terraform plan
community.general.terraform:
project_path: ""
state: planned
plan_file: "/k8s_hosts.tfplan"
force_init: true
- name: Confirm terraform plan is present
ansible.builtin.wait_for:
path: "/k8s_hosts.tfplan"
state: present
- name: Apply terraform plan
community.general.terraform:
project_path: ""
plan_file: "/k8s_hosts.tfplan"
state: present
register: outputsWith the infrastructure provisioned and nodes configured, the final step is deploying Kubernetes. Ansible playbooks can be used to install Kubernetes components, initialize the cluster, and join nodes to the cluster.
In the snippet below, the Ansible initialize cluster file creates the cluster and saves the join command, which is printed at the end of the cluster creation, to use for joining the worker nodes to the cluster.
- name: Initialize cluster
ansible.builtin.command: "kubeadm init --pod-network-cidr="
register: result
changed_when: "result.rc == 0"
when: "reset_cluster is succeeded"
- name: Run script to add kubeconfig
ansible.builtin.script: add_kubeconfig.sh
- name: Get the token for joining the worker nodes
ansible.builtin.command: kubeadm token create --print-join-command
register: kubernetes_join_command
changed_when: false
failed_when: "kubernetes_join_command.rc != 0"In conclusion, the project showcases the power of combining Ansible and Terraform to automate the deployment of a Kubernetes cluster on a Proxmox hypervisor. This approach not only streamlines the deployment process but also enhances the reliability and scalability of the infrastructure. By adopting infrastructure as code practices, development, and operations teams can achieve greater efficiency and agility in managing complex systems.
Do you want to learn more about the Opslogix DevOps Upskill Program?