Multi-Cloud Kubernetes Cluster Setup with Automation
In this article, we will create a Multi-Cloud Kubernetes Cluster (Master node on AWS EC2 Instance and a Slave Node on Azure VM) using the Ansible Roles. Terraform is used to create the entire Cloud Infrastructure required for this setup. In my previous articles, I’ve explained about Ansible Roles for single cloud setup of K8s cluster. We are going to use the same roles for multi cloud setup also.
Infrastructure Setup
We are using Terraform for creating an AWS EC2 instance with Amazon Linux2 AMI and an Azure VM with all it’s associated backend with an Ubuntu image.
First you need to install the AWS CLI and Azure CLI in your machine and the following commands to setup :
aws configure
az loginCreate a folder for terraform and write the following in providers.tf file :
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "4.22.0"
}
azurerm = {
source = "hashicorp/azurerm"
version = "3.14.0"
}
}
}provider "aws" {
region = "ap-south-1"
shared_credentials_files = ["/home/sagar/.aws/credentials"]
profile = "default"
}provider "azurerm" {
features {}
subscription_id = "YOUR_SUBSCRIPTION_ID"
tenant_id = "YOUR_TENANT_ID"
client_id = "YOUR_CLIENT_ID"
client_secret = "YOUR_CLIENT_SECRET"
}
Now create a file variables.tf and copy the following :
# AWS Variables
variable "aws_ami_id" {
type = string
}
variable "aws_instance_type" {
type = string
}
variable "aws_key_name" {
type = string
}
variable "aws_key_location" {
type = string
}# Azure Variables
variable "location" {
type = string
}
variable "vm_size" {
type = string
}
variable "admin_username" {
type = string
}
variable "azure_private_key" {
type = string
}
variable "azure_public_key" {
type = string
}
After that create another file with name terraform.tfvars :
# AWS Variables
aws_ami_id = "ami-08df646e18b182346"
aws_instance_type = "t2.micro"
aws_key_name = "YOUR_AWS_KEY_NAME"
aws_key_location = "/home/sagar/Downloads/YOUR_KEY_NAME.pem"# Azure Variables
location = "Central India"
vm_size = "Standard_D2s_v3"
admin_username = "adminuser"
azure_private_key = "/home/YOUR_USERNAME/.ssh/id_rsa"
azure_public_key = "/home/YOUR_USERNAME/.ssh/id_rsa.pub"
Note : For AWS, create a new key and download the key to your computer. For azure, you need to create a private and public key with the command :
ssh-keygenNow create a file aws.tf and copy the following :
resource "aws_default_vpc" "default" {
tags = {
Name = "Default VPC"
}
}resource "aws_security_group" "terr_aws_sg" {
name = "terr_sg"
vpc_id = aws_default_vpc.default.idingress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
}resource "aws_instance" "terr_aws_instance" {
ami = var.aws_ami_id
instance_type = var.aws_instance_type
key_name = var.aws_key_name
vpc_security_group_ids = [aws_security_group.terr_aws_sg.id]
tags = {
Name = "Master"
}
}resource "null_resource" "master_ip" {
depends_on = [
aws_instance.terr_aws_instance
]
provisioner "local-exec" {
command = "echo \"[master_node]\n${aws_instance.terr_aws_instance.public_ip} ansible_ssh_private_key_file=${var.aws_key_location} ansible_ssh_user=ec2-user\n\n[slave_nodes]\" > ../inventory.txt"
}
}
The above file is for creating an EC2 instance with default vpc and security group and allowing all traffic. At the last it copies the public ip of aws instance with the aws key location inside the inventory file which is used by the ansible roles later.
Now creater another file azure.tf and copy the following :
resource "null_resource" "depends_on_master" {
depends_on = [
null_resource.master_ip
]
}resource "azurerm_resource_group" "terr_azure_rg" {
name = "terr_rg"
location = var.location
}resource "azurerm_virtual_network" "terr_azure_vn" {
name = "terr_network"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.terr_azure_rg.location
resource_group_name = azurerm_resource_group.terr_azure_rg.name
}resource "azurerm_subnet" "terr_azure_subnet" {
name = "terr_subnet"
resource_group_name = azurerm_resource_group.terr_azure_rg.name
virtual_network_name = azurerm_virtual_network.terr_azure_vn.name
address_prefixes = ["10.0.2.0/24"]
}resource "azurerm_network_security_group" "terr_azure_sg" {
name = "azure_sg"
location = azurerm_resource_group.terr_azure_rg.location
resource_group_name = azurerm_resource_group.terr_azure_rg.namesecurity_rule {
name = "allow_all_inbound"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
security_rule {
name = "allow_all_outbound"
priority = 100
direction = "Outbound"
access = "Allow"
protocol = "*"
source_port_range = "*"
destination_port_range = "*"
source_address_prefix = "*"
destination_address_prefix = "*"
}
}resource "azurerm_subnet_network_security_group_association" "terr_azure_sgAssoc" {
subnet_id = azurerm_subnet.terr_azure_subnet.id
network_security_group_id = azurerm_network_security_group.terr_azure_sg.id
}resource "azurerm_public_ip" "terr_azure_ip" {
name = "terr_ip"
resource_group_name = azurerm_resource_group.terr_azure_rg.name
location = azurerm_resource_group.terr_azure_rg.location
allocation_method = "Static"
ip_version = "IPv4"
sku = "Standard"
}resource "azurerm_network_interface" "terr_azure_ni" {
name = "terr_ni"
location = azurerm_resource_group.terr_azure_rg.location
resource_group_name = azurerm_resource_group.terr_azure_rg.nameip_configuration {
name = "internal"
subnet_id = azurerm_subnet.terr_azure_subnet.id
private_ip_address_allocation = "Dynamic"
public_ip_address_id = azurerm_public_ip.terr_azure_ip.id
}
}resource "azurerm_linux_virtual_machine" "terr_azure_vm" {
name = "terr-vm"
resource_group_name = azurerm_resource_group.terr_azure_rg.name
location = azurerm_resource_group.terr_azure_rg.location
size = var.vm_size
admin_username = var.admin_username
network_interface_ids = [
azurerm_network_interface.terr_azure_ni.id,
]admin_ssh_key {
username = var.admin_username
public_key = file(var.azure_public_key)
}os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
}resource "null_resource" "slave_ip1" {
provisioner "local-exec" {
command = "echo \"${azurerm_linux_virtual_machine.terr_azure_vm.public_ip_address} ansible_ssh_private_key_file=${var.azure_private_key} ansible_ssh_user=${var.admin_username}\n\" >> ../inventory.txt"
}
}
The file is long because it has to create a resource group, virtual netork and a subnet inside it, security group, security rule for allowing all traffic, associate security group to subnet, public ip, network interface, and finally a Virtual Machine!
At last, the above file writes the public IP of Azure VM to the previous inventory file for ansible roles.
Now I’m not explaning again the whole ansible roles for configuring master, configuring slave, etc., You can find the roles in the GitHub link provided at the end of article.
The following is the main.yaml file which uses all the ansible roles in the order :
- hosts: localhost
gather_facts: False
tasks:
- name: "Initializing terraform providers"
shell:
cmd: terraform init
chdir: ./terraform
- name: "Provisioning 1 AWS Instance , 1 Azure VM"
community.general.terraform:
project_path: ./terraform
state: present
- meta: refresh_inventory- hosts: all
gather_facts: False
tasks:
- name: "Waiting for SSH port"
wait_for_connection:- hosts: master_node
gather_facts: False
roles:
- configure-master
tasks:
- name: "Getting token from master"
shell: "kubeadm token create --print-join-command"
register: token
- add_host:
name: "token_for_worker"
link: "{{ token['stdout'] }}"- hosts: slave_nodes
gather_facts: False
roles:
- configure-slave
tasks:
- name: "Worker Nodes joining the cluster"
shell: "{{ hostvars['token_for_worker']['link'] }}"- hosts: master_node
gather_facts: False
roles:
- wordpress-mysql
First it runs the terraform commands for the creation of entire infrastructure and later configures the master node and then the slave nodes and at last it launches the wordpress with mysql as backend in the kubernetes cluster.
GitHub link for the entire terraform files and ansible roles :
The following are the screenshots when the above main.yaml file is run :
The following is the inventory.txt file created automatically by terraform :
So, the multi cloud K8s cluster is ready. Thanks for reading.
