Home - About me - Browse by categories

How to: Use Terraform to deploy Azure Kubernetes Service in Custom VNET with Kubenet

Few months ago, I have written this post that explains how to deploy an Azure Kubernetes Service cluster inside a custom virtual network with the Kubenet plugin, instead of AzureCNI.

Note: The AKS docs has also been updated with this scenario, here.

In this new post, I describe all what you need to know/do to get the same result, but fully automated using Terraform :-)

There is a GitHub repository with everything here.

This repository contains all you need to use Terraform to deploy Azure Kubernetes Service with Kubenet plugin, inside a custom VNET.

It automatically creates:

How it works

All the AKS cluster definition is in the tf/aks.tf file. Some of the parameters are variable that can be overriden in the tf/variables.tf file.

When deploying Azure Kubernetes Service with Kubenet plugin inside a custom virtual network, there are additional steps required to attach the Network Security Group and Route Table to the subnet where the node are deployed.

This has been automated with the tf/config-network.sh script that is called by a Terraform provisionner after the AKS cluster has been created:

provisioner "local-exec" {
    command = "./config-network.sh"

    environment {
        AKS_RG = "${var.resource_group_name}"
        AKS_VNET_RG = "${var.resource_group_name}"
        AKS_VNET_NAME = "${azurerm_virtual_network.vnet.name}"
        AKS_SUBNET_NAME = "${azurerm_subnet.subnet.name}"
    }
}

NB: if you want to know more about those additional steps, read this page of the AKS docs.

For the same reason, there are custom actions required when you want to remove the AKS cluster. You need to detach the NSG and Route Table before. This has also been implemented in this repository, using a destroy provisionner that will execute the tf/clean-network.sh script before removing the AKS resource:

provisioner "local-exec" {
    when = "destroy"
    command = "./clean-network.sh"

    environment {
        AKS_VNET_RG = "${var.resource_group_name}"
        AKS_VNET_NAME = "${azurerm_virtual_network.vnet.name}"
        AKS_SUBNET_NAME = "${azurerm_subnet.subnet.name}"
    }
}

How to deploy

You need to have Terraform installed and Azure CLI 2.0 installed, obviously.

Go to the tf directory:

cd tf

Optional: update the variables.tf and aks.tf files with desired values.

Export the following environment variables for the service principal client id and client secret that should be used by the Azure Kubernetes Service cluster:

export TF_VAR_client_secret=YOUR_CLIENT_SECRET
export TF_VAR_client_id=YOUR_CLIENT_ID

Initialize Terraform

terraform init

Plan the deployment:

terraform plan -out out.plan

Apply the plan to start the deployment:

terraform apply "out.plan"

Wait for the deployment to be completed. Once done, go in the Azure portal, open the subnet where you’ve deployed the AKS cluster and check that the AKS NSG and Route Table have been assigned to it:

Subnet with Route Table and NSG

How to destroy

Go to the tf directory:

cd tf

Call Terraform destroy:

terraform destroy

Hope this helps!


Any question about this post? Feel free to drop a comment below or contact me on Twitter @jcorioland