Azure && Azure DevOps - Provisioning a Windows Virtual Machine using Terraform

azure azuredevops terraform cloud virtualmachine IaC pipeline devops

The intention of this post is to explain how to implement Azure DevOps Pipelines, which are capable to provision (and to destroy) a Windows Virtual Machine in Azure using Terraform.

Pipeline-Job-Provision-03

1. Introduction

Provisioning resources in the cloud is (or will be) part of the everyday work of probably each DevOps or Cloud Engineer, if your company would like to provide cloud solutions. Using infrastructure as code tools like Terraform make it easier to achieve that goal, but of course you don’t want to do that in a manual way - you probably also would like to integrate that tasks in DevOps platform tools like the Azure DevOps Server of Microsoft. In this post, I’d like to explain how to set up dedicated pipelines for doing the provisioning - in that case of a (Windows) virtual machine - and the destruction.

2. Overview

There are several services, which are mandatory for that implementation, let’s start with an overview:

3. The implementation of the Azure DevOps Pipelines

Let’s go through the list, step by step, for getting the implementation done.

3.1 The Azure DevOps Agent

Ensure, that you’ve an Agent ready in one of your Agent Pools, which are capable of applying Terraform configurations.

Agent

I’ve prepared a Linux Container (Terraform and the Azure CLI installed), running it on my machine and registering it in my “TestPool” with the random Agent Name “64a80fa7e409”. That’s just for the demonstration, for real productive environments I’d of course suggest to host the Container at dedicated Kubernetes cluster. Ensure that an Agent is registered and ready, you can also deploy the Azure DevOps Pipeline Service on your machine and in addition installing Terraform - but as I’ve already created a Container including the prerequisites, I prefer to run the Container.

3.2 The Storage Account including a Container

Terraform uses State Files, to determine the state of the resource, so if it is e.g.: provisioned or already destroyed. If you provision the resource in a manual way from your machine, than everything is fine, as the place from which you’re triggering the Terraform commands is always the same. This won’t work for pipelines - imagine you trigger a job for a pipline on Agent 1, which provisions the resource and after some time, you decide to destroy the resource - therefore you’re going to trigger the dedicated pipeline for doing the destruction - but what if not Agent 1 is choosen for running the job? An Agent which is not Agent 1 won’t contain the mandatory State Files, for determining the current state of the resource. In that case, Terraform could report that there is nothing available to destroy. Therefore, it is a good idea to find a unique spot for the State Files - that would be the Container, which is included in a Storage Account.

For that, I’ve created a Container named “virtualmachine” inside my Storage Account “patricksdemostorage”, including the key “vm.state”.

Storage-Account

The related snippet of the configuration can be seen below:

  backend "azurerm" {
    resource_group_name  = "patricksdemostorage"
    storage_account_name = "patricksdemostorage"
    container_name       = "virtualmachine"
    key                  = "vm.state"
  }

The Container and the backend configuration ensures that the State Files are stored and accessed at an unique spot only. So, it doesn’t matter whether you’re provisioning the resource from your machine and the destruction is triggered by an Azure DevOps pipeline or vice versa.

3.3 The Service Connection

You need to get access from your Azure DevOps Server to your Azure Subscription. To enable that, a Service Connection can help. I’ve named the Service-Connection “PrivateSubscription”. Doing a manual setup would include to enter mandatory information, e.g.: the Tenant ID. Setting up that kind of Service Connection was very easy - just choose the correct type - “Azure Resource Manager” - afterwards you can choose your subscription of choice - the necessary settings will be conducted in an automated way.

Service-Connection

3.4 The Azure Repo

I’m versioning the Terraform configuration (.tf file) in an Azure Repo, named “IaC”. Before running the pipelines, you probably would like to try the configuration in a manual way. If the file is already versioned, ensure to add statements to the gitignore file, that no State Files are going to be tracked.

Git-Repo

3.5 The Configuration

The configuration, which will be used for provisioning the virtual machine can be seen below. Please see my previous post if you’d like to get more information about the manual way of applying Terraform commands - patrickkoch.dev - Azure/Terraform Prerequisites. In contrast to the configuration used without provisioning from Azure DevOps pipelines, the current configuration contains the mentioned backend part, refering to the Storage Account, respectively the Container. So, applying that configuration leads to a running Windows Virtual Machine.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.65"
    }

    random = {
      source  = "hashicorp/random"
      version = "3.1.0"
    }
  }

  backend "azurerm" {
    resource_group_name  = "patricksdemostorage"
    storage_account_name = "patricksdemostorage"
    container_name       = "virtualmachine"
    key                  = "vm.state"
  }

  required_version = ">= 0.14.9"
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "germanywestcentral"
}

resource "azurerm_virtual_network" "example" {
  name                = "example-network"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
}

resource "azurerm_subnet" "example" {
  name                 = "internal"
  resource_group_name  = azurerm_resource_group.example.name
  virtual_network_name = azurerm_virtual_network.example.name
  address_prefixes     = ["10.0.2.0/24"]
}

resource "azurerm_network_interface" "example" {
  name                = "example-nic"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.example.id
    private_ip_address_allocation = "Dynamic"

  # new line to add
  public_ip_address_id = azurerm_public_ip.example.id 
  }
}

# new to add
resource "azurerm_public_ip" "example" {
  name                = "acceptanceTestPublicIp1"
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location
  allocation_method   = "Dynamic"

  tags = {
    environment = "Production"
  }
}

resource "azurerm_windows_virtual_machine" "example" {
  name                = "example-machine"
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location
  size                = "Standard_F2"
  admin_username      = "adminuser"
  admin_password      = "P@$$w0rd1234!"
  network_interface_ids = [
    azurerm_network_interface.example.id,
  ]

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2016-Datacenter"
    version   = "latest"
  }
}

3.6 The Pipelines

This section explains the structure of the Azure DevOps pipelines, which are capable of provisioning and destroying the resource.

3.6.1 Pipeline for Provisioning

I’m setting up a new pipeline and refer to the mentioned Azure Repo as Source for getting the Terraform configuration.

Pipeline_01_Provision_GetSources

Two Terraform tasks complete the pipeline. The first task takes care of the Terraform - “Init” command.

Pipeline_02_Provision_Init

Please note, that the Service Connection “PrivateSubscription” is used. In addition, this task contains information about the dedicated Storage Account, respectively Container which should be used for accessing the State Files.

Pipeline_03_Provision_Init

Compare those settings with the mentioned snippet related to the backend configuration of the whole Terraform configuration file:

  backend "azurerm" {
    resource_group_name  = "patricksdemostorage"
    storage_account_name = "patricksdemostorage"
    container_name       = "virtualmachine"
    key                  = "vm.state"
  }

The next command to be integrated should conduct the “Apply”:

That’s less effort than the “Init” task - just choose “validate and apply” and point to the specific directory containing the Terraform configuration file - which is in my case “Terraform/Azure/virtualMachineWindows”.

Pipeline_05_Provision_Apply

3.6.2 Pipeline for Destruction

Of course, you’d also like to destroy the resource - therefore I’ve set up a second pipeline. Again, the sources are located in the “IaC” Azure Repo:

Pipeline_01_Provision_GetSources

Same as for the previous pipeline: two Terraform tasks are integrated in the pipeline for performing the destruction. The first is again the implementation of the “Init” command - the connection to the backend.

Pipeline_03_Destroy_Init
Pipeline_04_Destroy_Init

The next task is about the implementation of the “Destroy” command. Select “destroy” at the “Command” section and again refer to the “Private Subscription”, so that the State Files can be updated.

Pipeline_06_Destroy_Apply

4. Provisioning and Destruction of the Resource with Azure DevOps Pipelines

This section is about running the jobs by triggering the pipelines. Let’s start with the pipeline for conducting the provisioning.

4.1 Provisioning of the Resource with Azure DevOps Pipeline

The pipeline named “Terraform - Provision Azure Virtual Machine” is now triggered …

Pipeline-Job-Provision-01
Pipeline-Job-Provision-02

…you should recognize the same logs as you’d conduct the Terraform commands in a manual way…

Pipeline-Job-Provision-03
Pipeline-Job-Provision-04

The job of the pipelien succeeded … This results in a running virtual machine, hosted in the Azure cloud.

AzurePortal_VM_Running
AzurePortal_VM_Running_Details

That’s it, the virtual machine was created and is ready for deploying your apps!

4.2 Destruction of the Resource with Azure DevOps Pipeline

Of course, you’d like to destroy the virtual machine if you don’t need it. Let’s trigger the pipeline named “Terraform - Destroy Azure Virtual Machine”…

Pipeline-Job-Destroy-01
Pipeline-Job-Destroy-02
Pipeline-Job-Destroy-03
Pipeline-Job-Destroy-04

After finishing the job, the virtual machine will be removed from the Azure cloud.

Let’s prove the State File during the destruction process and after finishing the job:

During the destruction process (conducting the Terraform destroy command) - the state is “leased”.

Blob-Container-Leased_State_During_Deletion_Process

After finishing the job, the stage get’s back to “Available”.

Blob-Container-Available_State_After_Deletion_Process

Troubleshooting - Permission for Service Principal

If you’re facing an error message like in the snippet below, then you have to provide your Service Principal dedicated permission for creating cloud resources.

The client '4ff**********************035' with object id '4ff**********************035' does not have authorization to perform action 'Microsoft.Resources/subscriptions/resourcegroups/read' over scope '/subscriptions
AzureDevOps_SP_NoPermission

5. Conclusion

Integrating the conduction of Terraform commands for provisioning resources in the (Azure) cloud is very efficient. There are three things, which I’d like to highlight:

Pipeline_04_Provision_Apply

So, IMHO Azure DevOps Server harmonises well with Terraform.

References

patrickkoch.dev - Azure - Provisioning a Windows Virtual Machine using Terraform

Terraform - Intro

Microsoft - Azure: Create free Azure account

Microsoft - Azure: Install Azure CLI

Terraform - Azure Get Started

Hashicorp - Terraform Recommended Guidelines

GitHub - Azure: Azure CLI List Locations

Hashicorp - azurerm - Windows Virtual Machine

Hashicorp - azurerm - public ip