The Road to Terraform and Azure Resource Provisioning

By all accounts Microsoft has experienced somewhat of a sea change in terms of both public perception and innovation in recent years. It is unlikely that the two are unrelated and one would suspect the effect is a virtuous circle that is self-reinforcing.

Over the past year I've invested time expanding my skills and knowledge of Azure, and more recently, Azure Stack. Although the new Azure Portal is quite usable I prefer taking an "infrastructure as code" approach whenever possible, especially when learning requires repeated build and tear down of Azure resources.

The approach I took is opinionated given the pre-existing tools and software management approaches I prefer and my client platform of choice. If your situation differs drastically these steps may be partly or wholly inapplicable. The basics include:

  • macOS 10.12
  • Terraform 0.9.2 for resource provisioning
  • Homebrew 1.1.2 for package management
  • Anaconda 4.3.1 for Python package and virtual environment management

Azure-CLI Installation

Start by creating a new conda environment that we'll use when working with the Azure-CLI:

$ conda create -n azure python=3.5
$ source activate azure

Next, install the azure-cli package from PyPI using pip:

$ pip install azure-cli

Confirm that Azure-CLI is working properly:

$ az --version
azure-cli (2.0.2)

acr (2.0.0)  
acs (2.0.2)  
appservice (0.1.2)  
batch (2.0.0)  
...output truncated...

Terraform Installation

One of the prerequisites for using Terraform with Azure Resource Manager is jq, a command-line JSON processor, so we'll install it next using Homebrew:

$ brew install jq

Confirm the installation is successful and that the executable binary is accessible in your shell path:

$ jq --version
jq-1.5  

Next we'll install the terraform binary using Homebrew.

$ brew install terraform

Confirm the installation is successful and that the executable binary is accessible in your shell path:

$ terraform --version
Terraform v0.9.2  

Login Using Azure-CLI for the First Time

Authenticate with the Azure API using az for the first time:

$ az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code ABCD1A1AB to authenticate.  

You'll be prompted to access https://aka.ms/devicelogin and enter the code displayed. Once you enter the code and authenticate with the Microsoft login service, the details of your Azure subscription should be displayed.

Note: The GUIDs displayed throughout this post aren't valid.

[
  {
    "cloudName": "AzureCloud",
    "id": "dcbcfcde-0933-42fe-bb1a-4148cad4886a",
    "isDefault": true,
    "name": "subscription-anem",
    "state": "Enabled",
    "tenantId": "6eb0d8d5-dd4b-4afa-9efe-0e00f81fde56",
    "user": {
      "name": "user@domain.com",
      "type": "user"
    }
  }
]

Set the Azure Subscription

If you only have a single subscription you can proceed. If you have multiple subscriptions use az account list to display them and then az account set --subscription <SUBSCRIPTION_ID> to set which one should be used for this exercise.

Securely record the id and tenantId information of the subscription you plan to use. Note that this id value will be referred to later as subscription_id in Terraform configuration files.

Create an App Registration with Azure AD

Azure requires that an application is added to Azure Active Directory to generate the values needed by Terraform. In order to do this you need to create a new Service Principal and grant it permissions to the Application Registration in your Azure Subscription.

$ az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/<SUBSCRIPTION_ID>"

The resulting output should look something like the following:

{
  "appId": "0346a5ed-11aa-4d01-a68c-6bf867b99f9f",
  "displayName": "azure-cli-YYYY-MM-DD-HH-MM-SS",
  "name": "http://azure-cli-YYYY-MM-DD-HH-MM-SS",
  "password": "b9d95348-866f-4964-918d-4d0491e61e7a",
  "tenant": "6eb0d8d5-dd4b-4afa-9efe-0e00f81fde56"
}

Terraform will use several of these values, so you'll again need to store the output in a secure location. Note that Terraform uses different names for these values in its configuration so take note of the mapping (Terraform Name => Azure Name):

  • client_id => name or appId (can be used interchangeably)
  • client_secret => password
  • tenant_id => tenant

The name or appId will be used later for the servicePrincipalProfile.servicePrincipalClientId, and the password is used for servicePrincipalProfile.servicePrincipalClientSecret. Terraform uses these credentials to authenticate with the ARM API.

Confirm you can authenticate using the Service Principal credentials by opening a new shell and running the following commands, substitute name or appId for <username>, password for <password>, and tenantId for <tenant>:

$ az login --service-principal -u <username> -p <password> --tenant <tenant>
$ az vm list-sizes --location eastus

Simple Example

To verify Terraform is able to manage resources for your Azure subscription we'll try to create a new empty resource group. Before beginning, choose an alpha-numeric name for the resource group you are about to create, which cannot already exist within your subscription.

In your text editor of choice, create a file called azure_create_rg.tf within an otherwise empty directory. The exact name of the file is not important, but note that all scripts in the directory will be executed. Paste the following code in that new file, replace the subscription_id, client_id, client_secret, and tenant_id values, and save.

# Configure the Microsoft Azure Provider
provider "azurerm" {  
  subscription_id = "..."
  client_id       = "..."
  client_secret   = "..."
  tenant_id       = "..."
}

# Create a resource group
resource "azurerm_resource_group" "production" {  
  name     = "production"
  location = "East US"
}

In the provider section, you tell Terraform to use the Azure ARM API provider to provision resources. The azure_rm_resource_group resource instructs Terraform to create a new resource group with the name and in the location specified.

Test Run

One of the great features of Terraform is that you can ask Terraform to read your script and report what the expected outcome will be without making any changes. In order to see what Terraform will do with our admittedly simple request run the following from a terminal with the azure conda environment activated:

$ terraform plan

It should output something similar to:

Refreshing Terraform state in-memory prior to plan...  
The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage.

The Terraform execution plan has been generated and is shown below. Resources are shown in alphabetical order for quick scanning. Green resources will be created (or destroyed and then created if an existing resource exists), yellow resources are being changed in-place, and red resources will be destroyed. Cyan entries are data sources to be read.

+ azurerm_resource_group.production
    location: "eastus"
    name:     "production"
    tags.%:   "<computed>"


Plan: 1 to add, 0 to change, 0 to destroy.  

If everything looks correct, provision this new resource group in Azure by executing the following:

$ terraform apply

If you examine the Azure portal now (https://portal.azure.com), you should see a new empty resource group called production.

A terraform.tfstate file will be created in the same directory which contains information dynamically obtained during the last Terraform run. You can see the contents by executing the following:

$ terraform show

In this simply example it simply displays the resource group details we just created.

azurerm_resource_group.production:  
  id = /subscriptions/dcbcfcde-0933-42fe-bb1a-4148cad4886a/resourceGroups/production
  location = eastus
  name = production
  tags.% = 0

Next Steps

At this point you should have all of the basics needed to provision Azure resources using Terraform. If you are new to Terraform I'd recommend walking through the official Getting Started Guide.