Terraform - Azure Managed Disks with For Each

We have a use case with Azure where we want to support the ability to add, remove or resize additional data disks on a VM. This is simple to do if I am manually writing the VM configuration with resource blocks, but I want to be able to just pass a list of disks I want to attach as a variable to a module that will create all of them for me.

The most common examples for this that I see are using the Count function. But the major drawback that count has over for_each is that if one of the disks is modified or removed, it removes all the disks and recreates them. Essentially, count is operating on the list of disks as a whole instead of considering each one to be a separate object.

The difficulty with dynamically attaching disks in Azure #

There is a one part of the Azure provider that makes this process more difficult. To add a data disk to a VM it’s actually two separate resource blocks; the first is used to create the disk, and the second is a data disk attachment which adds the disk to the VM. The data disk attachment also requires a unique number to be specified as the LUN on the VM.

I don’t want the disk attachment to be destroyed and recreated if a disk is removed, so I needed a method for mapping the disks to the LUNs dynamically that would also ensure the LUNs would not change. This required a little creativity with local variables, borrowing some ideas from Neal Shah.

The other little bit of complexity I had was that I had to be able to account for the fact that there may be no disks supplied, which required a little modification to the final configuration.

Variable Inputs and Local Variables #

I had the luxury of being able to adjust the variables being passed into the module, so to make my life easier in the LUN mapping the input variables I’m using for the data disk(s) that need to be attached are in a terraform map format. Each key is an index number (1 based instead of 0 based) of the disk, and thats mapped to the value which is the size of the disk that needs to be created. So if I need three disks, sized 50, 100, and 80 my disk input map would look like:

# Sample data_disks input variable:

data_disks = {
    0 = 50
    1 = 100
    2 = 80
}

Or since we’re using a tfvars.json file:

{
    "data_disks" : {
        "1" : 50,
        "2" : 100,
        "3" : 80
    }
}

Next we need to do some work in a locals block in the terraform configuration to create variables that can be used in our loops. First, I create a list of map objects, where each map contains a datadisk_name and lun keys, mapped to the corresponding values from our input:

# Generating local.lun_map:

lun_map           = [for key, value in var.data_disks : {
    datadisk_name = format("${var.vm_name}-DSK-%02d", key)
    lun           = tonumber(key)
}]

This gives us a local.lun_map variable that looks like:

# Sample local.lun_map variable:

lun_map = [
    {datadisk_name="VMNAME-DSK-01",lun=1},
    {datadisk_name="VMNAME-DSK-02",lun=2},
    {datadisk_name="VMNAME-DSK-03",lun=3},
]

Next I iterate through the lun_map and create a new map object that just maps the name of the disks to their LUN value:

# Generating local.luns:

luns = { for k in local.lun_map : k.datadisk_name => k.lun}

Which creates the local.luns variable that looks like:

# Sample local.luns variable:

luns = {
    VMNAME-DSK-01 = 1
    VMNAME-DSK-02 = 2
    VMNAME-DSK-03 = 3
}

So we end up with three variables, var.data_disks, local.lun_map, and local.luns. This may seem convoluted at this point but you’ll see everything come together as we’re generating the disks and disk attachments.

Creating the resource blocks with for_each #

So now that we have the extra local variables we need, we can build the managed disk and the disk attachment blocks using our for_each meta-argument. As you can see, I want to name my disk objects in azure with the name of the VM they are associated with followed by DSK and a two-digit number. So the format("${var.vm_name}-DSK-%02d", each.key) function below is creating this name.

First is the managed_disk block, the for_each block can use either a map or a set of strings as input. In our case, because the disk object is already a map we can use it for the loop. We’re using the key (the indexed disk number) to name the disk object, and then the value is the size of the disk so we can use that directly. You can see these referenced in the each.key and each.value references below:

resource "azurerm_managed_disk" "managed_disks" {
    for_each = var.data_disks

    name                 = format("${var.vm_name}-DSK-%02d", each.key)
    disk_size_gb         = each.value
    resource_group_name  = "my_resource_group"
    location             = "eastus"
    storage_account_type = "Premium_LRS"
    create_option        = "Emtpy"
}

Then using the local.luns variable we built earlier we can attach each disk. Notice that we can reference each managed disk by using managed_disks[each.key], and then in order to map the correct lun to each disk, we just use the lookup function on the luns variable we create to assign the lun based on the disk name we generate with our format function:

resource "azurerm_virtual_machine_data_disk_attachment" "disk_attach" {
  for_each = azurerm_managed_disk.managed_disks

  virtual_machine_id = var.vm_id
  managed_disk_id    = azurerm_managed_disk.managed_disks[each.key].id
  caching            = "ReadWrite"
  lun                = lookup(local.luns, format("${var.vm_name}-DSK-%02d", each.key))
}

And then if needed, you can use a regular for loop to generate a map of your disk ids as outputs:

output "vm_managed_disk_ids" {
  description = "The id(s) of the data disks attached to the VM"
  value       = {
      for k, disk-id in azurerm_managed_disk.managed_disks : k => disk-id.id
  }
}

Final configuration #

I mentioned this earlier on, but as it’s currently configured your configuration could error out if you want to build a VM with no additional disks. This can be fixed by reassigning the data_disks variable locally with an if statement, and then adding another on the disks_attach resource block in case there are no managed_disks created.

This is my final config for the example:

locals {
  data_disks = var.vm_data_disks == null ? {} : var.vm_data_disks

  lun_map           = [for key, value in local.data_disks : {
      datadisk_name = format("${var.vm_name}-DSK-%02d", key)
      lun           = tonumber(key)
    }]

  luns = { for k in local.lun_map : k.datadisk_name => k.lun}
}

resource "azurerm_managed_disk" "managed_disks" {
    for_each = var.data_disks

    name                 = format("${var.vm_name}-DSK-%02d", each.key)
    disk_size_gb         = each.value
    resource_group_name  = "my_resource_group"
    location             = "eastus"
    storage_account_type = "Premium_LRS"
    create_option        = "Emtpy"
}

resource "azurerm_virtual_machine_data_disk_attachment" "disk_attach" {
  for_each = length(azurerm_managed_disk.managed_disks) < 1 ? {} : azurerm_managed_disk.managed_disks

  virtual_machine_id = var.vm_id
  managed_disk_id    = azurerm_managed_disk.managed_disks[each.key].id
  caching            = "ReadWrite"
  lun                = lookup(local.luns, format("${var.vm_name}-DSK-%02d", each.key))
}

Additional Notes #

After some additional testing with different configurations, it looks like you can get away with changing the LUN number on a disk after it’s created and attached. This does destroy and recreate the disk_attachment object, but this appears to be transparent if you are connected to the VM when it occurs. The VM does not restart and no data is lost on the disk.

This means you could just dynamically assign the LUN using a dynamically generated index number if you need to, though in my use case I prefer them not to change.