Tips to take your Azure Resource Manager (ARM) template game to the next level

ARM templates are a great tool for deploying, updating, and deleting resources in Azure. In this blog, Sam Cogan (@samcogan) gives an ARM template tutorial to help you better understand how, where, and why to use this Azure service.

If you're deploying resources in Azure, and you're looking to move beyond using the portal, then chances are you’ve looked at Microsoft's tooling for declaratively creating resources, Azure Resource Manager (ARM) Templates.

Using ARM Templates, you can define your Azure Infrastructure-as-Code using the JSON-like language that can be understood by the ARM API. This allows you to build a library of infrastructure for you and your team.

Once you've been working with ARM templates for a while, you pick up time-saving tricks and learn some more advanced techniques. These can help make your life easier by helping you work around issues or allowing you to do more with your templates. In this article, I'll take a look at some of these techniques that you might find useful as you work with ARM templates.

ShareGate has been helping IT professionals succeed in the Microsoft cloud for over a decade. We hope this blog helps you better understand ARM Templates—but if you’re looking for a tool to make managing Azure even easier, check out how ShareGate Overcast can give you better visibility and control in Azure.

To see me go through some of these tips step by step, you can watch the recording of my presentation at Deploy, ShareGate’s expert-led event on Azure governance.

How to use ARM template functions

The ARM template language isn't just JSON, it also provides a wide range of functions that you can use to help make your templates more dynamic. ARM template functions allow you to define complicated expressions in your template that you don’t want to have to continually repeat.

There are functions to do mathematical and comparison operations, to retrieve information about your deployment, and to manipulate strings, arrays, and objects. Microsoft offers a full list of functions supported in ARM. It's a great idea to get familiar with these functions and what they can do.

A simple example is using the concatenate function to join two parameters together to form one string:

A more complex function like "list" allows you to generate a SaS token for a storage account you have just created: 

In addition to the built-in functions, you can also create your own functions through a feature called User Defined Functions (UDF). UDFs allow you to create your own functions that are constructed using the built-in functions.

So you can combine built-in fuctions to create a new one, but there isn’t an option yet to create brand new logic that you couldn't achieve without existing functions. UDFs are primarily useful for simplifying situations where you are chaining lots of built-in functions together and you want to hide that behind a simple interface.

The power of nested ARM templates

Did you know you can deploy ARM templates from ARM templates? Nested templates allow you to have your top-level template trigger another template to run and pass parameters and outputs between them.

A nested template is just another ARM resource where you can specify the location of another template you want to call. For example:

I've added more examples of nested templates in this github.

There are a lot of different reasons to use nested templates, especially as your organization grows and you’re creating more complex deployments.

Modularize and reuse your ARM templates 

If you write a lot of ARM templates, you'll often find you’re creating the same resources over and over again. Usually, this means not only creating the resource, but configuring it in a certain way that’s standard for your organization.

For example, maybe you always deploy Azure SQL servers with the firewall locked down to just your company IPs and you always enable threat detection and always-on encryption. Not only is that how you deploy Azure SQL servers, it’s how you want everyone else in your company to do it, too.

With nested templates, you can create a library of modules that do specific tasks, and then call these from your top-level template.

You can store these modules in a Github or storage account and allow your teammates to call them from their templates. Maybe they'll even find ways to improve them!

Bigger isn't always better—simplify your ARM templates

If you’re deploying a lot of resources, putting all of them in one big template can make things very confusing. It makes it difficult to see what you’re implementing, where the dependencies are, and so on.

By breaking your template into separate files and linking them together using a top-level template, they’ll be much easier to understand. And it’ll make updating them more manageable for you and your team.

Sometimes you need to use nested templates to achieve some functionality. For example, if you have a template where you need to deploy some of the resources into a different resource group or subscription from the one you are running the template against. In this scenario, you need to use a nested template to achieve this.

If you want to deploy your resources at different scopes, then you also need to look at using a nested template.

The four scopes where you can deploy ARM templates

As well as different modes, ARM templates now support different scopes. It used to be that ARM templates could only be deployed at a single scope: the resource group. All the resources you defined inside a template would be created inside the resource group you used as part of your deployment command.

Today, however, we can create templates at four different scopes:

• Resource groups: This is the original scope and where most resources sit. Using this scope creates your resources in a specific resource group. In PowerShell, this uses the New-AzResourceGroupDeployment command.
• Subscriptions: Some resources need to be created at the subscription scope, with the obvious one being resource groups. This uses a new PowerShell command called New-AzSubscriptionDeployment command.
• Management groups: If you want to deploy resources that apply to multiple subscriptions, like custom roles or RBAC permissions, then you can use the management group scope. This uses the New-AzManagementGroupDeployment command.
• Tenants: If you need a custom role to be available to the whole tenant, then you can use New-AzTenantDeployment command.

Through the use of nested templates, you can even combine these scopes in a single deployment.

If you’re looking for more details on how to deploy to specific scopes, this is an example of creating a resource group with a template and then deploying resources into it.

ARM template deployment modes

When you deploy ARM templates, there are two modes you can run your deployment in. The one that's used by default, and that nearly everyone uses nearly every time, is called "incremental mode." But there’s also "complete mode."

If you're deploying your template into an empty resource group, then you won't notice a difference. But when you use an existing resource group, you will.

When you deploy a template in incremental mode, it will do the following:

• Create any mentioned resources that do not exist in the resource group.
• Update any existing resources that have a different configuration from what’s in the template.
  • *This is only possible where the required properties to be changed support this.
• Ignore any resources that exist that are not part of the template.

When you use complete mode, however, it changes what happens to existing resources that are not part of the template. So, if you deploy in complete mode, your resource group will have exactly like what is in your template—nothing more or less.

If there are resources in the resource group that are not in your template then they will get deleted. This makes complete mode more dangerous than incremental mode, especially if you have people manually creating and editing resources outside of your template, so use it with caution.

Complete mode is best suited to situations where you want to make sure your resource group reflects precisely what is in your template and where no one is supposed to be making manual changes.

Best practice for testing ARM templates

Now that we're in a world where our infrastructure is defined as code, it means we can test it like code, even before we've deployed a single VM. By undertaking this testing, we can catch issues and errors before we deploy costly resources and before problems get into production where they are much more challenging to fix.

There are many different testing tools and frameworks out there that could be used for testing ARM templates. My personal favourite is Pester, the PowerShell testing framework.

We don't have time today to discuss Pester in great detail, but if you want to learn more about it, then you can find all the details in this github.

Testing your templates really falls into two areas: 

1. Testing the templates before deployment 
2. Testing the resources that your templates deploy

In an ideal world, we would test as much as possible before we run the template. This would save us time and money. Unfortunately, if you want to be able to check that what your template is deploying is what you think it’s deploying, sometimes you simply need to deploy it.

The first set of testing I would recommend looking at is something produced by Microsoft called the ARM Template Test Kit (ARM TTK). Using the TTK, you can run a pre-created set of tests against your templates to determine if they are following best practices.

These tests will pick up things like whether you re using the latest schema, that all required parameters exist, and that you don't have parameters that are not in use. You can download and run it against your templates using PowerShell and Pester.

If you use Azure DevOps, I’ve also created an extension to run this as part of your pipelines.

Next, we can look at more custom tests that will run before you deploy anything. These tests allow us to check the content of the template and pick up issues like:

• Invalid JSON: Is any of the language in the template invalid?
• Missing files: Do you have any missing template or parameter files?
• Schema Validation: Does your template have all the required sections and properties?
• Content Validation: Does your template actually have resources for all the things you’re expecting to be deployed?
• Template Validation: Does your template pass the checks in the Test-AzResourceGroupDeployment command?

All of these items can be tested using Pester before your template is deployed. This github offers some examples of how to run Pester tests. You can run your Pester tests locally, or as part of your CI/CD pipeline when you check your changes. Hopefully, they all come back green.

Here’s an example of our results after checking our template against best practices and validating the content we want to check is deployed where we’re expecting it to be.

At this point, we should be ready to deploy the infrastructure into Azure.

Once we’ve done that we can write more Pester tests that retrieve the details of the infrastructure from Azure and use them to compare this against our expectations.

For example, here's a test that checks a virtual network to confirm that it exists with the right name, has the correct subnets, and uses the right address range.

You can find more examples of testing infrastructure in this github.

While ARM templates often offer a simple solution to repeatedly deploying resources in Azure, as your business grows and your cloud infrastructure evolves, things can get more complicated—fast.

The best way to learn is to practice writing, testing, and deploying templates. The second best way is to learn from others who have practiced a lot.