Automated Test Deployments in Azure with Ansible, Azure DevOps
Test Environment Automation Concepts
Across many customer environments, where an advanced testing process is in place, stands the issue of deploying non-production environments. There are different variations of non-production environments – some of them less comparable to the production setup than others. Some are meant as almost exact copies even resourced the same as the production workloads in order to mimic as much as possible the business journey through production.
All of these environments and especially the last type, often labelled as Pre-Production or Golden Environments, pose a number of challenges to the deployment teams. They often go out of sync from production, pose risk with bearing critical data or their refreshes take a massive amount of time, which sometimes proves them unusable.
In that regard, one of the best possible ways, to tackle the challenge, is automating the process. If successful an automatic deployment of test environment within the cloud and especially Skytap on Azure and Azure Native, could provide years of stability to the testing teams. It could easily give them the means to provision the entire non-production environment as obfuscated copy of Production within a few hours.
How is that possible?
Utilizing Cloud Deployments
Obviously, we have to say a few words on one of the undisputable advantages that the cloud services provide – flexible API services. Realizing that this is not a new concept, it must be pointed out as often customers or partners underestimate the power of automated deployment within the cloud. As this is particularly focused around Azure Native and Skytap on Azure, Terraform, Azure DevOps and Ansible will be highlighted as the main instruments.
Cloud presents an opportunity for these deployments; however, it often can prove to be complex so one of our greatest responsibilities is to: design, document and optimize. This is especially true when working with automated provisioning as after a difficult start, it can start feeling easy and unnecessary to optimize further. This, however, is where the burden of costs starts to become really heavy and optimization is key.
As all complex processes, improvement while deploying and learning from mistakes is the basis of it. Documentation will help understand better why you did things in the first place, so it is absolutely crucial to the success.
But, let’s dive a bit deeper.
Islands of workloads
When talking about automatic deployment of test environment you have to think about the concepts:
- What is the scope of the test environment?
- How do you protect production data?
- How do resource the environments?
- How often do you refresh these?
- What is the granularity of deployments?
- All other infrastructural concepts – networking, storage, backup, monitoring, etc.
The islands of workloads means that even if your testing environment is an almost exact copy of Production, it still is treated as an environment in isolation. Most probably it will not:
- Have the same users that access it
- Have any contact to production environments
- Have the same SLA as production
Having said that one must think about how to automate two very important processes:
- Remove production users and deploy test ones
- Perform data masking on all sensitive data
Automation Deployment Basis
What we usually see and use within these projects are some form of combination between three main tools – Azure DevOps with its Pipelines, Terraform with its simple deployment logic and Ansible with its capabilities to configure AIX operating systems among all other standard operating systems.
In a lot of cases, customers do not take direct copies of production workloads but either take copies from DR systems or have a dedicated transition environment that is synched from production. This allows them to not affect production when having to deploy non-production.
Let’s provide an example.
A customer has an environment of 20 AIX LPARs with Oracle databases on them and 15 RHEL application servers all in Skytap. Together with them it has a small farm of Windows application servers based in Azure Native. Now – the APIs of Skytap and Azure Native are obviously incomparable, however, with Ansible API hits Skytap can start looking really flexible for deployment. Azure Native obviously will support a number of re-configurations especially for Windows servers.
However, what about the Oracle DBs on top of the AIX? This is your most important aspect – how to ensure that you are copying a consistent database when deploying the test environment? Usually that happens by either stopping the production database or putting it in “hot backup” mode, both of which WILL affect it work.
That is why some customers choose to have intermediate environments, synced from prod, that will provide that necessary copy. That of course requires additional support and funding.
Deploying in Skytap
In Skytap, deployments of entire LPARs or VMs have long been supported with something called templating. The templating is basically snapshot on storage level, however, performed for the entire virtual server. This easily provides the opportunity for customers to copy even the entire environment with a number of virtual servers in it and deploy it somewhere else.
Of course, it is far more complex than it sounds. Deploying the templates is just a step within a long process of designing resources, networking, backup, monitoring, etc. Even if you take a consistent copy of your environments and deploy it, you still have to think about:
- Configuration of network interfaces
- Configuration of resources – CPU, memory, disk
- Configuration of clusters if any – fun starts when we start talking about deploying POWERHA or Oracle RAC environments
- Re-naming file systems
- Re-naming DBs
- Working with host files
It is a process that you have to carefully design and implement mostly using Azure DevOps pipelines and variables as well as Ansible workers and environment variable there. In Skytap Terraform is hardly usable as the module is undeveloped.
One important aspect of Skytap is how the service handles environments. If you are deploying test servers into a single test environment in Skytap, you have to bear in mind that this environment will be busy a lot, so you will not be able to deploy servers in parallel. You either use multiple test environments (maybe as a copy of prod) or you make sure that you have enough time to wait for last deployment to happen before you start a new one.
Deploying in Azure
Azure Native comes with a lot of functionality for automatic deployment and a really developed Terraform module (at least from the past). That will allow you to easily snapshot disks, use scripts on the servers, re-configure resources and this all can happen in parallel. There are a lot of pitfalls of course when you automate deployments there as well, however, you will have a number of instrumental options to work around them.
Just make sure you design your networking as robust as possible and everything else will come to place.
Here the most important part is to optimize. You will easily loose the thread on the cost for both Skytap and especially Azure Native if you are not careful. Make sure to always know why you are deploying something with certain resources. At least in Terraform you have to describe resources for everything so it is hard to not think about it, unless you only copy and paste code from somewhere.
We come to the end of this small guide we tried to provide on mostly how to think about automatic deployments of test environments in Azure and Skytap on Azure. If you have a project, that you need some guidance on, here is how we can help:
- Design and provision Azure DevOps
- Integrate Terraform and Ansible
- Design and deploy landing zones
- Design and deploy intermediate zones
- Design and deploy test environment automations
Please contact us here.
