A couple months ago one of our customers migrated their production environment to a new Cloud provider. They still needed live access to portions of their legacy system, but wanted downsize and decommission a subset of about 500+ Virtual Machines to save cost. The process of decommissioning/resizing each VM was relatively easy but extremely time consuming as it required multiple clicks and waits on the Cloud management portal.
I was horrified to learn that a project had been put together which would’ve involved a team of people to sit 9am – 5pm maintenance slots to manually shut down, resize and turn VM’s back on.
I’d previously used the Cloud providers API to upgrade VMware tools on all of the VMs so I figured I could the same and complete the clients request single-handedly by writing some scripts and complete the job in a couple hours instead of days.
I wanted to write this post to share the details and logic of my scripts. You can read the actual scripts in my GitHub repository mcp-bash-api-script-repo. To get the full explanation of what’s going on continue reading…
According to the Cloud API documentation I would need to perform a POST action to specific paths on their API endpoints. Included in that request I would need to provide the Organisation ID, authentication details and the unique ID of the Virtual Machine in json format as part of the data payload.
In my script I would use curl on the command line to interact with the API. This is what the command would look like to shutdown a single server:
curl -X POST \
'https://api-eu.example.com/caas/2.10/00000000-4321-xxxx-1234-abcdefg12345/server/shutdownServer' \
-H 'Content-Type: application/json' \
-H 'Accept: /' \
-H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQK' \
-d '{
"id": "abc123de-1ab1-1234-92f8-987654abcdef"
}'
Now seeing as though I would be running this against multiple machines in two different regions, I would need to abstract portions of the command so that it could be used in a loop. I could simply export shell variables for the regions API entry point, the Organisation ID and the Virtual Machine ID. The script would start to look something like this:
export ORGID='00000000-4321-xxxx-1234-abcdefg12345'
export REGION='https://api-eu.example.com/'
export VMID='abc123de-1ab1-1234-92f8-987654abcdef'
curl -X POST \
$REGION/caas/2.10/$ORGID/server/shutdownServer \
-H 'Content-Type: application/json' \
-H 'Accept: /' \
-H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQK' \
-d '{
"id": "'$VMID'"
}'
The client had provided a spreadsheet of all the changes they wanted made to each VM, so I could use that as the input to my script. The spreadsheet only contained the friendly name of the VM, but the API expected the unique ID of the VM. Luckily I could use the API to query the unique ID of the machine and then pass that as a variable for the rest of the script.
To do that I would run a for loop which selected the second column of the spreadsheet to dynamically set a variable named VMNAME. I then ran that through a sub command which would GET the unique ID of the VM from the API and export it as variable VMID. In order to get just the value of VMID from the API response, I would need to filter, format and transform it using a combination of grep, xmllint and awk.
export ORGID='00000000-4321-xxxx-1234-abcdefg12345'
export REGION='https://api-eu.example.com/'
export INVENTORY='2_UKServerFormatted.csv'
for VMNAME in $( cat $INVENTORY | grep -v ToDoAction | awk -F, '{print $2}' )
do export VMID=$(curl -s -X GET $REGION/caas/2.10/$ORGID/server/server?name=$VMNAME -H 'Accept: /' -H 'Accept-Encoding: gzip, deflate' -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQK' -H 'Cache-Control: no-cache' -H 'Connection: keep-alive' -H 'cache-control: no-cache' | xmllint --format - | grep 'server id' | awk -F'"' '{print $2}' )
done
Within the same loop I would print a simple message telling me what action was about to performed on which machine. Finally I could combine it all to end up with a script which would cycle through the 500+ lines in the spreadsheet and generate the desired action to send to the API. It ended up looking something like this:
#!/bin/bash
#
#
#Set your Variables Here Shuping!
export ORGID='00000000-4321-xxxx-1234-abcdefg12345'
export REGION='https://api-eu.example.com/'
export INVENTORY='2_UKServerFormatted.csv'
#End of Variables Section
#
for VMNAME in $( cat $INVENTORY | grep -v ToDoAction | awk -F, '{print $2}' )
do export VMID=$(curl -s -X GET $REGION/caas/2.10/$ORGID/server/server?name=$VMNAME -H 'Accept: /' -H 'Accept-Encoding: gzip, deflate' -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQK' -H 'Cache-Control: no-cache' -H 'Connection: keep-alive' -H 'cache-control: no-cache' | xmllint --format - | grep 'server id' | awk -F'"' '{print $2}' )
echo shutting down $VMNAME
curl -X POST \
$REGION/caas/2.10/$ORGID/server/shutdownServer \
-H 'Accept: /' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQK' \
-H 'Cache-Control: no-cache' \
-H 'Connection: keep-alive' \
-H 'Content-Type: application/json' \
-H 'cache-control: no-cache' \
-d '{
"id": "'$VMID'"
}'
done
I used the same blueprint to update the spec of Virtual Machines in the resize.sh script and to start up all eligible machines in the startup.sh script. It took me about 3 hours to research and write all the scripts. Total execution of all the scripts was run in under 3 hours.
