Wednesday, November 29, 2017

Storage Automation: A Great Adventure - Jack Out of the Matrix



Welcome back to our adventure in Storage Automation using APIs! This post will be focusing on taking the Output from our Python Script and throwing it out to Slack. I chose Slack because it is a good way of communicating to the whole team involved, but feel free to adjust the script to use whatever tool works best for your team.

Now... Lets find a way to escape the Rabbit Hole with our Report in tow!

Below is the script that I came up with by building off of the original which was created in my last  blog (Downloadable version of the script at my GitHub):

1:   #------------------------------------------------------------------------------  
2:   #Imports modules to be used within the script  
3:   #------------------------------------------------------------------------------  
4:   import json  
5:   import requests  
6:    
7:   #Allows the API Call to Authenticate with username/password  
8:   from requests.auth import HTTPBasicAuth  
9:    
10:  #Allows you to ignore the Security warning associated with unsecured certificates  
11:  from requests.packages.urllib3.exceptions import InsecureRequestWarning  
12:  requests.packages.urllib3.disable_warnings(InsecureRequestWarning)  
13:  #------------------------------------------------------------------------------  
14:    
15:  #------------------------------------------------------------------------------  
16:  #Build Variables to be used within the VMAX API Call  
17:  #------------------------------------------------------------------------------  
18:  username="<User ID>"  
19:  password="<Password>"  
20:  vmax_ip="<VMAX IP>"  
21:  vmax_sid="<VMAX SID>"  
22:  number = 0  
23:    
24:  #Calls for all Volumes which are true tdevs (customer facing)  
25:  #NOTE: Use the specific Univmax version API Calls that you have installed on the VMAX  
26:  #(Ex: I use '83' API Calls because we are running UniVMAX 8.3)  
27:  url = 'https://' + vmax_ip + ':8443/univmax/restapi/83/sloprovisioning/symmetrix/' + vmax_sid + '/volume?tdev=true'  
28:    
29:  #Initialize WebHook URL to Post to Slack (The Key is given when setting up the WebHook Integration in Slack)  
30:  webhook_url = 'https://hooks.slack.com/services/XXXXXXXXX/YYYYYYYYY/ZZZZZZZZZZZZZZZZZZZZZZZZ'  
31:    
32:  headers = {'content-type': 'application/json', 'accept': 'application/json'}  
33:  verifySSL=False  
34:  #------------------------------------------------------------------------------  
35:    
36:  #------------------------------------------------------------------------------  
37:  #Build the Session to make the API call to the VMAX  
38:  #------------------------------------------------------------------------------  
39:  session = requests.session()  
40:  session.headers = headers  
41:  session.auth = HTTPBasicAuth(username, password)  
42:  session.verify = verifySSL  
43:  #------------------------------------------------------------------------------  
44:    
45:  #------------------------------------------------------------------------------  
46:  #Make a GET request to the VMAX for a list of LUNs/TDEVs  
47:  #------------------------------------------------------------------------------  
48:  lun_id_get = session.request('GET', url=url, timeout=60).json()  
49:  lun_list = lun_id_get.get('resultList')  
50:    
51:  #Send Complete Message to Slack  
52:  slack_message = "----------------------------------------------------- \n " \  
53:                  "Getting Started! \n" \  
54:                  "-----------------------------------------------------"  
55:    
56:  #Loop thru each of the LUNs and pull the relevant data for reporting purposes  
57:  for i in lun_list.get('result'):  
58:    lun_id=i.get('volumeId')  
59:    print ('-----------------------------------------------------------------')  
60:    print ('Volume ID: ' + lun_id)  
61:    print ('-----------------------------------------------------------------')  
62:    
63:    # Grab each LUNs relevant data for reporting purposes  
64:    url = 'https://' + vmax_ip + ':8443/univmax/restapi/83/sloprovisioning/symmetrix/' + vmax_sid + '/volume/' + lun_id  
65:    response = session.request('GET', url=url, timeout=60)  
66:    data = response.json()  
67:    lun_name = data['volume'][0]['volume_identifier']  
68:    lun_cap = data['volume'][0]['cap_gb']  
69:    lun_used_pct = data['volume'][0]['allocated_percent']  
70:    lun_used_cap = (lun_cap * lun_used_pct) / 100  
71:    print('LUN Name: ' + lun_name)  
72:    print('LUN Capacity: ' + str(lun_cap))  
73:    print('LUN % Used: ' + str(lun_used_pct))  
74:    print('LUN Used Capacity: ' + str(lun_used_cap))  
75:    print('-----------------------------------------------------------------')  
76:    print('')  
77:    
78:    #Only report on LUNs that are over 50% Full  
79:    if lun_used_pct >= 50:  
80:      #Setup WebHook Variables to make the call and post results to Slack Channel  
81:      slack_message = slack_message + "\n" \  
82:                      "######################\n" \  
83:                      "LUN Name: " + lun_name + "\n" \  
84:                      "LUN Capacity: " + str(lun_cap) + "\n" \  
85:                      "LUN % Used: " + str(lun_used_pct) + "\n" \  
86:                      "LUN Used Capacity: " + str(lun_used_cap) + "\n" \  
87:                      "######################\n"  
88:    
89:    #Keep Track of Number of LUNs processed  
90:    number += 1  
91:  #------------------------------------------------------------------------------  
92:    
93:  #Add Complete Message to Slack  
94:  slack_message = slack_message + "\nAll is done and Looks good! :thumbsup: \n " \  
95:                  "-----------------------------------------------------"  
96:    
97:  #------------------------------------------------------------------------------  
98:  #Make the Call to Post results to the SlackOps Slack Channel  
99:  #------------------------------------------------------------------------------  
100:  slack_data = {'text': slack_message}  
101:    
102:  response = requests.post(  
103:    webhook_url, data=json.dumps(slack_data),  
104:    headers={'Content-Type': 'application/json'}  
105:  )  
106:    
107:  if response.status_code != 200:  
108:    raise ValueError(  
109:      'Request to slack returned an error %s, the response is:\n%s'  
110:      % (response.status_code, response.text)  
111:    )  
112:  #------------------------------------------------------------------------------  


As you walk through the script, you will see the backbone is still the same as the previous version with added logic to send the created report to Slack.


  • Adjusted the Script to only report on LUNs that are over 50% Utilized (So as not to spam the team too much)
  • Added webhook logic to send a built message out to a Slack Community Channel
  • Added an Error Catch at the end of the Slack WebHook Call just in case

What's up Next?



    This is the end of the road for this Mini Blog Series on VMAX Storage Automation using Python and API Calls!
    We were able to escape The Matrix with a nice report in Slack in hand and ready for our team!
    I will be working on another Mini Series for the EMC XIO Gen2 in the near future following these same basic principles, so stay tuned!


    Previous Blogs in the Series




      Tools Used in the Making of this Episode



        • PyCharm: Python IDE that seems to have favor of the community
        • PyU4V Project: GUI for building API Calls to the VMAX

        Special Thanks


        • devStepsiz: For helping me figure out how to post a JSON Payload to Slack