Showing posts with label VMAX AF. Show all posts
Showing posts with label VMAX AF. Show all posts

Wednesday, November 29, 2017

Storage Automation: A Great Adventure - Jack Out of the Matrix



Welcome back to our adventure in Storage Automation using APIs! This post will be focusing on taking the Output from our Python Script and throwing it out to Slack. I chose Slack because it is a good way of communicating to the whole team involved, but feel free to adjust the script to use whatever tool works best for your team.

Now... Lets find a way to escape the Rabbit Hole with our Report in tow!

Below is the script that I came up with by building off of the original which was created in my last  blog (Downloadable version of the script at my GitHub):

1:   #------------------------------------------------------------------------------  
2:   #Imports modules to be used within the script  
3:   #------------------------------------------------------------------------------  
4:   import json  
5:   import requests  
6:    
7:   #Allows the API Call to Authenticate with username/password  
8:   from requests.auth import HTTPBasicAuth  
9:    
10:  #Allows you to ignore the Security warning associated with unsecured certificates  
11:  from requests.packages.urllib3.exceptions import InsecureRequestWarning  
12:  requests.packages.urllib3.disable_warnings(InsecureRequestWarning)  
13:  #------------------------------------------------------------------------------  
14:    
15:  #------------------------------------------------------------------------------  
16:  #Build Variables to be used within the VMAX API Call  
17:  #------------------------------------------------------------------------------  
18:  username="<User ID>"  
19:  password="<Password>"  
20:  vmax_ip="<VMAX IP>"  
21:  vmax_sid="<VMAX SID>"  
22:  number = 0  
23:    
24:  #Calls for all Volumes which are true tdevs (customer facing)  
25:  #NOTE: Use the specific Univmax version API Calls that you have installed on the VMAX  
26:  #(Ex: I use '83' API Calls because we are running UniVMAX 8.3)  
27:  url = 'https://' + vmax_ip + ':8443/univmax/restapi/83/sloprovisioning/symmetrix/' + vmax_sid + '/volume?tdev=true'  
28:    
29:  #Initialize WebHook URL to Post to Slack (The Key is given when setting up the WebHook Integration in Slack)  
30:  webhook_url = 'https://hooks.slack.com/services/XXXXXXXXX/YYYYYYYYY/ZZZZZZZZZZZZZZZZZZZZZZZZ'  
31:    
32:  headers = {'content-type': 'application/json', 'accept': 'application/json'}  
33:  verifySSL=False  
34:  #------------------------------------------------------------------------------  
35:    
36:  #------------------------------------------------------------------------------  
37:  #Build the Session to make the API call to the VMAX  
38:  #------------------------------------------------------------------------------  
39:  session = requests.session()  
40:  session.headers = headers  
41:  session.auth = HTTPBasicAuth(username, password)  
42:  session.verify = verifySSL  
43:  #------------------------------------------------------------------------------  
44:    
45:  #------------------------------------------------------------------------------  
46:  #Make a GET request to the VMAX for a list of LUNs/TDEVs  
47:  #------------------------------------------------------------------------------  
48:  lun_id_get = session.request('GET', url=url, timeout=60).json()  
49:  lun_list = lun_id_get.get('resultList')  
50:    
51:  #Send Complete Message to Slack  
52:  slack_message = "----------------------------------------------------- \n " \  
53:                  "Getting Started! \n" \  
54:                  "-----------------------------------------------------"  
55:    
56:  #Loop thru each of the LUNs and pull the relevant data for reporting purposes  
57:  for i in lun_list.get('result'):  
58:    lun_id=i.get('volumeId')  
59:    print ('-----------------------------------------------------------------')  
60:    print ('Volume ID: ' + lun_id)  
61:    print ('-----------------------------------------------------------------')  
62:    
63:    # Grab each LUNs relevant data for reporting purposes  
64:    url = 'https://' + vmax_ip + ':8443/univmax/restapi/83/sloprovisioning/symmetrix/' + vmax_sid + '/volume/' + lun_id  
65:    response = session.request('GET', url=url, timeout=60)  
66:    data = response.json()  
67:    lun_name = data['volume'][0]['volume_identifier']  
68:    lun_cap = data['volume'][0]['cap_gb']  
69:    lun_used_pct = data['volume'][0]['allocated_percent']  
70:    lun_used_cap = (lun_cap * lun_used_pct) / 100  
71:    print('LUN Name: ' + lun_name)  
72:    print('LUN Capacity: ' + str(lun_cap))  
73:    print('LUN % Used: ' + str(lun_used_pct))  
74:    print('LUN Used Capacity: ' + str(lun_used_cap))  
75:    print('-----------------------------------------------------------------')  
76:    print('')  
77:    
78:    #Only report on LUNs that are over 50% Full  
79:    if lun_used_pct >= 50:  
80:      #Setup WebHook Variables to make the call and post results to Slack Channel  
81:      slack_message = slack_message + "\n" \  
82:                      "######################\n" \  
83:                      "LUN Name: " + lun_name + "\n" \  
84:                      "LUN Capacity: " + str(lun_cap) + "\n" \  
85:                      "LUN % Used: " + str(lun_used_pct) + "\n" \  
86:                      "LUN Used Capacity: " + str(lun_used_cap) + "\n" \  
87:                      "######################\n"  
88:    
89:    #Keep Track of Number of LUNs processed  
90:    number += 1  
91:  #------------------------------------------------------------------------------  
92:    
93:  #Add Complete Message to Slack  
94:  slack_message = slack_message + "\nAll is done and Looks good! :thumbsup: \n " \  
95:                  "-----------------------------------------------------"  
96:    
97:  #------------------------------------------------------------------------------  
98:  #Make the Call to Post results to the SlackOps Slack Channel  
99:  #------------------------------------------------------------------------------  
100:  slack_data = {'text': slack_message}  
101:    
102:  response = requests.post(  
103:    webhook_url, data=json.dumps(slack_data),  
104:    headers={'Content-Type': 'application/json'}  
105:  )  
106:    
107:  if response.status_code != 200:  
108:    raise ValueError(  
109:      'Request to slack returned an error %s, the response is:\n%s'  
110:      % (response.status_code, response.text)  
111:    )  
112:  #------------------------------------------------------------------------------  


As you walk through the script, you will see the backbone is still the same as the previous version with added logic to send the created report to Slack.


  • Adjusted the Script to only report on LUNs that are over 50% Utilized (So as not to spam the team too much)
  • Added webhook logic to send a built message out to a Slack Community Channel
  • Added an Error Catch at the end of the Slack WebHook Call just in case

What's up Next?



    This is the end of the road for this Mini Blog Series on VMAX Storage Automation using Python and API Calls!
    We were able to escape The Matrix with a nice report in Slack in hand and ready for our team!
    I will be working on another Mini Series for the EMC XIO Gen2 in the near future following these same basic principles, so stay tuned!


    Previous Blogs in the Series




      Tools Used in the Making of this Episode



        • PyCharm: Python IDE that seems to have favor of the community
        • PyU4V Project: GUI for building API Calls to the VMAX

        Special Thanks


        • devStepsiz: For helping me figure out how to post a JSON Payload to Slack

        Thursday, October 26, 2017

        Storage Automation: A Great Adventure - Entering the Matrix



        Welcome back to our adventure in Storage Automation using APIs! This post will be focusing on getting the Python Script in place so we can eventually build out some nice reports.

        Since you were willing to take the Red Pill and weren't scared off by the after math, Let's keep digging in!

        Below is the script that I came up with by building off of the original which was created in my last step of the process (Downloadable version of the script at my GitHub):

        1:  #------------------------------------------------------------------------------  
        2:  #Imports modules to be used within the script  
        3:  #------------------------------------------------------------------------------  
        4:  #Allows API Calls to be made to the VMAX  
        5:  import requests  
        6:    
        7:  #Allows the API Call to Authenticate with username/password  
        8:  from requests.auth import HTTPBasicAuth  
        9:    
        10:  #Allows you to ignore the Security warning associated with unsecured certificates  
        11:  from requests.packages.urllib3.exceptions import InsecureRequestWarning  
        12:  requests.packages.urllib3.disable_warnings(InsecureRequestWarning)  
        13:  #------------------------------------------------------------------------------  
        14:    
        15:  #------------------------------------------------------------------------------  
        16:  #Build Variables to be used within the API Call  
        17:  #------------------------------------------------------------------------------  
        18:  username="<User ID>"  
        19:  password="<Password>"  
        20:    
        21:  #Calls for all Volumes which are true tdevs (customer facing)  
        22:  #NOTE: Use the specific Univmax version API Calls that you have installed on the VMAX  
        23:  #(Ex: I use '83' API Calls because we are running UniVMAX 8.3)  
        24:  url = 'https://<VMAX IP>:8443/univmax/restapi/83/sloprovisioning/symmetrix/<VMAX SID>/volume?tdev=true'  
        25:    
        26:  headers = {'content-type': 'application/json',  
        27:        'accept': 'application/json'}  
        28:  verifySSL=False  
        29:  #------------------------------------------------------------------------------  
        30:    
        31:  #------------------------------------------------------------------------------  
        32:  #Build the Session to make the API call to the VMAX  
        33:  #------------------------------------------------------------------------------  
        34:  session = requests.session()  
        35:  session.headers = headers  
        36:  session.auth = HTTPBasicAuth(username, password)  
        37:  session.verify = verifySSL  
        38:  #------------------------------------------------------------------------------  
        39:    
        40:  #------------------------------------------------------------------------------  
        41:  #Make a GET request to the VMAX for a list of LUNs/TDEVs  
        42:  #------------------------------------------------------------------------------  
        43:  lun_id_get = session.request('GET', url=url, timeout=60).json()  
        44:  lun_list = lun_id_get.get('resultList')  
        45:    
        46:  #Loop thru each of the LUNs and pull the relevant data for reporting purposes  
        47:  for i in lun_list.get('result'):  
        48:    lun_id=i.get('volumeId')  
        49:    print ('-----------------------------------------------------------------')  
        50:    print ('Volume ID: ' + lun_id)  
        51:    print ('-----------------------------------------------------------------')  
        52:    
        53:    # Grab each LUNs relevant data for reporting purposes  
        54:    url = 'https://<VMAX IP>:8443/univmax/restapi/83/sloprovisioning/symmetrix/<VMAX SID>/volume/' + lun_id  
        55:    response = session.request('GET', url=url, timeout=60)  
        56:    data = response.json()  
        57:    lun_name = data['volume'][0]['volume_identifier']  
        58:    lun_cap = data['volume'][0]['cap_gb']  
        59:    lun_used = data['volume'][0]['allocated_percent']  
        60:    print('LUN Name: ' + lun_name)  
        61:    print('LUN Capacity: ' + str(lun_cap))  
        62:    print('LUN Used: ' + str(lun_used))  
        63:    print('-----------------------------------------------------------------')  
        64:    print('')  
        65:  #------------------------------------------------------------------------------  
        


        As you walk through the script, you will see the backbone is still the same as the previous version but with a bit of a twist.


        • Adjusted my URL call to use the specific version of UniVMAX (8.3) as to match up exactly what the array is expected to see. 
        • Added the "tdev=true" filter to the API URL to only return the LUNs that I need to see, and not the backend LUNs behind the scenes.
        • I added a Loop, which allows us to take the list of LUNs/TDEVs returned from the array and pull specific data we would like to report on.
          • Volume Name
          • Volume Max Capacity
          • Volume Allocated Percent
          • etc...

        What's up Next?



          Now that we have the desired data that we can play with, our next step is to take that data and create a nice and pretty report to show off to your co-workers and friends. I'm hoping to tie a report into Slack and post it within a Slack Channel.


          Previous Blogs in the Series




            Tools Used in the Making of this Episode



              • PyCharm: Python IDE that seems to have favor of the community
              • PyU4V Project: GUI for building API Calls to the VMAX

              Special Thanks


              • Paul Martin (@rawstorage) for helping to continue down this rabbit hole