Showing posts with label Python. Show all posts
Showing posts with label Python. Show all posts

Wednesday, September 13, 2017

Storage Automation: A Great Adventure - Red Pill

Image result for matrix take the blue pill picture

Welcome back to our adventure down the Storage Automation (Notice I didn't use DevOps, I've noticed that the term has gotten some negative vibes as of late) using APIs! This post is going to highlight my "Major" breakthrough on an API call to the VMAX.
Now that you've decided to take the Red Pill and see how far the rabbit hole goes, Let's dive in!

I started my quest with the help of the the Unisphere for VMAX REST Client, which is a GUI that allows you to browse and experiment with the API calls available for the VMAX. Below is a screenshot of the API call that we will be diving into today (I know it's an extremely basic API call, but we have to start somewhere).


My next hurdle to clear was to figure out how to make this same call within Python and fortunately, along came Paul Martin (@rawstorage) to introduce me to the PyU4V Project, where some genius engineers at EMC took the initiative to help the community out and put in some tremendous leg work in making this leap into the VMAX API world much less daunting.

At this point, now that I had PyU4V setup, configured, & running, I wanted to take a step back and learn (at it's base) what was truly happening behind the scenes. I like to take a look under the hood and learn how something works before I just jump whole heartedly into using it because it gives me a better perspective and base to work with later down the road.

I tore apart the modules within PyU4V and with a bit of cohersing, I was able to come up with a way to make the call outside the tool and pull back the results I was wanting in a JSON format. Below is that simple Python Script/Call to accomplish the same thing we did with the RESTClient from above (GitHub Project).


This may not seem like much at the moment, but we will be taking this building block and working on creating a reporting tool which will give us much more visibility into the array, and even further down that road, maybe even taking some automated action against the array based on the input from the API call.

Thanks for sticking with me for another episode of this adventure. Let's continue to see where this takes us!

Whats up Next?



    Now that we have a simple API call to build upon, I plan to work down the route of fleshing out the Python Script to pull in all LUNs Storage Capacity & Allocated Percent and format it in a usable fashion. This will set us up for our next step of integrating it it into a monitoring tool for visualization.

    Tools Used in the Making of this Episode



      • PyCharm: Python IDE that seems to have favor of the community
      • PyU4V Project: GUI for building API Calls to the VMAX


      Special Thanks


      Thursday, August 31, 2017

      Storage DevOps: A Great Adventure - The Prelude

      Related image

      I recently came to the realization that I just CAN'T do it all. 
      The past year I have been trying to:
      • Get Field Specific Certs
        • EMC Storage
        • VMWare DCV & NSX
      • Learn Everything I can about Storage (EMC Specifically)
      • Learn all I can about Python, Perl, & several other programming languages popular in the DevOps world
      • Learn ALL I can pertaining to the DevOps way of life and the mentality behind it


      This, in turn, has left me run ragged after a year of constant 100% balls to the wall action. Due to this I had to pull away for a week or so and lay everything out on the table to re-priortize & focus my goals to be a bit more realistic and reachable (It's all about those baby steps).

      I've done some initial research and digging into what DevOps truly is and what impact it has within Storage. 
      Below is what, in my opinion & Point of View, I have unearthed:
      • Definition: Automating Tasks for a layer of hardware that would otherwise be an arduous task manually, which more than likely will also result in inconsistent results
      • 3 Steps/Layers
        • Monitor/Reporting (This is where I sit today): Using APIs/CLI to pull information out of the system and formatting that data into a usable report
          • This is the least intrusive place to start your journey into the DevOps world
        • Move/Add/Change: Using APIs/CLI to create or adjust config within the layer of hardware
          • This is your 2nd step in this crazy world because it is a little more intrusive in nature and means you need some experience and insight before implementing
        • Full Automation
          • This is the Big Daddy of them all because you are giving the hardware full reign to roll with the punches and even self heal

      As I mentioned in the list above, I have done some work within the Monitor/Reporting Layer for the last 6-8 Months. My Co-Worker and I have built a Suite of Modules that gives us some crucial insight & even predict (Yes! We have a Crystal Ball) into all of our Storage arrays across the enterprise. This work has saved our ass multiple times and has been well worth our investment in time and effort. That being said, at its' base we are using basic CLI commands to poll the arrays and there is nothing wrong with that but my goal here is to build on that foundation and move into the API side of things.

      Moving forward, I plan to focus on the more trending side of the Storage world and that is pretty obviously the Monitoring, Reporting, Move/Add/Change, & ultimately the Full Automation of Storage thru APIs (as the hook into the arrays) & Python (as the programming language to make it all happen).

      Please Join me in my adventure down the Storage DevOps Rabbit Hole and let's see what we can create.


      Special Thanks



      • Ryan Booth (@that1guy15) for helping mentor me and push me in the right direction
      • TheNetworkCollective for your PodCast on Network Automation (Inspiration)
      • Paul Martin (@rawstorage) for your in depth conversation on VMAX APIs thru Python
      • Bowie Poag for ushering me into & opening my eyes to the possibilities in Monitoring/Reporting of Storage