Category Archives: VMware

VMworld 2014 Wrap-Up

One of my favorite weeks of the year is almost complete. I’m at the airport heading home, looking forward to being back home with my family. VMworld was simply amazing this year. I have always enjoyed the experience, but the community brought the whole experience to another level.

I had some objectives to accomplish this year:

  1. Research SIEM tools
  2. DevOps integration
  3. OpenStack integration
  4. Get out of my predominantly introverted shell
  5. VCDX–do I really want to do it?

Continue reading

VMworld 2014

VMworld is coming up fast at the end of August and I’m getting pretty excited. From a professional perspective, it is the highlight of my year. This will be my 4th time attending and will be one of my busiest schedules yet.

For me, it is important to have a hotel close to the venue, the Moscone Center. In order for this to happen, a reservation needs to be made well in advance of VMworld registration, or make the reservation by registering very quickly after registration opens–I registered within a couple of hours of registration being open to get a reservation at a reasonably priced hotel in Union Square. My coworker who is attending with me ended up at a different hotel since her registration was a few hours later. One of the benefits of being a VMUG leader is that they offer perks associated with volunteer time at VMworld–one of which is getting into the Marriott right by the Moscone Center. I was able to make that switch and it will be a huge benefit. The swag that can be obtained at VMworld is second to none and being able to drop it off at the room means I can travel light during the day.

The process I use for planning my overall schedule has been refined over the last few years. Once the general schedule is released, I start blocking off my calendar with the general sessions, VMworld party, hall crawl, lunches, etc. For things like lunch, I use a different color and mark the it as available instead of busy. Where this helps is when I start to schedule sessions. I know I need to eat, so I don’t want to fill my schedule to the point of not being able to eat lunch. The 30 minutes between sessions is sufficient for lunch.

Schedule Builder is now live and the sessions are filling up fast. This is another one of those things that should be tackled as soon as possible after it is available. The best sessions can be filled up within a couple of hours. The most popular sessions might be repeated, and/or the room assignments will get adjusted to bigger rooms. Aside from my first year, I’ve had one or two focus areas, and this year is no different. At $job, we are building a long term SDDC strategy and my goal is to get a better handle on the technologies that will be part of it–OpenStack and storage are the primary areas. I build my schedule first based on Advanced Technical sessions in the focus topics, then specific speakers who have helped me with their blog posts and Twitter posts (Chris Collotti, Duncan Epping, Scott Lowe, William Lam, Rawlinson Rivera, Cormac Hogan, Chad Sakac, Jason Nash, Josh Odgers, etc), and finally, I look for other VCDXs that are presenting. The value of VMworld is that I can get the equivalent of many expensive training courses compressed into one week by listening to the right people. As a whole, I try to avoid purely vendor sessions. I’ve been to some sessions that say they won’t talk about their product and then espouse the benefits of their approach over everyone else for an hour–I can get that in the vendor hall.

The Welcome Reception is when I scope out vendors for swag and determine which ones warrant additional time talking to engineers–the challenge is getting past the sales people. I may have a management role at $job, but I also do hands on administration and design, so I want to talk to the people who develop and/or support the products we are interested in. I’m also coming at things from a different perspective this year… I want to know more about technologies that don’t apply directly to $job.

Current Session Schedule:

  • SDDC1580 – OpenStack in the Enterprise
  • STO2996-SPO – The vExpert Storage Game Show
  • STO1491 – From Clouds to Bits: Exploring the Software Defined Storage Lifecycle
  • SDDC1176 – Ask the Expert vBloggers
  • SDDC1600 – Art of IT Infrastructure Design: The Way of the VCDX – Panel
  • STO3162 – Software Defined Storage: Satisfy the Requirements of Your Application at the Granularity of a Virtual Disk with Virtual Volumes (VVols)
  • STO1965 – Virtual Volumes Technical Deep Dive
  • INF1736 – Designing Next Generation Software-Defined Data Centers: A Panel with VMware Certified Design Experts
  • STO2480 – Software Defined Storage – The VCDX Way Part II : The Empire Strikes Back
  • VAPP2305 – Extreme Performance Series – Understanding Applications that Require Extra TLC for Better Performance on vSphere – Deep Dive
  • SDDC2492 – How the New Software-defined Paradigms Will Impact Your vSphere Design
  • NET1592 – Under the Hood: Network Virtualization with OpenStack Neutron and VMware NSX
  • OPT2668 – DevOps Demystified! Proven Architectures to Support DevOps Initiatives
  • INF1552 – SDDC: Buzzword to Reality. Discussions with SDDC Architects that Goes Well Beyond Markitecture
  • HBC1534 – Recovery as a Service (RaaS) with vCloud Hybrid Service
  • STO3247 – VMware VVOL Technical Preview with Dell Storage

Other Stuff:

  • VMUG Leader Reception and Lunch
  • VCDX/vExpert Reception
  • VMworld Party
  • Welcome Reception
  • Hall Crawl
  • vBreakfast

Storage Functional Testing

I’ve spent a fair amount of time in the last couple of months testing storage. At $job, we have some very interesting datasets and workflows. These result in large bursts of intense I/O. This can lead to I/O latency issues when multiple databases attempt to write to disk at once. In the physical server world, these workloads were isolated, not impacting each other. In the virtual server world, shared storage can lead to bottlenecks that impact many systems at once. For this reason, storage selection is critical in our environment.

We have a mix of storage arrays, primarily iSCSI with some fiber channel, and a small amount of NFS. At this point, my favorite storage arrays are the Nimble Storage arrays. We have 4 of the CS240G-X4 models, two each in two data centers. The performance is stellar, analytics are awesome, and the reliability on every front is unmatched in our environment. I’ve done firmware updates under load in our DR environment with absolutely zero downtime and no real noticeable impact in performance. They just work… <shameless plug over>

Now for the main topic… when testing storage, what should we do to validate a new array, particularly one from a new-to-us vendor, and one that isn’t as mature as EMC, NetApp or HP? I deal with a VMware-based environment, so that’s my focus, but most of this directly applies regardless of environment.

I’ll outline my steps…

  1. Manual Controller Failover Testing – both quiesced and under heavy load
  2. Controller Software Upgrades – this should be done under load
  3. Simulated Drive Failures – this applied to both spinning and flash
  4. Power Supply Removal – they say it is hot swap, so test it before you have to bet your data on it
  5. Forced Controller Failure – remove one and see what happens
  6. Network Connectivity – start pulling a cable or two while under load
  7. Multipath Testing – know how things work in the recommended configuration (MRU vs ALUA) and what happens when the opposite is used
  8. Performance – iometer is very useful, not only to generate load for the aforementioned testing, but also for knowing what the limits are for performance

Storage is physical and involves a lot of interconnects. There are many aspects where issues can appear. Test them all before you put production critical data on the array. Know what the impact is to performance when something fails.

When testing, I use the I/O Analyzer fling from VMware. Deploy, configure the IP, and start running tests. You can deploy a fairly large number of them on one host, or across many hosts. The options are practically endless.

I’ll go into more details on how I do performance testing at some point in the future. I know my process isn’t perfect, but I try to know what the limitation are and testing the relevant settings (NMP, queue depth, etc) to know the impact.


OpenStack Summit

I have the privileges of heading to Atlanta today for a few days. This is a dual-purpose trip, first to do some work at $job’s data center, but, more exciting, is the second reason, which is the OpenStack Summit.

My role at $job is a mix of management and technical, with my primary technical aspect being the virtual infrastructure. VMware was selected as the hypervisor because of how simple it is to implement and manage and the amazing options for automation. Whenever possible, we will use open source options if they exist and don’t increase the workload significantly. My team is made up of 5 guys (including me) and manage >500 physical and virtual systems, with a wide variety of technologies (Linux, MySQL, Spacewalk, Puppet, VMware, storage, networking, security, etc).

My goal with attending the keynotes and wandering the vendor hall is to understand how OpenStack has been evolving over the past couple of years and where we could or should be using it within our infrastructure. I tend to be someone who tries to keep up on what is happening with competing and complementary solutions to what we use, and OpenStack has been maturing nicely where I don’t think it would be a significant management burden to implement and maintain.

One area of particular interest is how we could leverage OpenStack with Docker and Vagrant to provide a better environment for the software developers. We’re considering using vCloud for our developer environment, and I think the OpenStack ecosystem might have a comparable offering.

I’m also looking forward to meeting some of the people I follow on Twitter.


My VCAP5-DCA Experience

Like many people who go through the process of attempting to achieve VCAP (VMware Certified Advanced Professional) level certification, I’m going to share some of my journey. To summarize, for those who are mostly curious about the result, I passed on my first attempt after being 100% convinced there was no way I would pass. Since this site was down during the earlier stages, I’ll also include my VCP and VCA experiences.

VMware’s VCP certification requires that you attend one of their classes. This set the barrier reasonably high from a cost perspective. Since I have worked with vSphere for over 5 years, I opted to do the Optimize and Scale class because it has the most relevance to my current role. During the week of the online class, I used the many breaks to read Bill Furguson’s “The Official VCP5 Certification Guide” published by VMware Press. I scheduled the exam for a week after my class to ensure what I learned was fresh.

The testing center PC was pretty antiquated and slow, but I completed the test within the allotted timeframe. I didn’t really pay attention to the official blueprint prior to the test, otherwise I would have known to dig deeper into some areas. I won’t mention the specifics, simply because of the NDA around the test, sorry. Just know that you need to know everything on the blueprint! This test gives you the score immediately and I was surprised to have passed. I credit my many years of working with various operating systems for giving me the ability to make educated guesses when I wasn’t sure of the answer. This was the hardest test I think I have ever taken–even compared to the VCAP test.

I scheduled the exam for late in the afternoon, which also happens to be one of the times I have the least amount of focus. I didn’t know that I wouldn’t be allowed to have my phone in the room with me. I’m sure that was an oversight on my part with some of the communication. Being on a rotating on-call schedule makes availability very important so it is worth noting, especially for scheduling any future exams.

I didn’t pay much attention when VMware announced the VCA entry level certifications at VMworld. I had completed my VCP at this point, so it didn’t make sense to spend any money on the tests. Then they started giving out free testing vouchers… I opted to go into the VCA-DCV cold and passed with a very strong score. I went through the online class for the VCA-Cloud and then did that exam, and got a passing score… definitely not a stellar score, but passing. I’m planning to spend some time playing the vCloud stuff soon.

On to the main topic… After passing the VCP5-DCV test, I wasn’t sure how far I wanted to take the certification process. And I don’t know that I have that answer yet, either. I started to research the VCAP5-DCA exam since that was the next logical step with my hands-on admin experience. It was during this process that I came across the many VCAP exam experience posts that walked people through the process of taking the exam. During VMworld Europe, a 50% discount code was available and I took that opportunity to register for the exam. The exam had to be scheduled for no later than Dec 31st, so I scheduled it for Saturday, December 28th in the morning. I wanted a weekend so that I didn’t have work-related stress knowing that I can’t have my phone in the room with me. After retrieving my phone, I had a message about some problems and I spent the drive home discussing them with a colleague.

The DCA (Data Center Administration) exam has 26 multi-part tasks testing hands-on administration and troubleshooting skills. The single biggest factor in this exam is time management. The approach I used was to write 1 through 26 on the dry-erase board, then go through all of the tasks to determine if they are creation (C) or admin (A) tasks and make a note of the type of task (DRS, Networking, Storage, etc). Most of the tasks are fairly independent, however, there are some admin tasks that build on creation tasks. For example, you might have a creation task that sets up DRS and then an admin task later that has you configuring DRS rules. I went back to the beginning and started moving through the exam as quickly as I could, focusing on the ones I knew the subject matter the best. By the end of the allotted testing time, I still hadn’t started on 5 or 6 of the questions, if I recall correctly. I thoroughly enjoyed taking the test, and simply chocked it up to a good experience that I would have to redo. I just didn’t think I had finished enough of the questions to pass. At the end of the exam, you simply get a message stating that the exam will be scored and that you should expect the result within a few weeks (I think they say 15 business days).

Well, I got my results on January 6th. I was nervous opening the attached PDF even though I was sure I didn’t pass. I opened it on my phone, so I had to scroll down to see the score and grade. I was practically jumping for joy when I read my score and that I actually passed! And then a certain level of disappointment set in… I was looking forward to taking it again!

For studying, “The Unofficial Official VCAP5-DCA Study Guide” by Jason Langer and Josh Coen was my primary go to guide. They link to a number of other sites, whitepapers, and reference materials that dig fairly deep into the topics. I also did a simply Google search on “VCAP DCA exam experience” and read just about every post I could find.

Thanks to the wonderful community that has put together all the resources! Now on to the VCAP5-DCD…