Home Lab

IsilonSD – Part 2: Test Platform

This post is part of a series covering the EMC Free and Frictionless software products.
Go to the first post for a table of contents.

So… last post I figured out IsilonSD Edge needs (at least) three separate ESX hosts, with (at least) 7 independent, local disks. So I cannot deploy this on the nested ESXi VSAN I made, but I really want to get IsilonSD up and running.

Since my previous attempt to wing-it didn’t go so well, I’m also going to do a little more planning. I need to create ESX hosts that meet the criteria, plus some plan out things like cluster name, network settings, IP addresses and DNS zones.

For the ESX hosts, my solution is to run nested ESXi (a virtual machine running ESX on top of a physical machine running ESX). This allows me to provide the independent disks, as well multiple ESX hosts, without all the hardware. As well, this will help facilitate the networking needs, through making virtual switches to provide the internal and external networks.

To build this test platform, we’ll cover 4 main areas:

  • ESX Hosts
  • External Network
  • Internal Network
  • SmartConnect

ESX Hosts

For testing IsilonSD Edge, I’m going to make four virtual machines, and configure them as ESX hosts. Each of these will need four nNICs (two for ESX purposes, two for IsilonSD) and nine hard drives (2 for ESX again, and 7 for IsilonSD). I’m hosting all the hard drives in a single data store; it happens to be SSD. For physical networking, my host only has a single network card connected, so I’ve leveraged virtual switches without a network card to simulate a private management network.

A snapshot of my single VM configuration is below:

IsilonSD_NestedESXDiagram1

With the first virtual machine created, now I simply clone it three times, so I have four exact replicas. Why four? It will allow a three node cluster to start; then I can test adding (and removing) a node without data loss; the same we would with a physical Isilon.

Note the ‘guest OS’ for the virtual machines is VMware ESXi 6.x. This is nice feature of vSphere to help you keep track of your nested ESXi VMs. Though keep in mind, nesting vSphere is NOT supported by Vmware; you cannot call and ask for help. Not a concern here given I can’t call EMC for Isilon either since I’m using Free and Frictionless downloads. This is not a production grade configuration by any stretch.

IsilonSD_NestedESXDiagramx4Once all four virtual machines existed on my physical ESX host, installing ESX is just an ISO attach away.

After installing ESX on all my virtual hosts, I then add them to my existing vCenter as hosts. vCenter doesn’t know these are virtual machines and treats them the same as a physical ESX host.

I’ve placed these virtual hosts into a vCenter cluster. However, this is only for aesthetics purposes to keep them organized. I won’t enable normal cluster features such as HA and DRS given Isilon cannot leverage them, nor does it need them. Plus given there is no shared storage between these hosts, you cannot do standard vMotion (enhanced vMotion is always possible, but that’s another blog).

Here you can see those virtual machines with ESX installed masquerading as vSphere hosts:

IsilonSD_NestedESXCluster

I’ll leverage Cluster A you see in the screenshots for the Management Server and InsightIQ server. Cluster A is another cluster of nested ESXi VMs I used for testing VSAN; it also has the Virtual Infrastructure port group available so all the IsilonSD Edge VMs can be on the same logical segment.

External Network

The Isilon External network is what faces the NAS clients. In my environment, I have a vSphere Distributed Virtual Switch Port Group called ‘Virtual Infrastructure’ where I place my core systems. This is also where vCenter and the ESX Hosts sit as well, and what I’ll use for Isilon as there is no firewall/router between the Isilon and what I’ll connect to it.

Virtual Infrastructure network space is 10.0.0.0/24; I’ve set aside a range of IP addresses for Isilon in this network.
.50 is the management server
.151-158 for nodes
.159 for SmartConnect
.149 for InsightIQ.
You MUST have contiguous ranges for your nodes, but all other IP addresses are personal preference.

For use in the deployment steps:
-Netmask: 255.255.255.0
-Low IP Range: 10.0.0.151
-High IP Range: 10.0.0.158
-MTU: 1500
-Gateway: 10.0.0.1
-DNS Servers: 10.0.0.11
-Search Domains: lab.vernex.io

Internal Network

The physical Isilon appliances use dedicated Infiniband switches to interconnect the nodes. This non-blocking, low latency, high bandwidth network allows the nodes to communicate with each other to stripe data across nodes, providing hardware resiliency. For IsilonSD Edge, Ethernet is used over a virtual network for this same purpose. If you were deploying this on physical nodes, you could bind the Isilon internal network to anything that all hosts have access too, the same network as vMotion, or a dedicated network if you prefer. Obviously, 10Gb is preferable, and I would recommend diversifying your physical connections using failover or LACP at the vSwitch/VDS level.

For my lab, I have a vSphere DVS for Private Management traffic; this is bound to the virtual switch of my host that has no actual NIC associated with it. It’s a private network on the physical host under my nested ESXi instances. I use this DVS for VSAN traffic already, so I merely created an additional port group for Isilon named PG_Isilon.

Because this is essentially a dedicated, non-routable network, the IP addresses do not matter. But to keep things clean I use a range set aside for private traffic (10.0.101.0/24) as well use the same last octet as my external network.

For use in the deployment:
-Netmask: 255.255.255.0
-Low IP Range: 10.0.101.151
-High IP Range: 10.0.101.158

SmartConnect

For those not familiar with Isilon, SmartConnect is the technique used for load balancing clients across the multiple nodes in the cluster. Isilon protects the data across nodes using custom code, but to interoperate with the vast variety of clients standard protocols such as SMB, CIFS and NFS is used. For these, there still is not an industry standard method for load to be spread across multiple servers (NFS does have the ability for transparent failover, which Isilon supports), the approach is a beautiful blend of powerful and simplistic. By delegating a zone in your enterprise DNS for the Isilon cluster to manage, SmartConnect will hand out IP addresses to clients based different load balancing options appropriate for your workloads, such as round robin (default) or others like least connection.

To prepare for deploying an IsilonSD Edge cluster, we’re going to modify the DNS to extend this delegation to Isilon.Configuring the DNS ahead of time makes the deployment simple. If you’re running Windows DNS, here are the quick steps (if you’re using BIND or something similar, this is a delegation and should be very similar in your config files).

Launch the Windows/Active Directory DNS Administration Tool

IsilonSD_NewDNSDelgation

Locate the parent zone you wish to extend, here I use lab.vernex.io.

Right click on the parent zone and select New Delegation

 

 

 

 

 

 

 

 

 

 

 

Enter the name of the delgated zone, this ideally will be your cluster name, for my deployment Isilon01IsilonSD_Delgation1

 

 

 

 

 

 

 

 

 

Enter the IP address you intend to assign to Isilon SmartConnect

IsilonSD_Delgation2

 

 

 

 

 

 

 

That’s it, when the Isilon Cluster is deployed and Smart Connect running, you’ll be able to navigate to a CIFS share like \\Isilon01.lab.vernex.io, your DNS will pass this request to the Isilon DNS server, which will reply with an IP address of a host that can accept your workload. This same DNS works for managing the Isilon cluster as well.

QuickTip, you can CNAME anything else to Isilon01.lab.vernex.io, so I could make File.lab.vernex.io CNAME pointed to SmartConnect. This is an excellent way to replace multiple file servers with a single Isilon.

For use in the deployment:
-Zone Name: Isilon01.lab.vernex.io
-SmartConnect Service IP: 10.0.0.159

 

By | March 28th, 2016|EMC, Home Lab, Storage, VMWare|1 Comment

IsilonSD – Part 1: Quick Deploy (or how I failed to RTM)

This post is part of a series covering the EMC Free and Frictionless software products.
Go to the first post for a table of contents.

As I mentioned in my previous post, EMC recently released the “software defined” version of Isilon; their leading enterprise scale-out NAS solution. If you’re familiar with Isilon, you might already know there has been a virtual Isilon (called Isilon Simulator) for years now. The virtual Isilon would run on a laptop with VMware Workstation/Fusion, or on vSphere in your datacenter. I purchased and installed Isilon in multiple organizations, for several different use-cases; the Isilon Simulator was a great solution for testing changes pre-production as well familiarizing engineers with the interface. The Isilon Simulator is not supported and up until recently you had to know the right people to even get ahold of it.

With the introduction of IsilonSD Edge, we now have a virtualized Isilon that is fully supported, available for download and purchase through your favorite EMC sales rep. It runs the same codebase as the physical appliances, with some adjustments for the virtual worlds. As we discussed, there is a “free” version for use in non-production as part of EMC’s Free and Frictionless movement. I’ve run the Isilon Simulator personally for years, so I want to leverage this latest release of IsilonSD Edge as my new test Isilon in my home lab.

A quick stop to the IsilonSD Edge download page and I’m quickly pulling down the bits. While waiting a few moments for the 2GB download, I review some of the links; there is a YouTube video on Aggregating Unused Storage, another that covers FAQs on IsilonSD Edge and one more that talks about Expanding the Data Lake to the Edge. These all cover what I assumed, you’ll want at least three ESX hosts to provide physical diversity, you can run other workloads on these hosts, and the ultimate goal of this software is to extend Isilon into Edge scenarios, such as branch offices.

Opening the downloaded ZIP file, I find a couple OVA files, plus the installation instructions. I reviewed a couple of the FAQs linked from the product page, though didn’t spend much time on the installation guide; nor did I watch the YouTube Demo: Install and Configure IsilonSD Edge. I like to figure some things out on my own; that’s half the fun of a lab, right? I did see under the system requirements, it mentions support for VSAN compatible hardware, referencing the VMware HCL for VSAN. I just recently setup VSAN in my home lab, so that coupled with the fact I’ve run the Isilon Simulator; I’m good to go.

Fast forward though a couple failed installations, re-reading the FAQs, more failed installations, then reading the actual manual… here’s the catch.

You have to have local disks on each ESX node.

More specifically, you need to have directly attached storage… 
        without hardware RAID protection or VSAN.

Plus, you need at least 7 of these directly attached unprotected disks, per node.

While this wasn’t incredibly clear to me in the documentation, once you know this, you will see it’s said; but given IsilonSD is running on VMDK files, I glossed over the parts of the documentation that (vaguely) spelled this out. If you’ve deployed the Isilon Simulator, or any OVA for that matter, you’re used to selecting where the virtual machines are deployed, I assumed this would be the same for IsilonSD and I could choose the storage location.

However IsilonSD comes with a vCenter Plug-In that deploys the cluster, as part of that deployment, it scans for hosts and disks that meet this specific requirement. Moreover, during the deployment IsilonSD leverages a little used feature in vSphere to create virtual serial port connections over the network for the Isilon nodes to communicate with the Management Server, this is how the cluster is configured; so deploying IsilonSD nodes by hand isn’t an option (you can use still use the Isilon Simulator, which you can deploy more manually).

I’m going to stop here and touch on some thoughts, I’ll elaborate more on this in a later post once I actually have IsilonSD Edge working.

I do not know any IT shop that has ESX hosts that has locally attached, independent disks (again, not in a RAID group or under any type of data protection). We’ve worked hard as VM engineers to always build shared storage so we can use things like vMotion.

The marketing talks about capturing unused storage, about running IsilonSD on the same hosts as other workloads; in fact the same storage as other VMs; but I’m not sure who has unused capacity that’s also independent disks.

I certainly wouldn’t recommend running virtual machines on storage without any type of RAID-like protection. Maybe some folks have a server with some disks they never put into play, but 7 disks; and at least three servers with 7 disks?

I know there are organizations that have lean budget and this might be the best they can afford, but are shops like licensing and running vCenter ($), are they looking at virtual Isilon($)?

Call me perplexed, but I’m going to put off thinking about this as I still want to get this running in my lab. Since I don’t have three servers and 21 disks laying around at home, I’ll need to figure out a way to create a test platform for IsilonSD to run.

Be back soon…

 

By | March 25th, 2016|EMC, Home Lab, Storage, Uncategorized|3 Comments

Free and Frictionless – A Series

One of the most common statements I’ve made to vendors over the years is “why should I pay to test your software?”. To this day I still don’t understand this; if I’m going to purchase software to run in my production environment, why should I have to pay to run this software for our development and testing needs? It seems counter-intuitive; in my mind having easy access to software which IT can test and develop against increases the probability of choosing it for a project. Having software be free in non-production allows developers to ensure that it is properly leveraged, as well it encourages accurate testing and facilitates operations ensuring it’s ready to be run in production. In my experience not only does this result in more use of the software in production (which means more sales for the vendor), but more operational knowledge (which means less support needed from the vendor).

Companies offer difference solutions to attempt to solve this. Microsoft does it well with TechNet and MSDN subscriptions; which for a small yearly fee you can license your IT staff, rather than the servers; you get some limited support and recently even cloud credits. Many companies will provide time-bombed versions of the software; this helps in the evaluation phase to test installation, but falls short in ongoing development needs, not to mention operations teams gain no experience. Some vendors will steeply discount non-production; though most of them only do this through during the purchasing process, and I’ve seen a wide range of how well this gets negotiated (if at all).

There is no doubt in my mind that this challenge is a significant factor in the growth of open-source software. With the ability to easily download software, without time limits and without a sales discussion; the time to start being productive in developing a solution is dramatically reduced. I’ve made this very choice; downloading free software and beginning the project while things like budget are still not finalized. The software can be kept running in non-production and when moved into production, support contracts can begin. You don’t need to pay upfront, before prototyping, before a decision is made and before any business value is being is being derived.

This is why I’ve been ecstatic EMC is making a movement towards a method that allows the free use of software in non-production, even in products they are not using an open-source license model. They refer to this approach as ‘Free and Frictionless’. It doesn’t apply to all their software, but the list is growing. Currently, products like ScaleIO, VNX, ECS and recently added; Isilon. The free and frictionless products are available for download, without support, but without time-bombs either. In most cases there are restrictions, such as the total amount of manageable storage. These limitations are easy to understand and work with and fully deliver on my age old question “why should I pay to test your software”.

I’m going to spend a little time with these offerings, many of them I’ve run in production, at scale; so I’m interested how well they stack up in their virtual forms. I’ll also explore some products I haven’t run before.

By | March 24th, 2016|EMC, Home Lab, Storage, VMWare|10 Comments