Isilon Archives - FastStorage

OneFS 8.2 firmware improvement: isi_upgrade_helper

Everything Storage

I’ve previously wrote a howto to upgrade your Isilon cluster’s firmware. The process hasn’t changed much over time: you upload a node firmware package to the cluster, install it, and then apply it to the nodes themselves. Time wise though, it’s not a quick process. Each node will reboot during this process, and the process takes at least 10 minutes per cycle. Imagine a 100+ node cluster and running this process one node at a time…

Fortunately, OneFS 8.2 has recently introduced a nice new tool to make this node firmware upgrade a bit easier and much faster: isi_upgrade_helper. It figures out which nodes can upgrade simultaneously and spits out the relevant commands for you.

Continue reading →

Connecting your Dell EMC systems to SRS, the easy way!

Everything Storage

Dell EMC uses Secure Remote Services (SRS, formerly known as ESRS) to enhance the tech support experience for their products. There’s two sides to this support: connect home, and connect in. Connect home is your device itself dialing back home to Dell EMC to report various things such as errors, automatic support uploads, etc. If either of this results in a Service Request at Dell EMC, a engineer can then use SRS to dial in / connect in and have a look at the faulty system. The latter saves you from having to host a Webex session.

Dell EMC likes to have all Dell EMC systems connected to SRS, again for two reasons. First of all, it reduces the time spent by engineers in troubleshooting an issue. If an engineer can dial in himself, without having to negotiate a Webex session with the customer, that means more SRs per engineer per day and lower support costs for Dell EMC. Secondly, it will result in faster incident resolution, and thus a happier customer. The support engineer can look up the state of a defective drive independently, and order new parts while the customer is sleeping. Win-win!

Continue reading →

Isilon SyncIQ uses incorrect interface: careful with mgmt DNS records!

Everything Storage

I’ve installed quite a few new Isilon clusters in 2019. All of them are generation 6 clusters (H400, H500, A200), using the very cool 4-nodes-in-a-chassis hardware. Commonality among all these systems is an 1GbE management port next to the two 10GbE ports. While Isilon uses in-band management, we typically use those UTP ports for management: SRS, HTTP, etc. We assign those interfaces to subnet0:pool0 and make it a static SmartConnect pool. This assigns one IP address to each interface; if you do it right, these should be sequential.

Recent addition to my install procedure is to create some DNS A-records for those management ports. This makes it a bit more human friendly to connect your browser or SSH client to a specific node. In line with the Isilon naming convention, I followed the -# suffix format. So if the cluster is called cluster01, node 1 is cluster01-1, node 2 is cluster01-2, etc. However, it turns out this messes up your SyncIQ replication behavior!

Continue reading →

PSA: Isilon L3 cache does not enable with a 1:1 HDD:SSD ratio

Everything Storage

I recently expanded two 3-node Isilon X210 clusters with one additional X210 node each. The clusters were previously installed with OneFS 7.x, and upgraded to OneFS 8.1.0.4 somewhere late 2018. A local team racked and cabled the new Isilon nodes, after which I added them to the cluster remotely via the GUI. Talk about teamwork!

A brief time later the node actually showed up in the isi status command. As you can see in the picture to the right, something was off: the SSD storage didn’t show up as Isilon L3 cache. A quick check did show that the hardware configuration was consistent with the previous, existing nodes. The SmartPool settings/default policy was also set up correctly, with SSDs employed as L3 cache. Weird…

Continue reading →

Reassign Isilon node IP addresses; go OCD!

1 comment

Everything Storage

A while ago I installed two new Isilon H400 clusters. With any IT infrastructure, consistency and predictability is key to a trouble-free experience in the years to come. Cables should be neatly installed, labeled and predictable. Wiring in the internal network cables, it helps if the nodes 1 through 4 are connected to switch ports 1 through 4 in order, instead of 1,4,2,3. While some might consider this OCD, it’s the attention to detail that makes later troubleshooting easier and faster. Like a colleague said: “If someone pays enough attention to the little details, I can rest assured that he definitely pays attention to the big, important things!”.

So I installed the cluster, configured it, then ran an isi status to verify everything. Imagine my delight when I saw this:

Aaargh!

Continue reading →