This post is a long time coming and is extremely overdue. The only reason why I am finally getting off my rump and writing something is because today I was forwarded an open letter, “mea culpa” article written by Tom Buiocchi, CEO of Drobo on Scott Kelby’s site.
Now, I only got the link to Mr. Buiocchi’s response, and I have no desire to read through Scott’s initial problems with his Drobo. Unfair? Possibly, but I think that it’s unimportant to the story aside from the fact that 1) it caused a response and 2) it sounds like Scott was extremely frustrated.
Boy, can I sympathize.
While still under the standard Drobo warranty (I had a few months left to go), I began having very curious issues. One Friday morning I received a warning from the Drobo that it was running out of disk space, and it instructed me to insert a drive into the empty slot at the top of the chassis.
The issue: the Drobo had *no* empty slots.
So, I did what any good geek would do. I turned to Drobo’s software to look through the logs to find out what was going on. This software, called “Drobo Dashboard,” had been open and running for weeks, so I switched to the software and find out what was happening. However, the Dashboard program had never reported any issues with any of the drives – much less forgetting that one was there.
Obviously, in a RAID environment losing a drive is a big deal, as suddenly you’re ‘flying without a net,’ but that is a different problem than the chassis reporting that a drive was missing altogether. In fact, aside from the fact that the Dashboard was claiming a low memory problem (and not seeing the drive that was in the bay), it didn’t show there had been any issues.
My first attempt to fix the problem was to free up some space. There were several files in the Drobo Trash, so I attempted to delete them. After watching the progress bar (or should that have been the ‘no progress’ bar?) for hours (no exaggeration) I reluctantly stopped the process. After all, you never know whether or not the process was ‘almost finished’ or just wasting time. Nonetheless, the “used space” did not change at all.
Drobo has an option to rescan the chassis to refresh the information, and so I thought that after attempting to delete the files perhaps the information display was ‘stuck.’
Hey, why not. Stranger things have been known to happen in software.
So, I did a rescan for more information and Dashboard suddenly told me that there were Drobo Dashboard components that are suddenly ‘missing,’ and I should re-download and restart.
That’s right. The software had suddenly lost parts of itself.
I did this, and found that there was a new version of the Dashboard which I had hoped might ‘find’ the drive that was in the chassis but that Drobo was ignoring. To my horror, the “new” Dashboard opened up and suddenly I have more warnings warnings all over the my screen than the Chernobyl nuclear power plant. Not only was the low memory error was back, the drive not found, but there were two others:
1. “Data protection is in progress, but you may continue accessing your data. Do not remove any hard drives with blinking YELLOW/GREEN lights” (which of course, was all of the drives), and;
2. “Drobo cannot currently protect your data against hard drive failures.”
At that point, I had no means using the Dashboard to determine whether or not the data protection was continuing, nor did I have confidence that, even when the data protection is finished, the Drobo will give me accurate and reliable information about the data integrity.
Fortunately, I managed to mount the volume and backup the read-only data off the Drobo onto separate drives, but the Dashboard was going into automatic “Data Protection” mode.
During the next 120 hours (that’s more than 5 days), I tried calling Drobo, tweeting Drobo, writing to their forums, effectively trying to get someone – anyone – to contact me for more information. Because there was an issue with the Dashboard software – both previous and new versions – identifying that all the bays were occupied, I was beginning to suspect a hardware fault. (For the record, in the months since I tested each of the hard drives using multiple diagnostic tools and found that there were no actual errors in any of the drives).
Growing increasingly frustrated, I vented on Twitter. Others retweeted my frustrated 140-character-or-less complaints, even suggesting (in a not-so-subtle hinting way) that Drobo should begin paying attention.
In the meantime, very little was happening on the device side. The Dashboard seemed stuck at around 60% of the progress bar completion with “approx. 24 hours remaining” staring me in the face for 4 straight days. Looking at the console logs resulted in no joy, either. Apparently nothing was being written to the logs, which meant that I was at the mercy of a lying piece of software that refused to give up its secrets behind the scenes.
Now, basically, I’m a very patient man. But data and volume corruption are things that I don’t take kindly to, if you catch my drift. And I don’t particularly like being ignored, especially since I paid well over a grand for my Drobo S system. Okay, not exactly enterprise-level budget, but that’s a lot of hamburgers to flip to pay for it!
Synology to the Rescue
In the meantime, I began getting courted by Synology.
Despite the fact that the outcome may have seemed to have been inevitable, it wasn’t an easy seduction. I had spent a lot of my hard-earned cash on a Drobo and switching to a new system – especially while I was still technically “under warranty” (despite not being able to actually get in touch with the company) – really made me wary.
This time, though, I did my homework. I got rave reviews from people on Twitter who have far greater knowledge about the technical innards of things like this than I do. I read the data sheets, the manuals, the reviews, and everything pointed to Synology being not just a stellar box to put on my desktop, but the company seemed to be responsive.
I like responsive. I like responsive a lot!
Synology contacted me via twitter, something that Drobo had refused to do despite being included in every single bemoaned tweet about my situation, and offered me a fair deal on their DS1511+ chassis (the previous version of their DS1512+).
Oh, it was a pretty box. Like the boyfriend who is being abused by his existing lover this sexy little machine came along and showed me what a relationship should be like.
“Don’t make up your mind now,” she said, so sweetly, to me. “Think about it until the end of the week.” Don’t want to be the rebound girl. I get it.
I was tempted, so tempted. I flirted around the Synology website, danced with the order page several times, picking up to call her for that date. I kept thinking about her figures, and everything started to fall into place.
Yes, do it! She’ll treat you right! Why torture yourself any longer? She’s. Right. There! What was I waiting for?
As I reached for that ‘submit’ button of destiny, the old girlfriend reached out and paid me some attention.
“Hello J,” the old Drobo girlfriend said, after weeks of silence. “I came across your twitter post and saw that you could maybe use some help.”
I stopped myself from pulling the trigger and went through internal hemorrhage trying to decide what to do. Maybe she had changed. Maybe she wanted me to stay after all. Maybe I could salvage the time, energy, and money that I had put into this relationship.
I looked longingly at the cute little Synology, and sighed. Turning back to Drobo, I had to give it one more try. For the sake of the relationship.
It took me time, but I poured my heart out and explained all the issues I had been having. I mentioned about how the lack of communication from the Drobo was driving me crazy, how I’d been waiting for ages for some feedback that I could count on. I mentioned that the software lies were stacking up and I didn’t know where I stood.
I left it unspoken, but I had hoped that the tone of my voice (in my email) would make it clear – this was her last chance to make it right.
Sending the email felt good, cathartic. Finally Drobo was willing to listen, willing to hear what I had to say. I felt hopeful.
My hope turned sour as an entire day went by with no response. When the first day turned into two, I became convinced that I had been lured back from breaking things off, and that things weren’t actually going to change.
All those feelings came back, the frustration, the feeling of being ignored. As the day slogged into the following I grew spiteful, resentful. I grew angry with myself for having chosen poorly in the first place. I grew angrier thinking that I could have been two days into rebuilding my data with a new Synology drive, instead of waiting around for Drobo to call.
Defiant, I returned back to Synology and asked her out. With an almost joyful glee I submitted my order and felt – for the first time since I plugged in my Drobo and had inconsistent experiences that I was back in control of my digital data life.
That night I got an email from Drobo, asking for more information. It was mostly a form copy-and-paste request for operating system information, model numbers, serial numbers. It was cold, impersonal, not to mention also information that I had provided several times before when I had had previous issues with the device.
The emotional cord had been severed for me by this point: I came to realize that Drobo didn’t pay attention, didn’t care. It was too little, too late, and too repetitive. I simply didn’t want to go through the cycle all over again.
I couldn’t find the energy to respond.
The Synology Experience
To say that the experience with Synology was different than Drobo is like saying that listening to Chopin is different than listening to your kid brother’s first-grade piano recital. To even make the comparison is ridiculous on its face.
From the moment I opened the box I was enthralled. I swapped out the drives from the Drobo into the Synology (and here, dear readers, is where I’ll abandon the metaphor of the new ‘girlfriend’ altogether) and connected it to my Ethernet switch. The software that came with the drive was a simple little application that searched for Synology devices on the network and politely asked if I wished to connect to it.
The web interface that came up was simple, intuitive, and… expressive. There are a lot of possible uses for the Synology and each one is clearly laid out and easy to follow. I don’t need fancy graphics (though the Synology’s is just fine): I just need to find the information that I want when I want it. And it gave me that.
I had wanted to know whether or not I should connect the Synology volume via NFS, iSCSI, or SMB, and whether or not I would get better performance using Jumbo frames or not.
So what did I do? I called them up and they walked me through each option.
I am not kidding. The support guy on the call was FREAKING AWESOME!
He showed me how to properly set up the system (the manual was good, but to be fair I don’t think it was the clearest-written documentation I’ve seen) for each of these different types, helped troubleshoot on the phone, and basically was there to answer any question I had.
Synology Faces A True Test
A few months ago, I decided that I needed to upgrade the drives inside the Synology. I was running out of space, and I wanted to kick up the 1.5TB drives to 3TB. The Synology is designed to “hot upgrade,” which means that you can remove the drive and swap in a new, bigger drive and it will rebuild the RAID volume and yet still make the data available. As you can imagine, this can take a long time, and I started the process and prepared myself for the overnight rebuilding.
I awoke early in the morning to a soul-deadening beeping sound. This could not be good. When I went to the office, I had found that two drives had failed – the new drive and the one next to it.
In my arrogance of having a smoothly-working Synology drive from Day 1, I had neglected to back up my data before taking out the drive, and my last backup was, well, old.
Yeah, yeah, I know. It’s always those of us who think we know what we’re doing that we’re likely to cut corners and look like idiots.
In desperation, I flooded Synology with messages. Phone, twitter, email, forums, etc. This time, I was in a panic.
To be honest, Synology could have looked at me (as I would have looked at me) and said, “Sorry, you should have backed up your data. By all rights, I should have been left out to dry, as I’m sure Drobo would have done. And I would have deserved it.
Synology rose to the occasion, however, and almost as soon as business hours began I was getting emails from tech support – the same guy, as it turns out, who had sat with me months before trying out different storage protocols for my new array.
Going through data logs can be time consuming, but Synology always kept me updated as to what was going on. They remotely logged into my system (with my permission), and I watched them as they went through the settings, logs, and double-checked for themselves. The first thing they did was to get the system up and running to where I could copy the files off of the volume, and I didn’t lose a single file. That alone made them heros in my book!
The bottom line – it looked like it was a hardware defect. Synology sent me a brand new system and even paid for the postage to send the old one back. The new system didn’t cost me one more dime.
Within a couple of days my new Synology system was rebuilding, my data was backed up to several external drives, and I was crossing my fingers that my butt would be saved.
Finally, the Synology gave the all-clear, and I was relieved – for the most part. With everything that had happened (this had dragged on for two weeks by this point), I was convinced that Murphy’s Law was in full effect.
So, I called up Synology, and asked them to go through the logs and run some tests to make sure that what I was looking at was actually what was going on. Fool me once, Drobo, shame on you…
Over the next hour, Synology patiently went through my configurations and data logs and double-checked to ensure that there were no additional issues. Being the kind of person I am, I asked several technical questions, and they answered each and every one of them. Patiently.
It takes a lot for me to be satisfied when I get my hooks into a subject, and especially when something goes wrong. But Synology patiently answered every single question I had, and if they didn’t know the answer they contacted a developing engineer, who then got back in touch with me (those pesky ‘why’ questions!).
Knowing What You Sell
It’s because of this that I am a devoted Synology customer. Quite frankly, every company should have customer service like this: attentive without being over-solicitous, informative and detailed without risking obfuscation.
I love my Synology. I feel valued as a customer. I love being treated like there is a value exchange between the vendor and me as a customer. It’s really that simple.
Drobo’s CEO Tom Buiocchi opened up his opinions on Scott Kelby’s site (as well as his work email and phone number, an interesting olive branch to be sure) but was promptly razzed by the commenters who were suspicious of his motives. If someone who didn’t have as popular a blog had written, would anyone have ever heard from Mr. Buiocchi? Is this simply a case of the squeakiest wheel getting the grease?
I can’t quite say that I fall into this camp, not completely anyway. I briefly met Tom at an event a few months ago, and my impression then – as now – is that we are not talking about an opportunist. I believe that Tom’s sentiments on Scott Kelly’s blog are genuine, and his motives are aligned with what he would actually like to see the company be.
What Drobo is and what he wants it to become, however, are farther apart than I think he realizes. This is far more than not having a responsive Twitter handle, and it’s far worse than the lackluster (and apparently erroneous) response Scott apparently got. This is a systemic problem for Drobo, reported anecdotally by several of the people who commiserated with me during my week+ agony of a non-responsive Drobo system.
In the world of storage word-of-mouth is a far greater influence on buying habits than data sheets and top-line specs.
Drobo still hasn’t realized that it isn’t selling boxes: it’s selling so much more. It’s selling autonomy, independence, memories, and lifestyles. It’s selling all the photos, videos, documents, dissertations, and personal data that make up the customers lives. Synology got that from the first time I got them on the phone – aside from Drobo’s sales guy when I bought the device, I have never managed to get anyone from Drobo on the phone. Ever.
To that end, this is a very simple equation. If you sell me something with the promise of providing resiliency for my personal data, you’re engaging in a very real contract relationship. If you tell me that you are going to provide me with the ability to keep my life – at least the important digital constructs of it – safe and secure using your equipment, it is an absolute must that your recognize the nature of what’s at stake.
Data storage is a long-term relationship, not a one-night stand.
Drobo did not (and still does not, it appears) recognize what’s at stake. They’ve got customers, “a couple hundred thousand to choose from,” so it’s easy for them to whitewash over a single little Drobo S with a (possibly) faulty drive bay.
But Synology didn’t seem to have that problem at all. Every interaction I’ve ever had with the company, through their social media representative to the salesperson to the tech support to the development engineer, has treated me and my data with the same respect and importance that I do. It’s just not that hard to treat people decently. It cost Synology nothing more to pay attention.
So, I have a Drobo S, very slightly used, in its original packing box. Could be something wrong with it, could be nothing wrong with it. Don’t know. Make me an offer and you can find out for yourself.
But I wouldn’t count on Drobo offering any insights. You’re on your own.
This morning I got a surprise phone call from Tom Buiocchi who had read this blog. In much the same way he was trying to reach out to Scott Kelby, he wanted to reach out to me personally and talk about my particular situation.
It was a pleasant conversation, and I do appreciate the call. After all, so much time has gone by that he could have written me off as a lost cause. Instead, he wanted to find out how he could make up for the past problems. Did I want to swap out my Drobo, do nothing, what would I like to do?
As I’ve mentioned (repeatedly!) I’m a big believer in exchange of value. I made Tom an agreement: we’ll do the exchange of my Drobo S and I’ll put the new one through its paces and, after giving enough time to work it out, I’ll write an update to this blog to describe my experience.
Of course, many of the problems that I experienced went beyond the device itself, so being higher on Drobo’s radar changes the situation a bit. However, I do appreciate Tom (and Drobo’s) attempt to improve the situation – and my impression of Drobo as a company – even after all this time.
Now I just have to try to find some hard drives to put in the Drobo…