We need to argue for a local server

October 22, 2009 8:32 PM

Our office is losing hours of productivity a week due to networking decisions from head office. We'd like to price out a server to replace the dodgy technology that's been foisted on us. Not surprisingly, HQ gets kind of defensive if we use that language. Please help me cost out a server, and make it seem like it was their idea all along.

I work in a regional office of an energy company. Head office is many thousands of kilometres away. Head office likes to keep complete control of their data centrally, so offices that used to have servers were replaced by WAFS boxes on the end of a 10Mbit link.

Since the WAFS boxes went in, users in remote offices have had nothing but trouble. Accessing files over the network feels like dialup speed. MS applications zone out for minutes on end thinking about autosaves. We have been known to start applications, go down to the coffee shop, come back, and the application might just have initialized. The WAFS machines themselves are unreliable, needing replaced every few months and rebooting weekly.

Our office has about fifteen professionals in it - engineers, lawyers, developers, environmental and admin staff. We're told that we never quite manage to fully fill our 10Mbit allocation, and that our connection maintains a supposedly quite speedy ping time of 50ms to Head office.

We're not digging the slowness. We estimate that we're each losing several hours per week waiting for the system. Some users have taken to working from home, as using a Citrix connection over DSL to head office is many times faster than running local applications in the office. We've lost staff over this issue.

Head office is all about the centralization, though. So much so that, a couple of years back, a fire alarm knocked out our central data server for 18 hours and we had no offsite failover. We'd need to make a very good case that having a local server would be cheaper than what we're using now.

Head office admits that the WAFS didn't deliver all it was supposed, and are looking at a similar technology from a different vendor. We're worried it will be a big spend with the same result - slow performance, and yet more technology that local techs barely understand. We have some allies in IT at head office, but not the ear of the head of IT.

So just how much would a robust small server with local support and backup cost for a 15 user office? We're in a big Canadian city with good access to service and support.

posted by scruss to computers & internet (8 comments total)

My questions are both tactical and strategic...How long has WAFS been in production? Did IT pilot this change in a small number of sites before rollout to the branches? How many branches are there, and at what is the scope of this -- is it everywhere? Some places? Just you? Did IT plan a back-out strategy at each site in case the project went sideways? Are the project rollout and post-rollout operations well documented? What is the cost of not having centralized backup of the branches, as compared to say, a single 18 hour site outage once every three years?

Playing devil's advocate here (and anticipating the pushback you're going to get)...I'd imagine central IT is well aware of the issue -- this WAFS initiative is almost certainly there as a less painful alternative to to something worse...big spending and changes typically don't happen unless they're trying to mitigate some worse pain. Depending on the answers to some of those questions, they may be either too invested and/or unprepared to back out. Depending on the answers, you may get better results trying to make WAFS work than you would trying to go backwards. Backing out the change may not make the most sense from a corporate perspective for any number of valid reasons. Performance is the bottom line anyway...endusers needn't be opposed to WAFS per se, but they are (and should be) opposed to subpar performance that impacts their productivity. Consider the possibility that If IT can make it work, and you can help drive that, it may benefit you and your branch in the long run.

In my business, this might be a critical situation that would push other priorities out of the way, and that designation comes with daily or even more frequent updates, around the clock focus, calling in vendors, etc. I have no idea how your org works, but you should be escalating and asking for status updates on a daily basis as to problem determination, what steps are being taken to address the problem, weighing alternatives or fixes, etc.

In any case, I think that this sounds like it is not a technology problem, it's a leadership problem.
posted by edverb at 9:35 PM on October 22, 2009 [1 favorite]

To echo something edverb said, we had a similar situation in our office when we instituted a WAFS. It worked great for Files... but Exchange / Outlook suddenly curled up and died. We were able to back out of the WAFS momentarily, but the boost it gave to our file system was good enough that we wanted to move forward with it in place. It took calling in the Vendor to address the issue, but it eventually was addressed, and everything worked great.

Working WITH your IT team rather than against them will more likely produce the results you need. But a well-documented case with metrics you can report back with is definitely your ally. If they dispute your metrics, they will be forced to collect their own, and then witness what you are dealing with firsthand.
posted by GJSchaller at 9:51 PM on October 22, 2009

You need wan acceleration and probably some work on the servers, they shouldn't reboot that often.
posted by iamabot at 11:33 PM on October 22, 2009

There's no fundamental reason a WAFS can't deliver near-LAN performance - certainly the issues you describing sound like a misdesign (insufficient cache etc), bad platform choice or simply an incompetent vendor.

Bear in mind, the head of IT might be under other constraints you're not aware of - data security requirements, imposed by senior management for example. Contrary to popular belief, IT don't hate the users, but we're not always able to give them what they want due to time and money contraints - we have limitations same as everyone else.

So bearing that it mind, your best bet is going to be being the sticky wheel. It's clearly having an impact on your productivity, and you need to keep escalating it up the chain at IT, and in other parts of the org that you report to. Get concrete performance metrics, get IT themselves onsite - vague complaints of it being slow will not get you far.

In the meantime - is there a particular reason you can't use the 10Mb WAN connection to run citrix sessions instead of/and the WAFS? It should be more than adequate for a 15 person office. Failing that, get some DSL connections into the office and run via the head office external gateway? It's cheap and wouldn't take a competent router tech long to rig up the hardware side of things.

With regards getting a local server; it depends upon what you're doing with it, how much storage, failover etc etc so you probably need to talk to a local reliable IT supplier. For a basic moderate fileserver with backup but no failover, I'd probably pay no more than about £2000 ($3500 CDN I think) with local tech doing the setup. The way to sell this would be as an interim workaround solution until they can get the WAFS up to acceptable speed that doesn't impact job performance of your staff. Just bear in mind you may get pushback with regards data security and backup reliablity.
posted by ArkhanJG at 11:51 PM on October 22, 2009

We've had WAFS since late 2006; previously, we had a server, and it was good. There are probably 3-4 branch sites, all have WAFS, and I think we were the pilot - we complained about the performance, but the system was rolled out anyway. We don't know about IT's internal strategy for some of the other questions. The WAFS boxes don't reboot - they lock up periodically, and need to be rebooted.

Head office has carried out metrics several times, but the results are not usually shared with us. I know that IT aren't out to get us; I was a sysadmin myself in a past life. The company has recently gone through big changes, with our group being moved to the more dynamic development side, with the more staid utility side staying out west. Previously, all installations had to meet NERC security standards, for reasons that were never made clear to office staff.

Some of us do use Citrix locally, but we create and handle large files locally, which sometimes take hours to transfer to the server.
posted by scruss at 5:01 AM on October 23, 2009

If WAFS has been in place for three years, I'd guess chances are slim to none of backing out or becoming a "one off" branch. (No IT org wants to create unique branches...they want cookie cutter operations to the greatest extent possible.)

Your best bet is to make the system work, and not second guessing the WAFS architecture wholesale. If they did the diligence on this architecture at all, it would have been a fairly extensive and highly documented effort, and significant time, money, bandwidth would have been dedicated to design, test, pilot, rollout...your job is to get them to invest in maintenance and responsiveness to internal customer needs.

Try to amplify your requests for action by coordinating your efforts with the other branches who may be experiencing the same impact to production. Ask if there is a branch SLA (service level agreement) for which you're being billed for IT service, and if so, does this fall short? "Spin" your communications positively...be sure to communicate that you're not interested in finding fault, just a remedy.

But, speaking from experience... you can probably forget backing out or switching to a different architecture. Your best bet on that score would be to try to influence the next refresh cycle.
posted by edverb at 6:38 AM on October 23, 2009

Hire a management consulting firm to tell them what you're telling them and, because said firm would be getting paid lots of money, they'll finally listen!
posted by StarmanDXE at 9:54 AM on October 23, 2009

Technically, it's not the best answer, but StarmanDXE, you nailed what they're doing ...
posted by scruss at 5:35 PM on October 30, 2009