Jon Alper's probably insufficiently humble opinion

I wish I could tell you all the horror stories. They’ve happened with some of the high budget projects I’ve done at every gig I’ve had. I’ll do my best to warn you without being able to actually show you all of the skeletons.

Somebody, usually somebody with more management experience than technical field experience will have a legitimate and very serious problem with how some workflow or business function is done at your company. They’ll meet somebody, read something and an email will arrive asking you to look into a new product or service that promises to solve their problem. The problem will be real and it will be important. Now, as a technology manager, you’re at a crossroads. How do you communicate your assessment of their proposed solution? Sorry, can’t help you. Too many variables. Is your boss ‘science minded’? Is she somebody you can have a real conversation with who will work alongside you to evaluate their idea? Damn, you’re a lucky devil. Is he a puffed up MBA-having-suit-wearing-testosterone-juiced-schmuck who got the job despite all reason? You’re hosed. The reality is, you’re almost surely waist deep in the vast expanse of gray in between. What I can tell you is, do some due diligence and don’t rely on IT industry press ‘case study’ articles and customer testimonials.

I first learned this lesson the hard way at my first real job managing a network and support for a medium sized business. I sold my boss on a service contract for CE Software’s QuickMail. We were already using QuickMail (served with damned near bulletproof reliability from an SE-30. I was pitched a long term contract by CE for a roadmap of upgrades. I was handed all sorts of documentation about how other customers were doing well in these arrangements and given prospective ROI analysis that painted a nice rosey picture for the future. I pitched my then boss with passion. I didn’t do enough homework. CE never shipped what they promised. The roadmap was vapor. My employer got screwed for a few grand. I felt terrible about it. My boss at the time, I think actually saw it coming and let it happen to give me some on the job training. He wouldn’t no matter how much I begged him to, let me point the company’s pit-bull lawyers at CE.  There were lots of lessons for me in this experience and some of those will be the topic of future posts. But the key lesson is, don’t believe what you read. Test, test, test.

What I can tell you is this. If you read an article in the I.T. press talking about how Enormocorp successfully deployed the proposed solution, it’s a lie. I mean it. If you read an article in “How to succeed in IT” magazine  featuring an interview with a CIO who just deployed a multimillion dollar CRM system and the quotes talk all about the amazing ROI, it’s a lie. It can’t be used as meaningful data in making your decision about whether your company needs the product. If the sales guy shows you this article? If he has a stack of five others like it? if his whole presentation is all about how deploying his solution will help make your small company able to be as successful as his Fortune 500 poster child clients? You are in serious trouble.

Now, did the CIO interviewed actually lie? Probably not. The article wasn’t long enough for enough truth. Here’s how it happens. How do I know? I was the guy you shouldn’t have relied on.

I was an avid user and advocate for Retrospect (A LAN backup package then made by a company called Dantz). I made a decent portion of my living for years implementing a series of successful deployments of Retrospect. Honestly, and I mean this, I saved companies I worked for literally millions deploying Retrospect for them. I enjoyed a great relationship with Dantz during this time and some of those people I knew back then at Dantz are, to this day, people I’d call friends. Retrospect was, at the time, a truly spectacular product.

My relationship with Dantz was so good, I got seeded with betas before they even admitted they were working on a new version. Back then, everything wasn’t in perpetual beta. I got surreal levels of support and I ate a lot of good meals on their dime. At the height of their success I was one of a half dozen or so consultants and I.T. managers asked to do a promotional interview for them. I can count on one hand with a spare finger or two, the number of companies I’ve ever trusted enough to say yes to that kind of request. I did the interview, I had a few laughs. The somewhat colorful  out-takes apparently made an internal reel shown to employees. I told the absolute truth. At the time, Retrospect essentially had a monopoly the automated backup business on MacOS and deserved to. I never had access to sales figures but I personally had well more than a thousand nodes in the  field  that I’d supported, deployed or spec’d at some very big name clients.

The day that tape was made, if you bought into Retrospect and had a clue, you’d have probably done very well for your users and bottom line. If, a year later, you saw that clip and invested in a large scale Retrospect deployment, you’d have done OK. If, two years later you had? Perhaps, not so much.

This was because the Mac, the only platform Retrospect really supported fully at the time, was in a major transition to OS X. This was not easy for a company like Dantz to cope with. Apple was in a certain amount of chaos. MacOS was being fundamentally changed from the ground up and Dantz was investing in more cross platform support. Ultimately, Dantz was sold to EMC. By then, while still a useful tool, things were far from the “This damned thing just works” truth I’d spoken on the tape. By then? What I’d said on tape became a lie. It wasn’t when I said it but it became one.

Also, the promotional case studies tape, because its purpose was to talk about Retrospect, no deeper discussion of backup strategies, archiving and good disaster planning was even discussed. If you relied on Retrospect as your entire solution you’d have been in deep trouble. I didn’t lie about how great Retrospect was but anyone who deployed Retrospect without a lot of other work and concurrent procedures could well have ended up screwed. Anyone who added up the number of seats they needed, added the most expensive Mac they could buy to the budge for a server and thought that actually represented the real budget to deploy was in deep trouble. If you heard me say “buying Retrospect will solve all your backup, archiving and disaster planning problems” then you heard a lie. The fact that didn’t say that, and would never have said that doesn’t help you feel any better about having trusted what you’d heard however much you heard was different from what I’d said on that promo tape.

So, that’s a best case where the well intentioned quotes become lies. Here’s another reality behind most of the quotes and short case study articles you read and I’m saying this because I have read them about projects I’ve worked on. The case studies are often written and most publicized before the projects are even finished. They’re written during the deployments and, as is usually the case, right when everyone working on the deployment is in a honeymoon with the vendor and the vendor is psyched to have a poster child client. The foot soldier employees probably aren’t working with the system on a day to day basis yet. The total costs of the training haven’t been accounted for yet. The costs of changes in workflow imposed by the tool haven’t been accounted for yet (and often never are). You’re being pitched the success of the John Hancock Tower in Boston right before the windows have started to pop out in the wind.

One particularly sordid example I need to be circumspect about was a client who spent, literally, millions on big iron to serve a website using what was, for about ten minutes, the high fashion CMS du jour. The CMS maker’s CEO was driving around in his ‘explosive dot com ere growth funded’ Ferrari talking about how well the CMS was doing in an emerging marketplace. The hardware vendor was crowing about the partnership. There was press about the client’s choice of and success with the CMS.

The truth, however, was that the website hosted on that CMS never worked properly. The developers who ultimately had to wrestle with it on a day to day basis were in agony.  It never produced increased revenue. It had downtime incidents that amounted to days in a year. The CMS vendor folded in a hail of internal acrimony. The language of the CMS fell into a level of deprecation that would make studying Cobol seem a good career move today. The hardware ended up being a hundredfold or more in excess of what was ever necessary with the price tag and sever room infrastructure costs to match. None of the experienced technical staff working for the client wanted it to happen. Everybody, including me, counseled against it. Press quotes, books, testimonials and buzzwords overcame seasoned judgement and millions were lost. The press about it? About this deployment? The case study quotes? All one hundred percent positive.

If you’re being pitched an enterprise product to revolutionize your business, don’t believe what you read. Insist on much deeper dialog with existing and current customers. Plan a staged process of test deployment and evaluation. Listen to your internal staff’s concerns. Don’t listen to lies even when they may well have been spoken with all sincerity.