How can I fend off the boss with data while I do my job of building a great website?

February 23, 2008 11:48 AM

What kind of web stats can I provide my bosses, to satisfy their hunger for "measurable stats" but avoid going down a rathole and losing focus?

I joined a very old line company mid-way through last year as the manager of the web team. The company is a few hundred years old, and is roughly in the retail business. The company's clients are generally older, well-heeled, and relatively technophobic. The website itself is mainly brochure ware for the company, but has extensive and detailed on-line product catalogues. There is no e-commerce component. My hiring was coincident with the re-launch of its website. There's little interactive "2.0" functionality to it, but some pretty cool stuff with regard to displaying items for sale.

When I was hired, I knew all of this, and basically took the challenge on to, over time, redo the website and bring it into the 21st century (employer user-centered design techniques, rich interfaces, etc... make the whole experience a lot more engaging). I have built out a product roadmap with a lot of this functionality, but my boss and the CEO (his boss) don't want to hear any of it, until we can establish some benchmarks for success and some reasonable goals. The first thing they are looking to show is the adoption rate, i.e.: how have the traditional customers taken to the new platform? Has this led to additional sales activity, has it lessened costs (i.e.: do we need to do shorter runs of print catalogues because more customers are now exclusively doing product research with us on-line?), and so on.

Those are worthy pursuits, but the problem is that we have not even been through a full business cycle, and the site's functionality is so basic, I cannot think of anything past basic stats like pageviews/customer or unique visits by each customer trended over time (however, until we get through a business cycle or two, these data will be skewed). Just the most basis stuff.

Problem here is that the CEO has a monomaniacal focus on everything being data-driven and my boss seems unable to push back on him, if he even wants to. My viewpoint, based on all of the above is that if we build a state of the art site and make elegant, engaging and highly useful, we will attract a new generation of customer. Any attempts now to justify what's been done, or what we're going to do (other than focus groups), I think might be premature or misleading.

Can anyone offer any help?

posted by anonymous to work & money (10 comments total)

Your CEO is right. Web design and development is often driven (incorrectly) by half-assed hunches. You need to collect and analyze as much actionable data as possible on your current system, and propose how "elegant," "engaging," and "useful" might be measured for your proposed enhancements.
posted by b1tr0t at 12:11 PM on February 23, 2008

You're getting caught up in the politics of a chicken-egg problem vis a vis approval for unmeasurable improvements. Hedge your idealism for the time being until they get the baseline they're looking for. Just throw Google Analytics on every page and get your graphs going that way. Once they see some pretty pictures you'll be able to figure out what their priorities actually are and you can figure out a strategy for bringing their web presence into the 21st century.

My viewpoint, based on all of the above is that if we build a state of the art site and make elegant, engaging and highly useful, we will attract a new generation of customer.

From the CEO's perspective this plan probably appears to be a pig in a poke.
posted by rhizome at 12:12 PM on February 23, 2008

Tell them it's hard to measure things like conversion without implementing changes, like shopping cart functionality. If the site is truly just a way to showcase products without selling, then how could you measure anything but eyeballs? There's no real behavior to be measured.

Also seconding Rhizome - Google Analytics shows pretty graphs that you can screengrab. Show them things with cool names like stickiness and bounce rates, and hopefully they'll think you know what you're doing enough to give you the reigns and implement some more functionality to the site.
posted by mamessner at 12:36 PM on February 23, 2008

Also, they may be less concerned about seeing the actual results of the data tracking, and more concerned about knowing that you're set up to track it and show it in the future. Keep the emphasis on what you WILL be able to track after the changes you've made. Good luck!
posted by mamessner at 12:38 PM on February 23, 2008

I'm incredibly cynical about the gathering of statistics. Most of the time, the people who want statistics want them because "they're supposed to have statistics." The information gathered almost never causes any changes in a decision-making process. It's just part of that MBA IT checklist. "Do we have statistics? CHECK. Do we have a support contract? CHECK." To amuse myself, I have put up stats and then run stats on how many people actually looked at the stats: they don't.

That having been said, the most important questions are: 1) How many "hits"? 2) What pages are hit the most and the least? 3) Geographic region (and that's only a guess, you'll get a lot of people from "Virginia" from AOL alone), and 4) Who are your biggest referrers?

The rest is voodoo guesswork. Take this part: how have the traditional customers taken to the new platform? Well, how would you know? How could you tell the difference between a lot of new customers and a combination of some new customers and some old customers?

Additionally, how can you tell if it has lessened costs? Both questions require that your site be tied into the already-extant systems for billing and whatnot. These don't exist in a vacuum.

My suggestion would be to collect data and then push it to the accountants, who have systems for measuring this kind of thing. They can do the handwaving.
posted by adipocere at 12:44 PM on February 23, 2008

Where does your traffic come from?
What are visitors interested in?
How long are visitors staying on the site?
If you have specific goals for visitors how often are those goals being met?
For what markets (or keywords) are you high-ranking in the important search engines?
Where do your competitors stand on those same markets?
Do internet search engines correctly identify your competitors?
Is the website creating new customers? Is it creating more sales from existing customers?

Most stats are ignored by CEOs because most web teams produce useless stats, and after a go the executives just give up and assume it's too hard for their team. I mean, who really gives a fuck about the # of hits or the browser they're using if you can't break that down into some sort of business-useful metric.
posted by mosch at 1:09 PM on February 23, 2008

"My viewpoint, based on all of the above is that if we build a state of the art site and make elegant, engaging and highly useful, we will attract a new generation of customer"

Put the bosses aside for a minute. If that is your goal, how will you know you've succeeded? How would you measure your progress?

Elegance is subjective. Winning some design awards or other recognition (ie mentions in general business magazines and mainstream press) might provide a more objective metric of having created a site that appealed to cultural arbiters.

"State of the Art" is also a vague term. I would assume you have specific features in mind. Your progress towards implementing those features would be one metric, use of the features you've implemented would be another. I'd be careful with this one though. If you are making appropriate course corrections along the way, you might find that some of your initial targets for features to implement were mistaken, and that you'd be better off putting your efforts in another direction. This might be the right thing to do, but it could look like you've failed at your original goals.

"Highly useful" is similarly vague. Specifics?

"Attracting a new generation of customer" is more measurable, but probably not something you can measure well, or claim credit for, on your own. You'll need to build alliances in other parts of the business, like the marketing organization.

The first thing they are looking to show is the adoption rate, i.e.: how have the traditional customers taken to the new platform? Has this led to additional sales activity, has it lessened costs (i.e.: do we need to do shorter runs of print catalogues because more customers are now exclusively doing product research with us on-line?), and so on.

Adoption by traditional customers and generation of additional sales activity from that existing customer base seem like hard things to measure passively without an ecommerce component. You'll probably need to resort to things like small user polls, and also various incentive schemes that you execute in conjunction with other departments. For example, you could have deep URLs that aren't linked from other pages, only printed in your, catalogs that list web-only specials. You could have free shipping offers that require people get the code from the website, or just special web-only SKUs for items in your catalogs. If people use the code, or the special SKUs then you'll know they made use of the website, and your firm should already know whether they were existing customers based on name, address, credit card info, etc.

Regarding cost savings on catalogs, don't look at reducing the size of catalog runs. Look at sending out smaller catalogs because people can get detailed information on line. Look at doing catalog runs less frequently because people can find out about new items on the web.

Focusing on efficiencies serving existing customers is all well and good, but it isn't necessarily going to justify your larger ambitions. You need to come up with ways to demonstrate that the web site helps tap into a new market.

One simple way you can do that is to focus on referrer metrics. What are the trends for people discovering your site through a search engine? What are the trends for the keywords that people are using when they arrive on your site? What sites are the sources of your other incoming links? What pages or types of pages are being linked to?

Searchers who arrive at your site using search terms that don't contain one of your company's brands arguably represent new customers (or prospective customers). Searchers who arrive at your site via a search that include one of your brand names at the very least represent an important subset of your existing customer base.

Who else is linking to your site? Are they linking to specific products, product categories, or the home page? Do you get a lot of links from myspace, hotmail, etc?

This is data that you probably have right now, sitting in your webserver logs. You could probably improve it further by adding in some data from a 3rd party web metrics firm who could add some demographic information (for a start, you might just try the free offering from someone like quantcast). For an exec who is ignorant of the web, but has half a clue about business, it would probably blow their minds if you can present it well, you might be able to show pockets of customer interest in areas far outside their traditional customer base that could grow with just a little attention.
posted by Good Brain at 1:39 PM on February 23, 2008 [4 favorites has favorites]

When I did some web development I recorded:

• Hits for every page
• Off-site referrer URLs, with bonus parsing for search-engine URLs so we could see what the popular searches that came to us were
• Searches on our site, by phrase and keyword

If management wanted anything else they needed to come up with what it should be.
posted by krisjohn at 5:56 PM on February 23, 2008

Your bosses may be asking for stats you can't provide:

+ Adoption rate presumes you have stats of some kind for the old platform... do you? And if you do, did you build out in a way that allows you draw any correlation at all between the old site and the new? Probably not, unless this was a specific goal of your initial implementation. More, there's probably no real to believe that "traditional customers" who had no interest in the prior platform would have any more interest in the new one, especially if they are traditional to the tune of "I want to talk somebody / see somebody / have a conversation before I make a purchase."

+ Has this led to additional sales? Since you appear to not be doing e-commerce (actually converting folks from shoppers to buyers online) then you have to use other mechanisms to track conversion rates, and just where your customers are coming from (i.e. customer match-backs across multiple channels.) Are you doing this today? If you're not, then you're flying blind and your management has every reason to be skeptical.

+ Are there cost-savings? I'm presuming that as you haven't actually implemented any mechanism for customers to convert online, you may have likewise not added customer self-service capabilities. Maybe you've been able to reduced, say, the number of call reps required to take orders or to otherwise service folks who've been able to make a product selection online? Failing that, you're likely gonna come up short here, too.

You've got quite a handicap to overcome here. Without e-commerce, an online catalog is still a brochure. It may have lots of gee-whiz stuff going for it, but it simply can't take advantage of the immediacy and efficiencies to be gained from online order-taking. More, you can't take full advantage of a multi-channel (catalog plus e-commerce) scenario, where two thirds or more of your catalog audience may choose to actual fulfill their orders online.

If you bosses really want to be data-driven, then the argument to be made is that by not offering full-services online you're missing the boat; you can gather only incomplete data about customer wants and needs, you have only a narrow view of how and where your products convert, you have no window onto how your various channels work together (or antagonize one another.)

Until you can offer that, the best that you can do is look at your unique and repeat visitor rates, and time spent per visitor, trying to draw a correlation between increased time spent on the site resulting in increased sales through your catalog channel while at the same time perhaps lowering some costs in customer care. That, and get a match-back study underway, right now... they time to implement well.

Finally, *you* need to lead here. If you leave it to your bosses to drive what data is useful and relevant, then you risk being something less than useful and relevant yourself.
posted by deCadmus at 10:20 AM on February 24, 2008

A parting shot:

I *do* run an e-commerce web site. And the stats that are most useful to me are:

1) Conversion Rate: of visitors to shoppers, shoppers to customers, customers to repeat customers.

2) LTBA - Likelihood to by again.

I keep lots and lots and lots of other stats, but these are my site's vital statistics, bar none.
posted by deCadmus at 10:34 AM on February 24, 2008