Testing is at the core of every direct marketer/cross-channel retailer's DNA. Without it, you're left wondering what works and what doesn't. A well-executed test — be it an A/B split, multivariate, in-house focus group, etc. — provides insight into what the consumer is going to do next. In a session at last week's
eTail East conference in Baltimore,
Archie Miller, senior manager of web design and development at consumer electronics retailer
Crutchfield, detailed how his company uses in-house testing to learn about consumers.
Running at least one test per month at its corporate offices in Charlottesville, Va., Crutchfield tests its website, in-store kiosks, emails and even catalogs. The tests are compromised of three to four participants, who are generally recruited from
Craigslist and are compensated with either $60 in cash or an $80 gift card (people almost always take the gift card, Miller said). The tests require two conference rooms — one in which the testing takes place, another where Crutchfield employees watch the tests via video — and generally take 45 minutes to an hour to conduct, then an hour to two hour debriefing period after the testing.
The equipment Crutchfield uses for its testing is minimal and rather inexpensive (Miller estimated that the total cost is around $2,500). It includes a web cam, microphone, basic computer,
TechSmith morae usability software and an A/V cart. One Crutchfield employee serves as the greeter and facilitator of the test. The paperwork Crutchfield puts together for any given test includes recruiting forms/letters, consent forms, intro scripts for test facilitators and thank-you cards sent to test subjects.
With everything in place, testing can begin. The stakeholders in the test (i.e., Crutchfield executives watching from another room) all see the same thing at the same time, Miller said. The goal is to find those “a-ha moments,” where Crutchfield can learn from the actions of the test subjects — the homepage doesn't provide for easy navigation to a particular product, for example.
The secret recipe Crutchfield bakes into every in-house test it conducts is one-part participation, one-part discussion, two-parts consensus and four-parts action. Participants can't do anything wrong; they're there to provide insight into consumer behavior. Just one person mentioning a problem is enough for Crutchfield to consider a change or action, Miller said. Here are some of his tips to running an effective in-house testing program:
Social Media ROI
Email Marketing that Works (2nd Edition)
