We hear the phrase "new and improved" all the time. Of course, the reality is that your business doesn't get to tell its customers what is an improvement and what isn't. Your customers will make that decision themselves. But you do get to decide whether you ask your customers before you roll out your new product, or wait for them to tell you afterward, possibly by taking their business elsewhere.
Preference testing is one technique that can be used for getting feedback before rather than after a change. In this example, the change involved the design of a schedule on a web page. Here's the original look of the schedule:
We were considering a new design similar to the "week at a glance" view in many scheduling programs:
Personally, I liked the new design better. But it wasn't my opinion that mattered, it was the users. So, we presented a dozen users with both schedule designs. For each design, we asked them to find three pieces of information in the schedule, so they would base their judgments on ease of use and not just appearance. We then asked them which design they preferred, and why.
Somewhat to my surprise, 11 out of the 12 users preferred the original version. When asked why, they talked about finding it easier to scroll up and down than to click through different screens. They also talked about using the "find" feature in their browser to locate specific content, which was easy to do in the original design but not the new design. Finally, they made reference to having control over the amount of information to view at one time. The original design allowed each week to be collapsed or expanded, while the new design only permitted users to view a week at a time.
Of course I was a little sad that users didn't like the new design. But that was more than made up for by the fact that we'd learned this before rolling it out for the entire world to see.
If you'd like to learn more about preference testing, check out this blog post: Preference testing: What to do before you run A/B tests