Hello. My name is Frankie Johnson and Research Arts is the company I founded as a full-service qualitative market research firm almost 30 years ago. For several years, researcharts.com was also a blog where I shared my thoughts on consumer trends and market research, with occasional rifts on marketing strategy. The most popular of these blog entries are now renamed “articles”. I’ll be adding content from time to time, with the intention of focusing on topics that have a longer shelf life than most blog entries. If you wish to be informed when new articles are added, please subscribe with your email address in the box to the right. If you are already a subscriber, thank you!

Yesterday was a slow day, slow enough to spend an hour or two browsing my Twitter stream. I came across this tweet from Dana Stanley of iCharts:

@DanaMStanley A one-page report can deliver more value than a 50-page report

Hmmm. Got me thinking about brevity in research. And being a VERY slow day, that led to issuing a challenge to my friends on Twitter to come up with a Haiku about market research under the hash tag #HaikuMR.

Tweets have a very short shelf life. I’d hate to lose the gems that were returned. So here they are, deep and sometimes quite lovely. I invite you to add your own in the comments or in Twitter under #HaikuMR. Would be nice to keep this going.

@frankie_johnson (me)

This short report | Even the monkey seems to give | A small bite of truth

Mirror on the wall | A window for watchers | Too timid to decide

@DanaMStanley, Dana Stanley, iCharts

Storms rage in my soul / Report is paid for and bought / Give me takeaways

Need I bear this dreck / Alas, it is a brief one / Now I can be free

@blexy Nicholas Blexrud – Decipher, another Portland lad

Pick One and No More | Fav Movie of ’94 | Dumb and Dumber, score.

Male or Female, No? | An Alien I deem you. | Male or Female though?

@lennyism Leonard Murphy of Greenbook Blog and BrandScan 360

Another boring scale / Where is the fun in that? / MR gets it wrong

@Kvistbo, Dan Kvistbo of Norstat, in Denmark, outperforming us all and worthy of Hamlet

Crosstabs in the Sand | Too fragile a Foundation | Washed out by the Sea

Cloud of unknowing | Data is your Enemy | Research – a Blue Sky

The Spring has arrived | Oh my Data is blooming! | A Scent of Insight

A River of Data | Flooding my Spreadsheet again | Bursting through Axes

@Itracks, Terri Sorenson in Canada… pointing out a tempting benefit of online qual.

Capturing Insights | In the summer from the beach | Online Focus Group

@philtysoe, Phil Tysoe, clientside in UK and a good dad

Please rate my short poem / On a scale from one to ten / Why do you say that ?

Thanks everyone for doing this. I’m going to file it under qualitative research… ;^)

You may have heard the quote from that greatest of scientists, Einstein. “Not everything that counts can be counted.” It sums up very nicely why quantitative research will never tell the whole story. And with the advent of social media and its spotlight on the richness of interwoven relationships, this is more apparent than ever.Take a look at this photo. How do we capture quantitatively the complexity of feelings and associations this may arouse in us? We can’t.

Qualitative is the type of research that doesn’t count – literally. No numbers, just listening to words, watching for the little things, looking for patterns, sometimes weaving in a little poetry. It attempts to (and sometimes does) understand people’s decision making in the context of the mess of their real lives. It recognizes that we live with ambiguity. It doesn’t force answers into neat packages that can be squeezed into a behavioral model. It doesn’t assume that people’s attitudes and behavior can be categorized as variables that are treated as independent of each other. There is none of the cetera paribus of economic theory. Other things have a way of not remaining static and equal; they don’t stand still long enough.

But let’s not forget the “research” part. Listening to people talk in social media settings, or behind you on the bus, can be highly informative, but it’s hardly research. Taking what you have heard, attributing it to what you know about the source and looking for patterns and connections – now that’s research. You have to actually do something with the ingredients to be called a cook.

After a number of problems with my home phones, I recently decided to buy a new system – the wireless kind with several extensions. I shopped at the sites where I usually go for this sort of thing, Amazon and Best Buy. The reviews on Amazon helped me choose a particular Panasonic model, but the price on the Best Buy site was $20 less with free shipping. I placed the order on a Monday morning and was very pleased to receive it via UPS the very next day. Up until this point, Best Buy did everything right. They carried the product I wanted, at the best price, made ordering simple on their site and exceeded my expectations in the speed of shipping. The UPS driver even left two dog biscuits on top of the box at my doorstep. Very happy customer. More likely to shop online from Best Buy in the future.

So everyone on the Best Buy team did a great job. That is until the market research folks got involved. A couple of days after receiving the product, I got this email from them:

Dear Francesca,
In an ongoing effort to improve our services, we’d like to hear about your experience of having BestBuy.com products shipped. Please click the button below to fill out a brief survey about your recent purchase on 03/13/2011, including Panasonic-DECT 6.0 Expandable Cordless Phone System-KX-TG6413T. It should only take a few minutes.
Thanks for helping BestBuy.com serve you better!

Well, yes, of course I’ll help. I’m eager to tell you how happy I am with the whole experience. And you promised it would be a brief survey. Love those. So I click the link and immediately had a bucket of ice-cold water thrown over my eagerness to help. THIS is brief?

Well first there’s the intro.

Customer Satisfaction Survey
Thanks for shopping at BestBuy.com. You’ve been randomly chosen to take part in a brief survey to let us know what we’re doing well – and most important, which areas could use some improvement. All input you provide is strictly confidential.
Required questions are denoted by an *

It could have been a little warmer, and not dominated by the researchy title, but it’s the “required questions” part that’s really a turn off. Translation: if you skip any questions, you will be eliminated. And of course, any normal person with a life, will quit the survey once they encounter a “required” question they can’t answer.

Then comes the “brief” survey. I’ve reduced it down but you can expand it by clicking on each part. It’s long, very long. It is also deadly boring, mostly because of the rating scales. I arrived at this point feeling very happy about my buying experience, and then I’m faced with this… this chore.

I don’t usually criticize companies in public, and I do unconditionally recommend Best Buy if you are looking to buy a phone system online. But come on, guys. Everyone on your team, everyone in the marketing chain is doing a great job and you are letting a survey muck it up.

There is this unspoken agreement among professional researchers not to name names when it comes to bad surveys. There is also a myth that market research should be left to the pros because users of DIY tools will write lousy questionnaires. Really?

This type of survey has to stop for many reasons:

1. It’s bad for business and actually leads to customer dissatisfaction. Speaking of which, we should be measuring dissatisfaction not satisfaction.

2. This is an example of the kind of market research nerdiness we are trying to grow out of. The market research nerd doesn’t know how to talk to a girl. If he did, the encounter with the customer would go something like:

Are we good?

Is there anything bothering you?

Wanna talk about it?

I’ll just listen while you talk. I won’t ask you any more questions until you tell me about it.

What can I do to make it up to you?

3. Most importantly, this is bad for the whole market research community. Not only am I less likely to respond to this survey, next time I am invited to give my feedback in a survey, I am less likely to respond to that one too. In fact, I was shopping a couple of days ago on the website of a home improvement store, and up pops an invitation to take part in a survey, also from Foresee Results. I couldn’t close the window fast enough. Fool me once, shame on you. Fool me twice, shame on me. This hurts us all, even those of us who have already cleaned up our act, shortened surveys and banished grids. And if the only way we can stop bad surveys is by exposing them to the sunlight, then let’s do it.

4. The people who do complete surveys like this probably are not representative of the customer base. Rather, feedback is from those few brave souls who made it to the end of the survey with no payment except the warm glow of knowing that they had helped a major brand. The survey even ends with the advice to call Best Buy if you personally want to be heard. In other words, no one is going to read your heartfelt comments or follow up with you. The implied message is that “our time is more important than yours”.

Although you will likely not receive a response from Best Buy for your survey comments, you can get an individual response to your concerns by visiting the Contact Us page on BestBuy.com, or by calling 1-888-BEST BUY.

5. So this leads me to ask just what is being measured here? Who exactly is being counted when it comes to satisfaction? What does a rating mean when there is no chance to opt out or say “I don’t know”?

By way of contrast, I give you Netflix. I’ve been a loyal customer for years. Very rarely I have a problem with a disk, and it immediately gets resolved by a quick replacement. Netflix assumes that everything is ok if they don’t hear from me. The only surveys I’ve received from them are one question emails to check the quality of instant streaming or the shipping speed of DVDs. Here’s a typical Netflix survey sent within email.

When you click a response, you’re taken to this message on the site:

This is not only a good way to get feedback from customers, it also drives traffic to the site. A good example of market research supporting customer satisfaction instead of eroding it.

So there you have it. I hope other researchers decide to call out those surveys that are continuing to damage the market research profession and drive down response rates. Though I recognize that doing so isn’t exactly a great career move. And I hope that Best Buy understands that I do this is out of love. I’m still a very happy customer.

I struggled with whether to write this post. It’s dealing with a subject touching on personal values and I don’t want to offend anyone. But it does seem important. And I’m wondering if I’m an outlier when I fret about what kind of work to take on and what to avoid. We in the market research industry seem to think about ethics mainly in terms of how we conduct ourselves as we interact with respondents, clients and field. We strive to act with integrity and to be honest and worthy of trust. Most people I know in our industry are decent and sincere in their efforts to be ethical as they go about providing the guidance to help their clients succeed.

But we don’t seem to talk very much about whether some clients are worthy of our help. Again, it seems akin to talking about religion, and a little dangerous. It’s also bad for business. Once you begin to openly criticize one company’s products and the harm they might cause, you may as well kiss that company’s business goodbye. And along with it, many other companies who fear they might be the next on your hit list.

Over the years I’ve avoided working in certain areas. I’ve not been very vocal about my exclusions, unlike some other people like Alex Bogusky who started www.fearlessrevolution.com to use his marketing skills to challenge the brands he once championed. At Research Arts, we never accepted assignments from tobacco companies – that was easy. We did a lot of work for food companies and that seemed ok with me – until it wasn’t. I conducted a study for a very large global company among 8 to 12 year olds. Their goal was to understand how to get kids to demand more of their products (soft drinks and junk food snacks) in the vending machines at school. Micro-marketing they called it – building sales one school at a time. They paid me very well to moderate the groups and write the report. But I didn’t like myself for doing it and still don’t. That was the last project I accepted in that category.

As a side note, I’ve never really bought into the idea that I can counterbalance harm with an equivalent amount of good – that the project I did for Heifer International somehow made up for the one I felt badly about. Or that the need to earn money to support my children made up for doing work for brands that relied on the cheap labor of someone else’s children. Well, you get the idea.

I prefer not to be explicit when I turn down work that I feel contributes to making our world worse rather than better. It seems smug and high-handed. I simply say I am busy. For example, I don’t take projects for products that exploit animals for food. I’m not a strict vegan but I won’t work on anything that depends on factory farming. In reality, it’s hard to be so absolute about everything. These days, many companies are trying their best to incorporate sustainable practices. And I believe that within these companies, there are many people who care about the same things that I do, and are working on the inside to implement change.

So it’s complicated, and I understand why this subject is rarely discussed professionally. Most of us compartmentalize and compromise, separating our paid work from our work on the causes that we care most deeply about. You are lucky indeed if you work exclusively in an area that feeds your soul.

You probably have your own lines. I’m guessing you wouldn’t take on a project from a group advocating child pornography or heroin sales. Those are easy black areas. It’s the gray areas that are so troubling. At the end of my career, which is now closer than the beginning, I want to feel that my work in market research did some good or, at the very least, did no harm.