Marketing Research Techniques
“To Qual or to Quant”, that is the question. Well Hamlet didn’t quite quote those immortal words, obviously. In the data world this question isn’t quite so rhetorically used. Typically a stakeholder will ask a question and they expect one result. But, if you were to ask a data scientist the same question, they will always reply with the same answer “It depends”.
This drove me mad for years but now I get it. People just don’t ask specific enough questions.
Let me explain further…
Information is interpreted differently by different people, we know this. As a human reading this (sorry bots) you will display prejudices even if you don’t realise it. A person will use their past experience and their current view of the world to process information. Subconsciously we take in all the information and we think we are making a rational decision.
(If you don’t think you have any prejudices then a good test is to change your media sources for a week and see if you think differently.)
One of the main reasons for these poor questions is that people don’t like to disagree with other people or to question the validity of what they hear. Why? Because it’s uncomfortable.
As Cialdini states in his book “Influence”, people in authority positions are able to have huge amount of influence over their subjects be they parents, doctors or teachers. This leads to a principle called Confirmation bias and highlights the importance of teaching critical thinking skills alongside debate and negotiation both in school and in the workplace.
As an example, if we hear a politician say “GDP has increased by 2.2%”. Is this true? Again it depends. The first thing we need to identify is the source and if they are reputable. Secondly what time period are we comparing. If they are comparing quarter 4 against quarter 3 then they are not using any basic analysis techniques as seasonality is a critical function is time series analysis.
The key takeaway here is whatever the statement, question it; go one level deeper (tip: the most useful information in any analytics software is always in the secondary reports).
Other things you can do…
If you watch Question Time or similar go to https://fullfact.org/ to do a fact check (even if you don’t watch it go there anyway as it will train your brain to question people). If you get a dataset ask for a glossary, ask your stakeholder the specific purpose of the project. Quite often you end up wasting a lot of time doing unnecessary work because the expectations were not clear.
So how does this answer the question “To Qual or to Quant”, which is best?
When I started studying marketing this question was debated over and over again. Do we use qualitative data to do marketing research or quantitative data. Firstly, let’s clarify the difference.
Qualitative data is what people think, feel and do. This is often referred to psychographic information. You can think of this as people’s attitudes, opinions or beliefs.
Quantitative information is anything that can be quantified (think of quant sounding like count). These are your numbers, percentage, graphs etc. This is typically the focus of many companies as they are trying to unlock potential bottlenecks in their business and numbers can be used to compare anything.
In my mind, qualitative information is more valuable but much harder to scale. Running 10,000 interviews with your target audience is time consuming and is difficult to analyse whilst getting 10,000 data points can take minutes. Costs are also significantly higher in any qualitative research.
The truth is both are equally important and should be used together. You may have a hypothesis that users on your website are leaving on a certain page and your analytics software backs this up with a 90% exit rate.
THEN you find out a large proportion of leavers are using mobile while the desktop users happily continue to peruse your site. At this point you should switch focus to qualitative research. There are a number of things you can test. Run a site survey, use a session recording software (Hotjar, Crazy Egg) to back up what the data is telling you. Or go find a people who fits your target customer and ask them to test your website.
A good analyst will always switch between the two. They will never spend 2 hours randomly watching Hotjar sessions. This would be like looking for a needle in a haystack (a sin I’ve been guilty of in the past). The correct way is to use you quantitative information to identify hypotheses then run record your session recording based on these.
After watching these recordings, you may find you need to go back to your analytics software to research a new hypothesis. This can be your “Circle of Discovery”
Any questions please let me know in the comments, I’m always happy to help (or have a debate!)
Ian Perry – Data Analyst at Rebelhack