Reading Kaushik (Part 5): Qualitative Analysis

Got Surveys? Recommendations from the Trenches

  • Work with an expert
  • Benchmark against industry: http://www.foreseeresults.com/, http://www.theacsi.org/, http://www.iperceptions.com/
  • Great insights are in the raw answers to your survey
  • Don’t show the survey too early
  • Think about your target segment that you want to survey
  • Treat them as a ongoing measurement system

Build A Great Web Experimentation & Testing Program

  1. Get over your own opinions
  2. State hypotheses
  3. Create goal and success metrics beforehand
  4. Don’t neglect side-effects of testing
  5. You can start small but get bigger
  6. Get people involved in testing: e.g. let them bid on outcomes
  7. Know the techniques and theories
  8. Evangelize people about testing

The Three Greatest Survey Questions Ever

  • What is the purpose of your visit to our website today?
  • Were you able to complete your task today?
  • If you were not able to complete your task today, why not?

Experiment or Die. Five Reasons And Awesome Testing Ideas

  • Reasons:
    1. It’s no expensive
    2. It’s quite fast
    3. It’s allows you to measure change
    4. You have the ability to take controlled risks
    5. It’s quite easy
  • Ideas:
    1. Fix the worst landing pages and be bold
    2. Test single page vs. multi page checkout
    3. Test number and placement of ads
    4. Test different pricing / selling tactics
    5. Test box layouts and other online stuff

Qualitative Web Analytics: Heuristic Evaluations Rock!

  • Trying to get tasks complete: e.g. place an order, look for support, etc.
  • You can try this in a group
  • Process:
    1. Write down task that you want to see completed
    2. Try to establish success benchmarks
    3. Walk through each task and note important findings
    4. Check with a best practices checklist
    5. Create a report: screenshots / screen recording
    6. Create a prioritized list with all the problems