Got Surveys? Recommendations from the Trenches
- Work with an expert
- Benchmark against industry: http://www.foreseeresults.com/, http://www.theacsi.org/, http://www.iperceptions.com/
- Great insights are in the raw answers to your survey
- Don’t show the survey too early
- Think about your target segment that you want to survey
- Treat them as a ongoing measurement system
Build A Great Web Experimentation & Testing Program
- Get over your own opinions
- State hypotheses
- Create goal and success metrics beforehand
- Don’t neglect side-effects of testing
- You can start small but get bigger
- Get people involved in testing: e.g. let them bid on outcomes
- Know the techniques and theories
- Evangelize people about testing
The Three Greatest Survey Questions Ever
- What is the purpose of your visit to our website today?
- Were you able to complete your task today?
- If you were not able to complete your task today, why not?
Experiment or Die. Five Reasons And Awesome Testing Ideas
- It’s no expensive
- It’s quite fast
- It’s allows you to measure change
- You have the ability to take controlled risks
- It’s quite easy
- Fix the worst landing pages and be bold
- Test single page vs. multi page checkout
- Test number and placement of ads
- Test different pricing / selling tactics
- Test box layouts and other online stuff
Qualitative Web Analytics: Heuristic Evaluations Rock!
- Trying to get tasks complete: e.g. place an order, look for support, etc.
- You can try this in a group
- Write down task that you want to see completed
- Try to establish success benchmarks
- Walk through each task and note important findings
- Check with a best practices checklist
- Create a report: screenshots / screen recording
- Create a prioritized list with all the problems
Again a great video, this time about Web Analytics by Avinash Kaushik. I just love his no-BS style.
- Ask the metric: So what? Three times, if it don’t give an action it’s useless
- Data should drive action
- Give people the information they need – don’t send them everything => no death by data
- Home pages of websites, are no longer the home page you want
- Where do people come from?
- What are they looking for?
- Context matters: previous months, years, etc.
- Relative numbers more important than absolute numbers
- Compare different metrics, e.g. conversion rate and page views
- non e-commerce sites:
- averages hide truth effectively
- How often do they visit?
- How recent did they visit?
- Depths of visit
- => Understand the value: Loyalty
- Segment people
- Survey people: What do they think about the content?
- Bounce rate: Came and left
- Segment by source, entry-page, landing pages, etc.
Rules for Revolutionaries
- 10/90 Rule: $10 Tools, $90 People: Understand Data & Business, Able to analyze => to extract value
- Reporting is not analysis: Reporting -> provide data; Analysis -> prove insights
- Data Quality can be low, but is still better than other data
- Faith-based initiative: e.g. magazine ad without tracking
- Make decisions, don’t argue about the quality of the data
- Over time understand why quality is different -> confidence will get better
- Decision making is a journey, not a destination
- => Put some level of process in place, mostly for tasks, e.g. what happens to implement a test, etc.
- if HiPPo (highest paid person’s opinion) makes the most decisions
- => make experiments
- Learn to be wrong, quickly
- => You probably don’t know what your customers want
- => Experimentation