Reading Kaushik (Part 5): Qualitative Analysis

Got Surveys? Recommendations from the Trenches

  • Work with an expert
  • Benchmark against industry: http://www.foreseeresults.com/, http://www.theacsi.org/, http://www.iperceptions.com/
  • Great insights are in the raw answers to your survey
  • Don’t show the survey too early
  • Think about your target segment that you want to survey
  • Treat them as a ongoing measurement system

Build A Great Web Experimentation & Testing Program

  1. Get over your own opinions
  2. State hypotheses
  3. Create goal and success metrics beforehand
  4. Don’t neglect side-effects of testing
  5. You can start small but get bigger
  6. Get people involved in testing: e.g. let them bid on outcomes
  7. Know the techniques and theories
  8. Evangelize people about testing

The Three Greatest Survey Questions Ever

  • What is the purpose of your visit to our website today?
  • Were you able to complete your task today?
  • If you were not able to complete your task today, why not?

Experiment or Die. Five Reasons And Awesome Testing Ideas

  • Reasons:
    1. It’s no expensive
    2. It’s quite fast
    3. It’s allows you to measure change
    4. You have the ability to take controlled risks
    5. It’s quite easy
  • Ideas:
    1. Fix the worst landing pages and be bold
    2. Test single page vs. multi page checkout
    3. Test number and placement of ads
    4. Test different pricing / selling tactics
    5. Test box layouts and other online stuff

Qualitative Web Analytics: Heuristic Evaluations Rock!

  • Trying to get tasks complete: e.g. place an order, look for support, etc.
  • You can try this in a group
  • Process:
    1. Write down task that you want to see completed
    2. Try to establish success benchmarks
    3. Walk through each task and note important findings
    4. Check with a best practices checklist
    5. Create a report: screenshots / screen recording
    6. Create a prioritized list with all the problems

Reading Kaushik (Part 4): Tactical Analysis

Kick Butt With Internal Site Search Analytics

  1. Understand site search usage: What are they looking for?
  2. On which site do people search?
  3. How good are the search results?
  4. What do people search for after they search for one term?
  5. Measure outcomes

PPC / SEM Analytics: 5 Actionable Tips To Improve ROI

  1. Compare keyword performance for different search engines / PPC sites
  2. Focus on what’s changed, otherwise there’s just too much data
  3. Look at your impression share
  4. How is your ROI distributed – exceeds, meets or underwhelms expectations?
  5. How are the keywords matched?

Analysis Ninjas: Leverage Custom Reports For Better Insights!

  • Start with goals: Where is the company spending money? How is the bonus of your boss calculated? What is the worst thing on your company’s website?
  • Include outcomes
  • Reduce the number of reports
  • Match metrics up to reader – Personalize
  • Talk to people and understand what motives them

3 Advanced Web Analytics Visitor Segments: Non-Flirts, Social, Long Tail

  1. Non-Flirts, Potential Lovers: Page Depth bigger than 3 (depending on the distribution)
  2. Social Media, Baby: Tag your links and track by referrer
  3. Search Queries With Multiple Keywords [3, 4, 5, 10, 20]: Match keyword ->
    ^\s*[^\s]+(\s+[^\s]+){2}\s*$

Three Amazing Web Data Analyses Techniques For Analysis Ninjas

  • Calculate costs / profits for micro-conversions
  • Bring ratios / quotas into context: 0.01% conversion = 100 of 1,000,000 visitors convert
  • Not every visitor is convertible: only return and of return only valuable traffic (e.g. people visiting the pricing info)

Reading Kaushik (Part 3): Strategic Analysis

Path Analysis: A Good Use of Time?

  • Customers like the path the want to, not the one you force
  • The tool don’t show which page in the path was most influential
  • Most tools don’t track the path correctly
  • Exception: Landing Page experience
  • Possible solution: Group relevant pages
  • Show most influential pages/li>
  • Make segmentation easier

Stop Obsessing About Conversion Rate

  • Overall conversion rate doesn’t allow for actionable insights
    • It only covers a small minority of all visitors
    • People are going to research on your site
    • People who want to learn about your company
    • People who need help with a product of yours
    • People that do something completely different
  • That is, you neglect a big part of your visitors
  • You’ll focus on short term gains
  • Better metric: Task Completion Rate
    • Helps you to cover all customers
    • Successes outside from conversions
    • Ultimately understand your customers better

Getting Started With Web Analytics: Step One – Glean Macro Insights.

  • Understand the macro level first
  • 1. How many visitors are coming to your site?
  • 2. Where re they coming from?
  • 3. What is the purpose of your website? What are your top three web strategies you currently working on?
  • 4. What are you visitors actually doing?

Consultants, Analysts: Present Impactful Analysis, Insightful Reports

  1. No data overload: Give value instead of data, provide recommendations
  2. Tie your data to business outcomes
  3. Use other data than just Clickstream
  4. Don’t make it boring
  5. Connect insights with actual data
  6. Meet the “exceptions of scale”: If you are a big agency or written a book on WA, then people expect more from you
  7. Do something unique

Paid Search Analytics: Measuring Value of “Upper Funnel” Keywords

  • Upper Funnel / Longtail Keywords can neglected because of the single session mindset
    1. Understand each stage of the customer purchase life cycle
    2. Map your keywords to each of those cycles
    3. Measure success for each cycle differently
    • Category Keyword: Bounce Rate
    • Category / Brand: Time on Site
    • Brand: Visitor Loyalty
    • Conversion/Product: Conversion Rate / Leads

Aggregation of Marginal Gains: Recession Busting Analytics!

  • Often web analysts don’t focus on the immediately achievable improvements
  • Simple things:
    1. Figure out where you are making money
    2. Check errors in your email campaigns
    3. Use funnels
    4. Fix your top landing pages
    5. Compare organic and paid keywords: Where are gaps between these two and why?
    6. Ask your customer
    7. Fix dumb stuff: e.g. check 25-point Website Usability Checklist

Analyze This: 5 Rules For Awesome Impromptu Web Analysis

  • Question: What would you change on this website?
  • Useful things to remember:
    1. Don’t start with your opinion: you are a proxy for customers / visitors => Better: State hypothesis
    2. Always offer alternatives
    3. Offer data, even when you don’t have access to the site’s data.
    4. State your assumptions about the site’s objectives
    5. Focus on obvious and non-obvious things: micro and macro conversions, competitor’s site, customer satisfaction, ideas for testing, demographic trends, etc.

Web Analytics Segmentation: Do Or Die, There Is No Try!

  • Pick at least some segments in: Acquisition, Behavior and Outcomes
  • #1: Acquisition: Where does the company spent its most money on?
  • How to segment:
    • Context: How many visits?
    • (Optional): How many were new?
    • Bounce Rate
    • What was the cost of acquisition?
    • What value could we extract at a per visit level?
    • How many were able to accomplish their goal?
    • Was what the total value added to our organization?
  • #2: Behavior
    • What are the visitors doing?
    • What do people want to do on your site?
    • How deep are they browsing on your site?
    • How long did they take time till conversion?
    • How often did they visited your site?
    • => Traffic source, conversion, average order value, actions, etc?
  • #3: Outcomes
    • Loot at macro and micro conversions!
    • i.e. video clicks, adding products to wish list, applying for a trial, downloading white papers, etc. etc. etc.

5 + 4 Actionable Tips To Kick Web Data Analysis Up A Notch, Or Two

  1. Go deeper — Don’t stop at the obvious border: compare off and online data, create CLV for ecommerce,
  2. Join the People against lonely metrics club
  3. Measure the complete site success
  4. Don’t just report one month: at least three months, understand your business’s cycles, create annotations
  5. Make insights in your data obvious: Better visualizations
  6. Segment your data (previous summaries)
  7. Don’t just look at the top 10 rows
  8. Step away from one-session thinking (later summary)
  9. Achieve multichannel analysis (previous summary)

Win Big With Web Analytics: Eliminate Data & Eschew Fake Proxies

  • Most reports are overloaded with data
  • Though, the readers want context and insights
    1. Eliminate all useless metrics and data in your reports
    2. Understand the desired outcomes

Rebel! Refuse Report Requests. Only Answer Business Questions, FTW.

  • Context is important – no business is the same
  • Answer business questions – what’s the driving request for the data?
  • Attributes of a business question:
    • They are usually open-ended and on a higher level
    • They need likely more information than just visits, bounce rate, etc.
    • They are seldom answered with tables

The Difference Between Web Reporting And Web Analysis

  • Data puking isn’t web analysis
  • Signs that you are doing web analysis:
    1. Actions instead of data
    2. Work with the business, measure economic value
    3. Use the web analytics measurement model (previous summary)
    4. You are doing a bit advanced statistics
    5. If you work with targets
    6. You provide context
    7. You segment your data effectively
    8. If you can provide an impact for a recommendation
    9. If you use less than four metrics in a table
    10. If you use multiple data sources

Reading Kaushik (Part 2): Digital Analytics

Data Quality Sucks, Let’s Just Get Over It

  • Data quality in the web is not great
  • Six step plan:
    1. Don’t dive deep into the data to find discrepancy between data
    2. Assume a level of comfort with the data
    3. Start making decision that you are comfortable with
    4. Over time start understanding small areas of data
    5. Get more comfortable over time with your data
    6. Absolute numbers rarely matter, segmented trends do

Tips for Web Analytics Success for Small Businesses

  1. Get top key phrases from search
  2. Get top referring URLs
  3. Which content is popular on your site?
  4. Percentage of Visitors on the homepage
  5. Check segmented click densities
  6. Learn about your site’s bounce rates

Measuring Success for a Support Website: A Point of View

  • Moment of Truth: hold or lose customers — web: often support problems
    1. Don’t measure unique or total visitors
    2. Identify top methods to customers find information
    3. Click Density Analysis for the Top FAQ pages
    4. What percent of site visitors call the support phone number?
    5. Measure: Problem resolution, timeliness, likelihood to recommend
    6. Check if the top solutions are corresponding to the top real problems (call center, user forums, blogs, etc.)

Seven Steps to Creating a Data Driven Decision Making Culture

  1. Go for the bottom-line (outcomes): What motives the people around you?
  2. Reporting is not Analysis
  3. Depersonalize decision making
  4. Proactive insights rather than reactive: Deliver insights before someone asks
  5. Empower your analysts: They are not reporting monkeys. Give them strategic objectives
  6. Solve for the Trinity: What, Why, How much?
  7. Create a understandable, repeatable framework for making decisions
  8. Business/Strategy should own web analytics

Five “Ecosystem” Challenges for Web Analytics Practitioners

  1. Lack of relevant talent / skills: No real format training; too much experience requested (5+ years) although the field is moving fast
  2. Entrenched mindsets: Decision makers still thinking in the old way
  3. The web is no longer a monologue
  4. It’s not about you, it’s about your customers: Less clickstream, more experimentation, usability, integration of multiple sources
  5. Web analytics is the first step

Web Analysis: In-house or Out-sourced or Something Else?

  • In the long run: in-house team
    • strategic implementation of WA can’t exist in a silo
    • Qualitative analysis is also needed
    • Tribal knowledge helps the decision making
  • But, it depends on the stage:
    • Stage 1 – No WA -> Implement and show promise from data
    • Stage 2 – Too much data -> Hire WA, customize dashboards, tag everything
    • Stage 3 – WA rocks; Asking the why -> Start testing, collect qualitative data, expand your team
    • Stage 4 – Trinity implemented -> new data, more people, a self-sustaining process

Five Rules for High Impact Web Analytics Dashboards

  1. Benchmark & Segment: Provide context
  2. Isolate your critical few metrics: less than 10 metrics
  3. Include insights
  4. Don’t create more than one single page
  5. Constantly adapt your dashboard to changes in the environment

I Got No Ecommerce. How Do I Measure Success?

  • Visitor Loyalty: How often does one person visit my website?
  • Recency: How long has it been since a visitor last visited your website?
  • Lengths of Visit: How long does each session last?
  • Depth of Visit: How many pages did they visit?

Convert Data Skeptics: Document, Educate & Pick Your Poison

  • Understand how your tools actually measure
  • Document your findings
  • Present everybody touching the data your findings
  • Report high level trends between the tools
  • Pick the best tool for each metric

Is Conversion Rate Enough? It’s A Good Start, Now Do More!

  • There is more stuff than just conversions
  • Esp. for non-ecommerce businesses, ask:
    • Have you found what you were looking for?
    • Will you go to our store?
    • Will your recommend our website?

History Is Overrated. (Atleast For Us, Atleast For Now.)

  • Value of web analytics data decays in time
  • Your visitors change too much: browsers, cookie deletion, etc.
  • Your computations change too much: new computations, maybe other tagging, etc.
  • Your systems change too much: Other hosts, new technology, etc.
  • Your website changes too much
  • Your people change too much
  • => no real tie to legacy tools and data
  • => keep some major benchmarks for comparison

“Engagement” Is Not A Metric, It’s An Excuse

  • Engagement is unique – therefore say what you actually measure instead of saying that you measure engagement.
    1. What’s the purpose of the website?
    2. How do you measure success?
    3. Define your metrics
    4. Call them what they are
  • Ideas:
    • Question: Are you engaged with us?
    • Question: Likelihood to recommend website
    • Use primary market research
    • Customer retention
    • RF of customers

In Web Analytics Context Is King Baby! Go Get Your Own

  • Never report data in aggregate, or by itself. Always always always test to see if you are including context!
  • Compare trends over different time periods
  • Compare key metrics and segments against site average
  • Report multiple metrics
  • Benchmark
  • Tap into the tribal knowledge

The “Action Dashboard” (An Alternative To Crappy Dashboards)

  • Why do dashboards suck?
    • Don’t provide a interpretation
    • Readers don’t trust the providers of dashboards
    • Not enough company context in the interpretation
    • Providers don’t have enough experience
  • How to make a good one?
    • Report only 3-5 most critical metrics
    • Show a trend in a graphic for a metric (segmented)
    • Give key trends and insights
    • Recommend actions
    • What are the impacts of the company

Multichannel Analytics: Tracking Offline Conversions. 7 Best Practices, Bonus Tips

  • Track your online store locator, directions, etc.
  • Use unique 800 numbers
  • Use unique coupons / promotions
  • Connect online and offline – e.g. club cards, delivery, etc.
  • Ask your customers (survey)
  • Conduct controlled experiments
  • Do primary research

Multichannel Analytics – Tracking Online Impact Of Offline Campaigns

  • Use vanity urls: Permanent redirects help you to differentiate between offline and online referals
  • Use unique coupons / offer codes
  • Survey, survey, survey
  • Correlate traffic patterns with offline ad patterns
  • Experiment

Slay The Analytics Data Quality Dragon & Win Your HiPPO’s Love!

  1. Change your boss
  2. Compare web data with their favorite source
  3. Put the data quality problem aside and give them actionable insights
  4. You get trends rather fast even without a complete WA implementation
  5. Start with the outcomes
  6. One WA is probably enough
  7. Realize if the data quality is good enough
  8. If you don’t have enough traffic, care about more traffic first
  9. There are inaccurate benchmarks and illegal customer behavior – don’t care too much about it
  10. Fail fast, i.e. test

Barriers To An Effective Web Measurement Strategy [+ Solutions!]

  • Note: Tools aren’t the real problem, still lots of people talk about them
  1. Lack of resources (45%): Start for free and ask your right for a budget
  2. Lack of strategy (31%): Change your job. If your a VP maybe you can help create one
  3. Siloed organization (29%): Start small and offer value
  4. Lack of understanding (25%): Do and show
  5. Too much data (18%): Limit yourself on the critical few metrics.
  6. Lack of senior mgm buy-in (18%): see previous summaries
  7. Difficulty reconciling data (17%): Whatever.
  8. IT blockages (17%): Show lost revenue by delay
  9. Lack of trust in analytics (16%): previous summaries
  10. Finding staff (12%): Don’t be to narrow minded
  11. Poor technology (9%): Concerns mostly only brand new technologies / platforms

Who Owns Web Analytics? A Framework For Critical Thinking

  • The biggest problem is most often the organization structure.
  • How long has the company been doing WA? -> New: find accepting division to embarrass the seniors – Older: see next point
  • Analytical maturity? New: find accepting people – Older: identify power centers
  • Who owns the power to make changes to the site?
  • Which model: Centralized? Decentralized? Something else? Agile team with satellites
  • Which division / department is best for WA? Marketing if they have the power -> Ideal situation: own department

10 Fundamental Web Analytics Truths: Embrace ‘Em & Win Big

  1. If you have more than one clickstream tool, you are going to fail: Implementing, understanding and communicating one tool is hard enough
  2. Expensive tools won’t give you insights: Real problems are lack of skills, terrible orga, no structure or no courage.
  3. It is faster to fail and learn then wait for an “industry case study” or find relevancy in a “industry leader white paper”
  4. You are never smart enough not to have a Practitioner Consultant on your side (constantly help you kick it up a notch)
  5. Your job is to create happy customers and a healthier bottom-line
    • Go to your own website
    • Read your own email campaigns
    • Buy something on your own website
    • Return a product on your own website
    • Do the same stuff on competitor’s websites
    • Do usability studies
    • Be a customer and ask yourself: What will create happier customers tomorrow?
  6. If you don’t kill 25% of your metrics each year, you are doing something wrong
  7. A majority of web analytics data warehousing efforts fail. Miserably: Too much irrelevant data, mostly anonymous, full of holes, BI are bad at answering WA questions and DWs are too slow
  8. There is no magic bullet for multi-channel analytics: see previous summary
  9. Experiment, or die.
  10. The single most effective strategy to win over “stubborn single-minded” HiPPO’s is to embarrass them.

Measuring Incrementally: Controlled Experiments to the Rescue!

  • Test everything
  • You may need additional personnel for conducting and analyzing the experiment
  • It gives you excellent insights in different methods
  • Act on your results as fast as you can