Lean Analytics Cycle

  • Published on
    11-Aug-2014

  • View
    33.296

  • Download
    3

DESCRIPTION

 

Transcript

  • Lean Analytics Lean Product & Lean UX Silicon Valley - July 2014 Hiten Shah hnshah@gmail.com Download the slides: h p://kiss.ly/leanprod2014
  • Continuous Learning & Improvement
  • Dont know where to start?
  • Wondering if youre working on the right things?
  • Or, you just want to hear me babble!
  • HTTP://KISS.LY/LEANLYTICSBOOK
  • Lean Analytics helps you Optimize for Learning
  • Lean Analytics Cycle AVINASH KAUSHIK HTTP://KISS.LY/LEANAC
  • Metrics Hypothesis Experiment Act
  • HTTP://HELLOBAR.COM
  • What is Hello Bar?
  • Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next
  • Our funnel showed a low installation rate Hello Bar Funnel Metrics Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 15068 1868 716 688 398 % prev step 12.40% 38.33% 96.09% 57.85% % drop off -87.60% -61.67% -3.91% -42.15% % of visitors 100% 12.40% 4.75% 4.57% 2.64%
  • We achieved a 40% increase for our installa2on rate with just this one rst experiment. The Lean Analytics Cycle At Work
  • Original Hello Bar installation screen
  • Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next
  • Find out why people dont install Hello Bar
  • Didnt Meet My Expectations
  • I Had Trouble Installing It
  • I Dont Have A Website
  • Not Ready To Install
  • Other
  • Hello Bar Experiment Hypothesis If more installa2on op2ons are given, then more people will add Hello Bar to their website due to the need for a WordPress plugin and ability to email the code to someone else.
  • Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next
  • Created a new ow to test
  • I can install code myself.
  • 41.67% responded with WordPress Plugin
  • 25% responded with email my developer / web designer
  • I need help.
  • Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next
  • Full funnel metrics for the experiment Variation Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7456 906 342 329 276 % prev step 12.15% 37.75% 96.20% 83.89% % drop off -87.85% -62.25% -3.80% -16.11% % of visitors 100% 12.15% 4.59% 4.41% 3.70% Control Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Totals 7469 925 353 341 197 % prev step 12.38% 38.16% 96.60% 57.77% % drop off -87.62% -61.84% -3.40% -42.23% % of visitors 100% 12.38% 4.73% 4.57% 2.64%
  • Resulted in a 30% installation rate increase HTTP://ISVALID.ORG Experiment resulted in a 40% improvement
  • ACer a dozen more experiments, our installa2on rate is now hovering between 5% and 7% Thats an addi;onal 89% increase!!! The Lean Analytics Cycle At Work
  • Metrics Hypothesis Experiment Act How You Can Put It Into Practice
  • Metrics Figure out what to improve
  • What gets measured, gets managed. Peter Drucker
  • Baselines DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA
  • Funnels KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL
  • SaaS Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL
  • eCommerce Funnel KISSMETRICS FUNNEL REPORT HTTP://KISS.LY/KMFUNNEL
  • Revenue BUFFER BAREMETRICS HTTP://KISS.LY/BUFFERBARE
  • Spreadsheets and Calculations HELLOBAR HTTP://HELLOBAR.COM Steps Visitors Homepage URL Entries Created First Bar Completed Registration Installed Created Second Bar Totals 15068 1868 716 688 398 193 % prev step 12.40% 38.33% 96.09% 57.85% 48.49% % drop off -87.60% -61.67% -3.91% -42.15% -51.51% % of visitors 100% 12.40% 4.75% 4.57% 2.64% 1.28%
  • Hypothesis Make an educated guess
  • A solid test hypothesis is an informed solution to a real problem not an arbitrary guess. The more research and data you have to base your hypothesis on, the be er it will be. Michael Aagaard, h p://kiss.ly/abthypo
  • Research Tools & Tac;cs
  • Customer Development HTTP://WWW.CUSTDEV.COM
  • Jobs To Be Done #JTBD HTTP://JOBSTOBEDONE.ORG
  • Usability Testing HTTP://USERTESTING.COM
  • Live Chat HTTP://OLARK.COM
  • Mouse Tracking HTTP://CRAZYEGG.COM
  • Long Form Surveying HTTP://SURVEYMONKEY.COM
  • On-Page Surveying HTTP://QUALAROO.COM
  • Personalized Messaging HTTP://INTERCOM.IO
  • Email a survey to people HTTP://SURVEYMONKEY.COM
  • Ask people why they canceled HTTP://SURVEYMONKEY.COM
  • Get feedback right when they cancel
  • Exit Surveys HTTP://QUALAROO.COM
  • Discover whats missing HTTP://QUALAROO.COM
  • Find out what words people use to describe your product HTTP://QUALAROO.COM
  • Learn about peoples expectations HTTP://INTERCOM.IO
  • Learn what frustrates people HTTP://INTERCOM.IO
  • More actionable reading QUALITATIVE DATA HTTP://KISS.LY/QUALABT
  • Experiment Hypothesis Formula DATA DNA EBOOK HTTP://KISS.LY/OPTDATADNA
  • Experiment Test your guess
  • Only 1 out of 5 tests win. 4 out of 5 tests cost you money. $
  • 1) Which top metric are we trying to improve? 2) What can you anchor your metric against? 3) What is your hypothesis about why this metric is at x%? 4) What do we expect from this experiment? 5) What was the result and what did we learn from it? HTTP://KISS.LY/6QSAB Answer key questions for each experiment
  • Prioritize your experiments HTTP://KISS.LY/ABPIE
  • Dont copy best practices blindly
  • Learn what works best for you
  • Act Decide whats next
  • Explore the results and make data informed decisions. THE ANALYTICS THAT MATTER TO FACEBOOK HTTP://KISS.LY/FBINFORM
  • KISSmetrics A/B Testing Report KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  • We know that people love Google Analytics
  • KISSmetrics Experiment Hypothesis If people authen2cate with Google, then more people will sign up due to their exis2ng familiarity with Google Analy2cs and alignment with our winning headline.
  • KISSmetrics Homepage Test Control
  • KISSmetrics Homepage Test Variation
  • KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  • KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  • KISSmetrics Homepage New Control
  • Inspiration
  • KISSmetrics Homepage New Test Variation
  • KISSmetrics Homepage Test Signup Conversion Rate KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  • KISSmetrics Homepage Test Received Data KISSMETRICS A/B TESTING REPORT HTTP://KISS.LY/KMABTEST
  • Growth! HTTP://KISSMETRICS.COM
  • Metrics Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide whats next
  • Hiten Shah hnshah@gmail.com @hnshah You will get all you want in life if you help enough other people get what they want. Zig Ziglar Download the slides: h p://kiss.ly/leanprod2014