This article retells the story of how I stumbled upon how an enterprise offering can affect the effect of confidence people have in a freemium product by 4%.

So A/B testing… I favourite topic of mine… And yours I hope!

Personally I will test anything! It is amazing how often this leads into insights you didn’t expect, but later seem like no brainers… here is an example:

Your website is unlikely to have a single goal, at import.io we have two main inbound goals when it comes to marketing.

  • Get as many new self service users as possible
  • Get as many people enquiring about our enterprise offering as possible.

So I am optimising the website to reduce barriers to entry, to make sure as many people signup and enquire as possible. Great. But as with any test I need to make sure there is no negative impact…

The homepage for instance is a prime example, it’s our main conversion page for self service users, but also a highly trafficked page from which we can direct traffic towards other goals. In an effort to funnel more traffic to an enterprise conversion page I added a blue ribbon with a link, and then I tested in which position on the page did a) the most traffic go to the enterprise page while b) having the least negative impact on the number of self service signups.

Original:

Screen Shot 2014-06-26 at 16.42.25

Test page:

co

In this test we saw that the higher up the page you put the Link the more people click it!!! I know right…*shocking*

enterprise page link clicked

Screen Shot 2014-06-26 at 16.34.44

Weirdly not only did the number of enterprise page visits go up, but also the conversion rate of visitors to ‘self service users’ increased too rather than decreasing as I expected.

Self service link clicked

dtr

Talking with a colleague and fellow ‘Data Punk’ Ignacio, we decided that the increase in ‘self service users’ was down to the increased confidence in the startup product given by the idea that it is a business tool too. After all, a new tool may not be worth the effort if you perceive it to potentially be buggy or incomplete.

At this point we don’t know if were are right in out theory.. but I will test it out.

TIL: Forget your pre conceptions when it comes to testing. If I had listened to the small nagging voice who didn’t want to risk user signup rates imagining they would be lower with a enterprise link taking people away from the Download button I would have never discovered the increase, and been able to form a new theory.

Leave a Reply

Your email address will not be published. Required fields are marked *