This doesn't sound like the complete set of metrics that should be measured. Surely, the key thing is how much money each model made? What is the average revenue per user in either case for those users converted?
Later on the average lifetime value of a user in each model is needed (plus the cost to get and convert them).
Yes, conversion is part of the equation, but this feels like a vanity metric on its own.
Just so I'm not sounding pedantic, our SaaS tool has a conversion rate of over 40% because it's completely free, permanently. That's the boundary case of moving the pricing.
Just looking at this for one of the first times (new to this part of the game), it seems like this is a step-by-step progression from 'we have users!' to 'users are paying!' If I were working my way forward yours would absolutely be my next question, along with figuring how to target the 20% of customers that drive 80% of my revenue.
Do you think that it would be better to have the revenue model fully thought out (through to revenue/user and lifetime value) before a/b testing the pricing models?
There really is no such thing as "fully thought out" here w.r.t what the customers will actually do with it. If you look again at the article, they went through this kind of progression:
- Best guess at a starting price model
- Revisited price model after poor test performance. Did research, revamped model based on research
- Revisited price model after poor test performance. Simplified plan to match the philosophy of business. Performance greatly improved
It's about testing the ideas, not keeping them in your head. What you think works and what actually works are often counterintuitively different.
I'm OK with optimizing this single step in the process. Things can change so much elsewhere that it's rarely all that prudent to perform a more complete analysis as you suggest, especially for a young, high growth service/company.
1. The pirate theme, and the fact that each blog post has it's own well-done image are freaking amazing. I also love the sand effect (on the homepage laptop too). Made me instantly feel good about your brand, and I went to the home page thinking, "Hey we already use Help Scout but maybe we should try this out", even though we are completely happy with Help Scout and I had no real good reason to even think of switching... you still got me.
2. All of that was completely undone by the two screen-invading modal windows with sleazy tricks like not having an obvious "get the fuck out of my way you annoying piece of shit" button. One on the blog itself (which pops up before I even start reading your post) and one on the homepage. I'm even surprised that ESC worked, but thank god it did. Thing is I actually really wanted to read your article, and send it out to the team, but now I feel like I don't really want to help a company that uses slimy tactics. I still will though because the title fits so perfectly with what we're working on.
3. Oh fuck me, while writing this comment another one popped up on a really delayed timer. Never mind, no longer reading.
I'd be a little careful here. I've personally dealt with the situation of creating a "customer-pleasing" revenue model, only to have to change it a year later because it was leaving way too much money on the table. It was a horrible process that resulted in losing a lot of customers. While we did eventually successfully make the transition, I think all of us wished we hadn't just said "oh if this takes off that will be a great problem to have". You don't want to A/B test your core business model unless you are very confident it is going to work long term.
Obviously, it is important to pick a business model that people will pay for, but it is also important to make sure that business model is going to support your business properly long-term.
I urge you to purchase and read The Strategy and Tactics of Pricing [0]. You are making a number of mistakes, not the least of which is to use low pricing as a differentiator. Bad idea. You've landed yourself on a potential slippery slope that can be devastating. I doubt your current competitors will follow you down. They are likely not suicidal. However, your position is easily attacked by anyone willing and able to take a dive for the weeds (in terms of pricing). Product being reasonably equal, the lower price wins. Customers nearly always want to pay as close to zero as possible.
I see a lot of "we had this revelation" type posts on HN. Perhaps this is a side effect of a younger, less experienced membership. A lot of these things are nowhere close to revelations. People --many people-- have trenched these paths before you. Anyone with a reasonable amount of business "street" experience could have told you that giving away EVERYTHING for a low price would convert better. What you need to do is optimize profit per unit and segment the market in order to maximize that function. The other thing you are doing is colloquially referred to as "leaving money on the table".
The pricing field is deep, wide and complex. The suggested book is one of many I studied over the years. For a number of years one of the most frustrating aspects of entrepreneurship for me was having to read piles of business books to understand what the hell I was supposed to do. Engineering school does not prepare you for this.
I've been through this same exact price confusion with customers. They told us they wanted to pay a la carte, but when we tested it they very much did not.
I think the psychology makes sense. When they told us they wanted to pay on a pay per use basis they based that completely on the past. They did the basic math. Last month I did 16 tickets. At $.25 a piece I would have paid $4! That's what I want to pay!
When faced with a future purchase decision, however, the mindset immediately changed. Now the decision was unbound. What if in three months I get to the point I have 1,000 tickets?!??! My god, now it's $250/month. That's crazy expensive! That fear caused a lot of issues.
So we tried the logical next step. $.25 per use, capped at a certain dollar amount ($25/month). That was too complicated to understand.
So we just started charging a flat $25/month (sales-leads fwiw). That worked really well.
As he said in the post, what a customer asks for and what a customer actually wants can be very different (and by wants he means what they will actually act on). Apparently they didn't act on paying per ticket but they DID act on a single no hassle price.
I started something around the same as Groove and I was told that our users only want to pay per ticket. After doing an AB test, it was clear that a small amount of people actually wanted this option but they were quite vocal about it.
My guess is that is what gives.
To add to this, even when we had implemented this for users, they were our worst customer by far.
Sometimes you have to read between the lines as well.... how many of the customers wanting to pay per ticket had signed up as a direct result of the first promotion which offered the ability to pay per ticket starting at $1?
Plus a chart that shows you significantly cheaper than three competitors is always a good incentive for some people to get the credit card out.
I would really like to see a comparison between this flat fee per "unit" (in this case users) model vs tiered pricing based on the number of these same "units", where the "unit" is one that will be predefined and remain mostly static.
So in this case, something like "$15 per user" vs "$39 for <= 3 users, $99 for <= 10 users, $249 for <= 30 users".
Cool writeup. I don't know much about this market, but the Competitor Pricing Comparison Chart shows them undercutting the others on price so much that I'm wondering how much money they're leaving on the table. Hopefully as time goes on they'll have enough differentiation that they don't feel the need to compete on price any more.
I'm sorry if I am not seeing this clearly, but did you change the pricing model? All I am seeing between Test #2 and Test #3 (the succesful one) is that you drop from $150 to $15. The $150 was the Unlimited Plan, and now you have all that for $15.
All I see is price lowering instead a diff price model. I apologize if I am wrong.
Have you considered freemium (or close to freemium, like Zendesk)? That would be interesting. My sense is there are three audiences: those who want to pay nothing or close to nothing, those who will pay a little and those who will pay a lot.
And this is based primarily on their financial situation, not needs or aspirations.
I've been following these posts and they've been really helpful, so thanks for writing them up. But when A/B testing your pricing model, what are you supposed to do for the customers that signed up for the now-defunct service prices?
I just want to say thank you to Alex Turnbull for writing about all the things they've learned at Groove. It's incredibly helpful and insightful to read.
Later on the average lifetime value of a user in each model is needed (plus the cost to get and convert them).
Yes, conversion is part of the equation, but this feels like a vanity metric on its own.
Just so I'm not sounding pedantic, our SaaS tool has a conversion rate of over 40% because it's completely free, permanently. That's the boundary case of moving the pricing.