A startup turns ideas into products, measures how customers respond, and then decides whether to pivot or persevere. All successful startup processes should be geared towards accelerating that feedback loop. In this post, we will go through how to experiment and learn in a WordPress Business.

• What should be monitored after an MVP is released?
• What factors can give us the knowledge we need to decide if we should persevere or pivot?
• And, what practices should we employ in order to acquire that knowledge?

All these questions can be addressed during the second phase of the process called “Measure”.

2- Measure

Audit the Performance Closely

Obviously, your newborn WordPress business has just been launched and you should closely monitor its performance to answer these pertinent questions:

• How many leads have you collected?
• How many have converted to become a paying customer and placed an order?
• Is your conversion rate close to what you were expecting?

Monitor User activity

Use data analytics tools such as Google Analytics to understand where your users are coming from, what they do in your website, which pages they visit, and what they do before converting. Here are some crucial steps to help you achieve this:

• Check behaviour flow

• Check top conversion paths

• Check top sources.
  

  

   

• Check top landing pages.

•  Check top Google keywords.
  Remember: Integrate Google Analytics with Google Search Console to find out which keywords are sending traffic to your website.

Run Interviews

Interview your customers. I’m not talking about feedback pop-ups at the end of a project or after a purchase. Talk to them. Send personal emails from your CEO right to their inbox! Invite them to a one-on-one interview with a C-level executive or the founder of your business. This will  allow them to share their valuable thoughts and insights with you as well ass make them feel important.

Talk to them about their experiences. What do they like about your product, and what improvements do they think will make it even better? Remember, people in our era are lazy (you and me included!) and benefit-oriented. Nobody wants to spend his or her own resources for someone else without anything in exchange.

Maybe some of your users are honest and interested enough in your product that they will be up for sharing their feedback in a detailed interview. But, majority of them will most likely ignore your invitation. So, use incentives to get them interested or encourage them to allot an hour for you. Perhaps you can offer an Amazon gift card, a coupon on their next purchase, or something of tangible and sensible value to them.

Run the Wizard of Oz test

In the Wizard of Oz test, customers believe they are autonomously interacting with the actual product. But behind the scenes, an unseen human being is doing the work or operating the product. If your WordPress product is an app or actual software where users login and interact with a group of features, you can use this method.

This may need some extra development resources to create the environment that will enable you to see how your users are working with the product, but it gives you the most immediate information about how your customers use your product.

Run Experiments

There might be dozens of ways for you to prove that your hypothesis is right, but using the wrong method may result to the failure of your project. There may be the perfect strategy out there to help you achieve your goal that you haven’t thought about just yet. But, it’s out there for sure! When it comes to experimenting and learning in a WordPress business, you need to think about different ways your product can make sense to your users.

With the knowledge you acquired over the past steps, you can draw a behavioural pattern of your users. Segment your users into groups —  those who hate it, those who are neutral about it, and those who just love it. You can also create segments based on the channel or source, such as organic search, paid search, Instagram referrals, and so forth.

Give each ‘cohort’ a name and list what each cohort likes and hates in a table. Now, ideate experiments for each of the cohorts in the table. For example:

Here is the findings:

• Feature X seems to be popular among many cohorts, which means majority of your users like it. You can consider keeping it in your product.

• Feature Y seems to be hated by most of the cohorts. Consider ditching it.

• Feature Z is liked in organic referrals cohort. Maybe you can consider testing it for paid referrals.

• Landing page 1 is not effective because it makes people hate you!

• Landing page 3 appears to be the most effective. Investigate and learn what might be the cause.

• Serve feature S only for fans and see the outcome.

• Landing page 3 is liked among your fans and one other cohort, so you can review and find what’s special about it. Amplify that in other landing pages as well.

• For organic search referrals cohort: the conversions take 30 days to happen and still in a very low rate, which means you don’t appeal to that cohort. Check what keywords you’re being found with on Google and try to come up with a strategy based on those keywords.
  For example, if you’re found more with the keywords “WP theme migration” and “wordpress speed optimization”, think about what is going wrong with these services in your team.
  Maybe the value you offer comparing to the price is too low? Or maybe, you need to guarantee a seamless and data-safe migration in your landing page? Think and explore various possibilities.

• For paid search referrals: think about what feature Z is about. That was the only thing they liked about your product! Is it about customizability? A design functionality? Or… Try to make a similar feature and let them know. They might be up for the product with that feature included.

• Both Instagram referrals and passives were served the same landing page, but one cohort didn’t convert and one did. Investigate what could make the passives group more interested in the product. Review the personas in the cohort and learn about their preferences. Maybe another landing page aligned with those preferences will work better. Think and explore…

• Use programs like Auditor to get more insights on your Instagram cohort preferences and apply the same strategy for any other cohort aligning those preferences in your funnel.

• For passives, publish more blog posts and provide clarification about why you think this product is relevant for them. Send them exclusive discounts as well to encourage conversion.

• For haters, provide feature W with an extra discount and monitor their reaction to the offer.

What we just did above is split testing different versions of the product to different groups of users at the same time. By observing the changes in behaviour between the different groups, one can make inferences about the impact of the different variations of the product.

This culture of experimentation is not only for the paunch period of your product. That should be an undetachable part of your product lifecycle. You need to commit to this culture of experimenting and learning in a WordPress business to stay relevant.

For example, one day we would debut a new marketing message aimed at first-time customers. The next day we might change the way new customers were initiated into the product. Other days, we would add new features, fix bugs, roll out a new visual design, or try a new layout for our website. Every time, we told ourselves we were making the product better, but that subjective confidence was put to the acid test of real numbers. Day in and day out we were performing random trials. Each day was a new experiment. Each day’s customers were independent of those of the day before.
– Eric Ries (Lean Startup)

Why should a WordPress business run experiments?

Maybe you will ask why you should experiment and learn in a WordPress business. How does it apply to my WordPress project? If you’re the founder of a similar WordPress customization service we exemplified earlier, whatever we call a feature in this article can be translated into a ‘service’ in your product or the features you offer within a service.

For instance, you offer three customization services that can be put into testing for different cohorts you designate for your product. Or, you offer one ‘custom coding service’ but you’re not sure about the scope of the customization you should offer within each pricing tier. You can define three cohorts and run experiments as delineated above and gain inferences.

In addition, if you’re a WP software development company, your product is composed of a group of features. The experimentation will guide your through what features you should add/keep and what to get rid of. But, in a software company you need a solid workflow for experimentation/testing.

How can I experiment and learn in a WordPress business?

Google Optimize and Optimizely are good for testing in a browser level. For a deeper experimentation of the multiple cohorts of your users, you may need to customize your development environment by defining a development sandbox.

“A sandbox is a testing environment that isolates untested code changes and outright experimentation from the production environment or repository in the context of software development including Web development and revision control.”
–Wikipedia

(more on that here)

1- You need this sandbox to run split-test experiments that affect a particular part of your users (cohort).

2- Each experiment should apply to a specific cohort and for a specific period, usually a few weeks if you’re not running a highly disruptive experiment.

3- Once an experiment is done (or while in progress), your ‘actionable metrics’ should be listed. You should then list what needs to be done after these findings.

Ok. Now, what should be done with all these findings?

3- Learn

So, you have not collected all the information you need via your sandbox experimentation such as interviews, KPI auditing, customers surveys and more. Now, it’s time for you to sit and analyze them. If the findings of your experiment proves your hypothesis, you will persevere. But, if the results are disproving your hypothesis, you need to pivot towards another directions.

But, how?

Pivot

Negative results for an experiment means your hypothesis was not right. And, it means something in your product such as idea, design, or execution is wrong or missing. You may have added wrong features or have neglected an important feature. You may have even targeted the wrong persona.

Your marketing channel could have been wrong or you have set the wrong KPIs to watch. Now, you should define a new hypothesis. Design new experiment(s) for it and put them into testing. If the results are positive, your hypothesis was right. If not, you should design another set of experiments and test them until one of them is proven.

Companies that cannot bring themselves to pivot to a new direction on the basis of feedback from the marketplace can get stuck in the land of the living dead, neither growing enough nor dying, consuming resources and commitment from employees and other stakeholders but not moving ahead.

– Eric Ries

Persevere

If the result of an experiment was positive, it means your hypothesis was right and it means that your product idea, design, and execution is vindicated. You have given the right features, developed in the right way, and served them to your customers properly. But, this is not the end-story for your product.

When a hypothesis is proven, it shows you that you’re in the right direction. This means you should continue in that same path and enjoy the process! You should define new hypotheses to try new ways to expand and grow your customer base. In fact, there is no ending for the experimentation cycle. The more you experiment, the more you learn, and ultimately grow.

Also, note that an easy approval of a hypothesis shows there may be something wrong with your experiment. As Eric Ries puts it, the use of vanity metrics instead of actionable metrics for your experiments, an overly long experiment cycle time, the use of large batch sizes, an unclear hypothesis, and a weak experimental design can all lead to inaccurate and illusive results.

Final Word

Build, Measure, Learn.

This is how you can experiment and learn in a WordPress business. Sticking to this threefold model will assure your product growth and your business growth the right way.

Innovation is a completely unpredictable thing. Many ideas that appear obvious to you may get simply disproved if you experiment them. Do not underestimate the power of experimentation. The results can shock you some time.