Trigger-based emails Growth Hacking for Toy.ru results in 44% CTR growth
We have previously already analyzed the collaboration with Toy.ru, an online toy and goods store for children. The last time we described the success of our Growth Hackers in improving the key performance indicators of the webshop. Today we share the results of another sales channel, namely the triggered-based emails.
As usual, the work was carried out using A/B testing mechanics, in which all recipients of emails are randomly divided into two segments. Segment A sends the original version of the email, whereas segment B sends an email with different several hypothesis, which, in theory, should improve the effectiveness of the emails.
Case 1. Optimal time for sending trigger-based messages
The time for sending emails is one of the eternal questions that email marketers are challenged with. However, they need to remind the user in time about the abandoned goods in the cart or help him with the choice, warm up the interest of the user and motivate them to purchase. Furthermore, it is important for the online store to not to be intrusive and to not to bother users with obsessive advertising. Therefore, one of the most common hypotheses in testing email campaigns is testing the delay time to send emails.
Hypothesis:
Reducing delay time to 20 minutes will not only positively affect the Open Rate without affecting other key indicators, but will, at the same time, increase motivation to buy and move users to other actions related to the email.
“The abandoned cart ” Scenario
| Segment A (original delay time): |
| 60 minutes |
| Segment B (alternative delay time): |
| 20 minutes |
Results:
| Open Rate (%) | |
| Segment A | 24.6 |
| Segment B | 27.6 |
| Δ |
+ 12.2% |
Statistical significance: 99.55%.
“Abandoned Browsing” Scenario
| Segment A (original delay time): |
| 60 minutes |
| Segment B (alternative delay time): |
| 20 minutes |
Results:
| Click-Through Rate (%) | |
| Segment A | 30.5 |
| Segment B | 32.9 |
| Δ |
+ 7.93% |
Statistical significance: 97.8%.
Case 2. Removing menu with product categories from email header: focus on products
When developing the design of e-mails, it is important to think about how users perceive each element of the email individually and all of them together; what helps to lead a subscriber, and what distracts his attention. When the template is overloaded with details, conversion will not be enough, and the probability that the user will follow the link will significantly decrease. It is then important to redirect the user’s attention to buttons and links that are most likely to lead to conversion.
Therefore, we decided to test the removal of links leading to information about other sections of the site in scenarios where you only need to slightly push the user to complete the order, for example, the “abandoned cart”.
Hypothesis:
The menu in the header of the “abandoned cart ” email cannibalizes a portion of the clicks near the block of products from the cart and, at the same time, its elements lead to non-conversion links. Hence, the user’s attention is dissipated, and he is less likely to make a purchase.
| Segment A
(Initial abandoned cart template) |
Segment B (Alternative version of abandoned cart template) |
 |
|
Results:
More than 12,000 people took part in the testing, and the following data was obtained:
“Abandoned cart ” Scenario
| Conversion Rate (%) | |
| Segment A | 9.29 |
| Segment B | 13.1 |
| Δ |
+41.5% |
Statistical significance: 98.1%.
Case 3. Increase the number of products in the recommendations block
Personal recommendations are the recognized strength of Retail Rocket and in this case, the effectiveness of those are brought to the test.
Children like different types of toys, which means that the parents would most likely like the idea of having more choice within the e-mail, so we decided to increase the number of products in the recommendations block of the scenario “Product View” from 6 to 12 items.
Hypothesis:
The increase in the number of product cards in the email will increase the clickability.
|
Segment A (without any changes) |
Segment B (Alternative product recommendations block) |
 |
 |
Results:
“Abandoned Browsing” Scenario
| Click-through rate (%) | |
| Segment A | 27.7 |
| Segment B | 32.1 |
| Δ |
+15,7% |
Statistical significance: 99.7%.
Case 4. Introducing “Bought today” element in the product cards – The use of the social proof principle
The principle of social proof is a psychological phenomenon, which we often use in constructing hypotheses. It is no secret that people trust other people’s opinion, and the more positive reactions are for a product (e.g. online reviews), the more chance for a new visitor to purchase it. One way to use the principle of social proof is by adding “Bought today” to the email.
Hypothesis:
Adding the number of purchased goods today to the product card will increase user’s interest given the interest already shown by other people in these products. This will create a sense of relevance. The element was tested with the mailing lists for the “Abandoned Category” scenario.
|
Segment A (original product card layout) |
Segment B (alternative product card layout with embedded element “Bought today”) |
 |
 |
Results:
“Abandoned category” Scenario
The A/B test took more than 140.000 people into account, and the following data was received:
| Click-through rate (%) | |
| Segment A | 25.0 |
| Segment B | 28.7 |
Δ |
+ 14.6% |
Statistical significance: 98.3%.
Case 5. Introducing the CTA-element “Show more” in the email template – the gestaltism principle
When constructing hypotheses, our Growth Hackers use different branches of psychology. In this case, the principle of gestaltism (i.e. the visual patterns and stereotypes) was used.
By adding the “Show more” element to the email, when some product cards are hidden in the email template, can be perceived as an extension of the product recommendations block and can fulfil the incomplete effect. Thus, the user expects that when clicking on the “Show more” link, the number of cards displayed will increase, activating a personalized stereotype.
Hypothesis:
The “Show more” element is added after the product recommendation block with the same design of the rest of product cards. According to the postulates of Gestalt psychology, the user will interpret the incompleteness of the image and will interact with the website.
|
Segment A (Original product recommendations block) |
Segment B (Alternative product recommendations block with embedded element “Show more”) |
 |
 |
Results:
“The abandoned cart ” Scenario
| Click-through rate (%) | |
| Segment A | 30.7 |
| Segment B | 35.6 |
| Δ |
+15.8% |
Statistical significance: 96.7%.
“Abandoned Browsing” Scenario
| Click-through rate (%) | |
| Segment A | 19.0 |
| Segment B | 23.4 |
| Δ |
+ 23.3% |
Statistical significance: 99.6%.
“Abandoned category” Scenario
| Click-through rate (%) | |
| Segment A | 15.9 |
| Segment B | 23.0 |
| Δ |
+ 44.0% |
Statistical significance: 99.9%.
Toy.ru comment:
“Thanks to Retail Rocket for a solid and fruitful cooperation in improving the quality of the work with triggered-based emails and newsletters. Even minor changes in A/B testing were significantly increasing our conversion. Thanks to this, our only shop is receiving additional profit from the same volume of trigger-based emails sent.
However, good work does not stop, and there are still many stages for increasing conversion and sales coming from triggered-based emails.”
Alexey Evseev, Deputy Head of Internet Projects