TV5MONDE Plus
SCOPE
Web Desktop & Mobile
ROLE
UX / UI Designer
PROJECT LENGTH
1 year and half
The Challenge
The TV5 MONDE Plus free international streaming interface was designed during the Covid-19 pandemic in a very short space of time. The team wished to audit their web interface, and optimizations were recommended via the ABtest method.
Challenging ideas through ABtest and the alliance of parties was a lever for agile, user-centric conversion. My ability to form a trio with AB Tasty and TV5MondePlus gave us real agility in managing the various AB test projects.
The combination of the tool’s skills, our digital and user-centric expertise, and our knowledge of the customer provided a broad and detailed vision of the scope, objectives and constraints.
Collaboration with :
- Head of Digital
- Vice President, Digital Operations
- Webmaster
- AB Tasty Support Team
- Front-end dev
- Project manager
Definition
A/B testing consists of putting two versions of the same web page online to determine which version is more effective.
This technique is mainly used to optimize a website’s pages for both desktop and mobile devices.
In concrete terms, we test an « A version » of the medium (which is the original version) against a « B version » (the version that involves a variation in content or form).
Methodology
Questions & opinions
Prioritize
Hypothèsis
Test
Learn & Build
- Framing workshop
- Interviews
- Data analysis
- Listing of questions and hypotheses
- Prioritization workshop
- Own formulation, categorization and evaluation
- Creation of a roadmap and back log
- Prepare, parameterize and test
- Quality control
- Collect and analyze
- Recommend and improve the site
- Formulate a new hypothesis
Use Case n°1
Test hypothesis
Redesigning the View, Details and Favorites CTAs on the program page will make the content more attractive and visible to all users, and increase the click-through rate to a video.
UI design
The View button is available in only one design.
Original version
Variant 1
Variant 2
Test settings
The primary objective was positioned on the View button, which is directly linked to the client’s conversion KPI: « player page access rate ».
The secondary objectives were the so-called secondary buttons: Details and Add to favorites.
Click trackers have therefore been installed on all 3 buttons, including the original version, for comparative purposes.
The target is broad: the test targets all visitors regardless of their profile (new or old visitor) and country.
However, the targeted URLS are in the French language. Other languages are not included in the test.
As this is the first test of this hypothesis, we’re keeping the original version to provide a point of comparison with the version users are used to using.
So, traffic is divided into 3: 33% by variation and 1% non-tracked.
The last stage is the recipe, where all variations, including the original version, are tested to ensure that the trackers function correctly (i.e. that when a user clicks, a hit is sent and counted in the reporting).
Analysis
1. Main Objective : 'View' button
Campagne fiable
Winning variation
Conversion rate
x2
Recommandations
In view of the outperformance of the Voir button variation, we recommend using this new design to achieve the KPI objective: Increased click rate to access a video. As a first step, a patch can be put in place in target countries (French-speaking) in order to put this into production and benefit from the gains more quickly. Further development will depend on schedules and availability.
2. Secondary buttons : 'Details' and 'Favorites' buttons
Campagne non fiable à 100%
Lack of reliability
This test did not reach the reliability index after more than 2 months online.
The results are therefore not tangible enough to be analyzed.
Recommandations
We can draw some interesting conclusions, despite of the lack of reliability
Detail button :
We’re still 1,042 conversions (clicks) short of reliability, as this button gets very few clicks, which raises the question of its usefulness for users at this stage of the journey.
It is recommended to rework the interest of the Details button as presented in the test.
A redesign project has already been launched on this page.
Favorite button :
We’re 1056 conversions (clicks) short of reliability, and this button is also very little clicked. It is recommended to rework the Favorites button and perhaps more broadly the add-to-favorites decision path in order to understand how users use it and how to improve it in order to achieve the KPI: Increase in the number of logins / account creations.
Use Case n°2
Test hypothesis
Displaying a "Connexion / Create an account" notification menu will alert and create a trigger for both logged-in and logged-out visitors, and increase the click-through rate to video content.
UI design
Original version
The original version has no notification center (no icon is displayed in the header).
Variant 1
Feature
Icons
Belt Icon inactive
Belt Icon active
Test settings
The main objective was positioned on the Bell icon.
The secondary objectives concerned content: the 3 notifications.
The first is to encourage users to log in or create an account.
The other two redirect to program pages.
Click trackers were installed on all these elements. As the original test version had no notifications (pure creation), we had to find trackers equivalent to the actions proposed in the variation.
The target is broad: the test is aimed at all visitors on desktop and mobile. The test adapts according to the visitor’s status (connected or not connected) so as not to present a connection notification to a visitor who is already connected.
The test is aimed at international French-speaking users. It was therefore launched in 3 separate groups of countries:
- Europe
- Maghreb
- Afric
However, the targeted URLS are in French. Other languages are not included in the test.
As this is the first test of this hypothesis, we’re keeping the original version to provide a point of comparison with the version users are used to using.
Traffic is therefore divided in 2: 50% for the original and 50% for the variation.
The last stage is the recipe stage, where all variants, including the original version, are tested to ensure that the trackers work correctly and that the display is correct, especially for logged-in visitors.
Analysis
- France
- Belgique
- Suisse
- Monaco
- Maroc
- Tunisie
- Algérie
- Burkina Faso
- Mali
- Côte d’Ivoire
- Sénégal
- Togo
- Gabon
- Congo-brazzaville
- Niger
- Tchad
- Guinée
- Congo-Kinshasa
- Madagascar
1. Main Objective : Bell icon
Campagne fiable
EUROPE
1 656
Conversion uniques
sur 27 204 visiteurs
MAGHREB
3 848
Conversion uniques
sur 21 581 visiteurs
AFRIQUE
1 125
Conversion uniques
sur 9 022 visiteurs
Conclusion
The Bell icon does have an attractive effect on users.
2. Secondary Objective : login notification or account creation
EUROPE
VS
1,68%
Conversion rate (original)
MAGHREB
VS
0,44%
Conversion rate (original)
AFRIQUE
VS
0,39%
Conversion rate (original)
Conclusion
The notification center validated the test hypothesis: displaying notifications about the connection and some programs resulted in a higher click-through rate.
The login / create account notification performed well, with a 26% improvement in the conversion rate.
Recommandations
Short term:
It’s possible to use the AB test of the notification center as a means of displaying notifications while waiting for a code implementation.
Nevertheless, a digital communication strategy is required.
Medium to long term:
It is recommended to implement this notification center in code and to study user preferences in order to move towards a content personalization strategy.