TV5MONDE Plus

SCOPE
Web Desktop & Mobile

ROLE
UX / UI Designer

PROJECT LENGTH
1 year and half

The Challenge

The TV5 MONDE Plus free international streaming interface was designed during the Covid-19 pandemic in a very short space of time. The team wished to audit their web interface, and optimizations were recommended via the ABtest method. 

Challenging ideas through ABtest and the alliance of parties was a lever for agile, user-centric conversion. My ability to form a trio with AB Tasty and TV5MondePlus gave us real agility in managing the various AB test projects. 

The combination of the tool’s skills, our digital and user-centric expertise, and our knowledge of the customer provided a broad and detailed vision of the scope, objectives and constraints.

Collaboration with :
  • Head of Digital
  • Vice President, Digital Operations
  • Webmaster
  • AB Tasty Support Team
  • Front-end dev
  • Project manager

Definition

A/B testing consists of putting two versions of the same web page online to determine which version is more effective.

This technique is mainly used to optimize a website’s pages for both desktop and mobile devices.

In concrete terms, we test an « A version » of the medium (which is the original version) against a « B version » (the version that involves a variation in content or form).

Methodology

Questions & opinions

Prioritize

Hypothèsis

Test

Learn & Build

- Framing workshop
- Interviews
- Data analysis

- Listing of questions and hypotheses
- Prioritization workshop
- Own formulation, categorization and evaluation

- Creation of a roadmap and back log

- Prepare, parameterize and test
- Quality control

- Collect and analyze
- Recommend and improve the site
- Formulate a new hypothesis

Use Case n°1

Test hypothesis

Redesigning the View, Details and Favorites CTAs on the program page will make the content more attractive and visible to all users, and increase the click-through rate to a video.

Bouton-version-original
site-TV5-monde-plus

UI design

The View button is available in only one design.

Original version

Variant 1

Variant 2

Test settings

The primary objective was positioned on the View button, which is directly linked to the client’s conversion KPI: « player page access rate ».

The secondary objectives were the so-called secondary buttons: Details and Add to favorites.

Click trackers have therefore been installed on all 3 buttons, including the original version, for comparative purposes.

The target is broad: the test targets all visitors regardless of their profile (new or old visitor) and country.

However, the targeted URLS are in the French language. Other languages are not included in the test.

As this is the first test of this hypothesis, we’re keeping the original version to provide a point of comparison with the version users are used to using.

So, traffic is divided into 3: 33% by variation and 1% non-tracked.

The last stage is the recipe, where all variations, including the original version, are tested to ensure that the trackers function correctly (i.e. that when a user clicks, a hit is sent and counted in the reporting).

Traffic modulation

Analysis

1. Main Objective : 'View' button

Campagne fiable

bouton-voir-analyse

Winning variation

Conversion rate

x2

Recommandations

In view of the outperformance of the Voir button variation, we recommend using this new design to achieve the KPI objective: Increased click rate to access a video. As a first step, a patch can be put in place in target countries (French-speaking) in order to put this into production and benefit from the gains more quickly. Further development will depend on schedules and availability.

2. Secondary buttons : 'Details' and 'Favorites' buttons

Campagne non fiable à 100%

Detail button results
Resultat-bouton-favoris
Favorite button results

Lack of reliability

This test did not reach the reliability index after more than 2 months online.

The results are therefore not tangible enough to be analyzed.

Recommandations

We can draw some interesting conclusions, despite of the lack of reliability

Detail button :

We’re still 1,042 conversions (clicks) short of reliability, as this button gets very few clicks, which raises the question of its usefulness for users at this stage of the journey.
It is recommended to rework the interest of the Details button as presented in the test.
A redesign project has already been launched on this page.

Favorite button :

We’re 1056 conversions (clicks) short of reliability, and this button is also very little clicked. It is recommended to rework the Favorites button and perhaps more broadly the add-to-favorites decision path in order to understand how users use it and how to improve it in order to achieve the KPI: Increase in the number of logins / account creations.

Use Case n°2

Test hypothesis

Displaying a "Connexion / Create an account" notification menu will alert and create a trigger for both logged-in and logged-out visitors, and increase the click-through rate to video content.

UI design

Original version

The original version has no notification center (no icon is displayed in the header).

Header-original

Variant 1

Feature

Variation

Icons

Bell-icon-inactive

Belt Icon inactive

Bell-icon-active

Belt Icon active

Test settings

The main objective was positioned on the Bell icon.

The secondary objectives concerned content: the 3 notifications.
The first is to encourage users to log in or create an account.
The other two redirect to program pages.

Click trackers were installed on all these elements. As the original test version had no notifications (pure creation), we had to find trackers equivalent to the actions proposed in the variation.

The target is broad: the test is aimed at all visitors on desktop and mobile. The test adapts according to the visitor’s status (connected or not connected) so as not to present a connection notification to a visitor who is already connected.

The test is aimed at international French-speaking users. It was therefore launched in 3 separate groups of countries:

  • Europe
  • Maghreb
  • Afric

 

However, the targeted URLS are in French. Other languages are not included in the test.

As this is the first test of this hypothesis, we’re keeping the original version to provide a point of comparison with the version users are used to using.

Traffic is therefore divided in 2: 50% for the original and 50% for the variation.

The last stage is the recipe stage, where all variants, including the original version, are tested to ensure that the trackers work correctly and that the display is correct, especially for logged-in visitors.

Traffic modulation

Analysis

  • France
  • Belgique
  • Suisse
  • Monaco
  • Maroc
  • Tunisie
  • Algérie
  • Burkina Faso
  • Mali
  • Côte d’Ivoire
  • Sénégal
  • Togo
  • Gabon
  • Congo-brazzaville
  • Niger
  • Tchad
  • Guinée
  • Congo-Kinshasa
  • Madagascar

1. Main Objective : Bell icon

Campagne fiable

EUROPE

1 656

Conversion uniques
sur 27 204 visiteurs

MAGHREB

3 848

Conversion uniques
sur 21 581 visiteurs

AFRIQUE

1 125

Conversion uniques
sur 9 022 visiteurs

Conclusion

The Bell icon does have an attractive effect on users.

2. Secondary Objective : login notification or account creation

EUROPE

Conversion rate
0 %

VS

1,68%

Conversion rate (original)

MAGHREB

Conversion rate
0 %

VS

0,44%

Conversion rate (original)

AFRIQUE

Conversion rate
0 %

VS

0,39%

Conversion rate (original)

Conclusion

The notification center validated the test hypothesis: displaying notifications about the connection and some programs resulted in a higher click-through rate.

The login / create account notification performed well, with a 26% improvement in the conversion rate.

Recommandations

Short term:


It’s possible to use the AB test of the notification center as a means of displaying notifications while waiting for a code implementation.

Nevertheless, a digital communication strategy is required.

Medium to long term:


It is recommended to implement this notification center in code and to study user preferences in order to move towards a content personalization strategy.