How Netflix measures copy success (round #2)
“Hey, what’s your favorite movie or TV show on Netflix?”, asked a friend.
This question may be familiar to you 👀
Yes, a lot of people are into it; whether they find trending movies, popular movies, or other movie categories, they will mention Netflix.
Netflix was first founded in August of 1997 by two serial entrepreneurs Marc Randolph and Reed Hastings. The company started out in Scotts Valley, California, and has grown to become the world’s leading internet entertainment platform. When it first opened, Netflix was purely a website-based movie rental service, allowed people to rent DVDs online.
According to Statista, Netflix had over 182 million paid streaming subscribers worldwide in the first quarter of 2020. Hence, Netflix becomes the world’s leading streaming entertainment service in over 190 countries 🙌
On May 6th, 2020 I got a chance of attending the webinar entitled “How Netflix measures copy success (round #2)” hosted by Yuval Keshtcher, Founder of UX Writing Hub. The keynote speaker was Jen Schaefer, the Content Design Manager at Netflix. She shared how the Content Design team at Netflix gets customer insights into the words and experiences they create. Previously, there was another event of “How Netflix measures copy success (round #1)” but unfortunately I missed it ☹️
As long as I can remember, there were 750 participants attended (has been confirmed by Yuval)! It was a huge number of participants during the webinar session. No wonder a lot of people were interested. I guess we were very curious about how Netflix handles its content considering they have users worldwide.
On Jen’s slide, she states that Netflix uses language and information strategically to design experiences people love.
Jen told us that the content design was deeply embedded in Netflix verticals. This means that their content design becomes the basis of identity on Growth, Partnerships & Payments, TV & Kids, Mobile & Web, Personalization, India, Interactive, Studio, and Messaging.
You might notice India haha! Yeah, me too. I wondered at that time. But I assumed this might be related to the Netflix users in India. According to Jane, she found an interesting case of Hinglish as it’s the most surprising A/B Test!
Interesting, right? During A/B Test we can discover many findings ;)
In the Content Design team, they also have shared goals regarding the style, voice, globalization, and testing. The style guide is shared within the team in conjunction with these other reference tools:
- Product Voice Guide, for tone, voice and how they write on and off the service
- Netflix Terms Glossary, for the spelling of words specific to their service
- the AP Stylebook, for grammatical or punctuation questions
- Merriam-Webster dictionary to check spelling
During the team transition period at Netflix, they must rearrange the brand, established it as well as the new team. The 4 focus areas as mentioned on the slide:
- Cleaning up legacy strings
- Creating reusable content patterns
- Infusing some delight (sparingly)
- Beefing up user education
In my opinion, Netflix has successfully educated their users regarding their established brand. They really do a great job!
You just can see from their incredible number of active users 🙌
This is it! The interesting part of the A/B Test of the Netflix CTA button. If you’re an active user of Netflix you should know which one of these CTA buttons below won the A/B Test. You might notice it when stumbled upon their landing page 😏
[Button A] Join free for a month.
[Button B] Sign up now!
[Button C] Try 30 days free
[Button D] Take me to content heaven!
Bravo! Button C won the A/B Test. The majority of participants guessed it right ;)
The question is why button C?
Let’s take a look at each button and my assumption below.
[Button A] Join free for a month.
You might think it has the same meaning as button C. Yes, you’re right! But, take a look at the copy “Join”. What’s crossing your mind when you hear that word? Engagement/ Commitment. The majority of people psychologically fear to commit to something that they don’t know the value or benefit of a product or something before they experience it first. By nature, people tend to like discovering something, feeling the experience, and then making a decision.
[Button B] Sign up now!
“Oh, no! It’s a paid service and I don’t know what Netflix offers me. I’m afraid that what I paid is not equal to the service I get.” That’s the first impression of a new user who’s just coming to the Netflix landing page.
[Button C] Try 30 days free
“Sounds less committal! I feel secure about it.” The advantage of the copy of “Try 30 days free” increases the conversion rates. It’s clear that I have 30 days free of discovering the service without any commitment. I will know whether it’s worth the money.
[Button D] Take me to content heaven!
Hmm, this one sounds like a demo button. I don’t even know how long they will take me to their content (15 minutes, 30 minutes, …). No clarity!
Well, perhaps you have another assumption? ;)
Jen suggested that the creation of content structure was started by thinking about the cause, the effect, and the result! By doing this, we will have a thoughtful content.
“During the A/B Test, we can analyze the “what” and “how”, but the most interesting part is finding the “why”. Why people behave like that?”, added Jen. Yes, I totally agree! I also love this part as I’m curious to death 😄
According to Jen, the Content Design team use qualitative research, customer survey, and Netflix call center in order to find the “why”.
On the next slide, Jen told us that the team at Netflix shares hypotheses with:
- PMs for business and product insights
- Designers and CI for user insights
- Language managers for global insights
- Data science for metrics insights
- Engineers for production insights
Well next question, is there any pain points working on the scale large of copy testing?
Of course. Considering that Netflix is a large company with active users worldwide there must be pain points of copy testing. It’s not only 10 or 100, but that must also be more than that! Keep in mind the word “worldwide”.
So, the pain points of copy testing at Netflix are:
- Every test required engineering support
- Tests took days to weeks to deploy
- There was no uniform way to test across platforms
- There was no easy way to test localized copy
Solution?
Yes, they have! The solution is called “Project Shakespeare”.
It’s a system for making testing faster, easier, and more streamlined. Developed for the globalization use case. The good news is the team has embraced and there’s an internalization engineering.
You might wonder what kind of tools the Content Design team uses at Netflix?
Well, as long as I can remember Jen or Yuval mentioned Frontitude. On Frontitude landing page, it says that it can manage all your product text in one place — supercharged with advanced search, smart tagging, version control, and text-first collaboration. Pretty cool, huh? 😎
The Frontitude team just launched the plugin on Figma. You can try it!
The last slide is about the product cycle…
The product cycle at Netflix is started from qualitative research, then hypotheses, then test the hypotheses, iterate, so on..
I think it’s common that might among us the designer teams that work on products use the same approach, too.
Thank you to Yuval who hosted this webinar and Jane who shared the knowledge about “How the Content Design team at Netflix gets customer insights into the words and experiences they create”.
See you at the next webinar, hopefully! Thank you for reading 😊
If you have any questions, drop me a line at livinda.christy@gmail.com!
Check out my projects, maybe? Personal website | Dribbble | LinkedIn
