A/B Testing Mistakes to Avoid in Your Data Science Interview: Tips and Tricks!

Подписаться 55 тыс.

Просмотров 29 тыс.

50% 1

In this video, I'm going to talk about a few mistakes that people often make when interpreting A/B testing results. You may often encounter these questions during an interview for data scientist positions. Want to know what these mistakes are and how to solve these questions correctly? Stay tuned!
Choosing the Right Metrics for A/B Testing
• A/B Testing Metrics: W...
A/B Testing "Real-Life" Example: A Step by Step Walkthrough
• A/B Testing Made Easy:...
Data Scientists Must Know: A/B Testing Fundamentals
• A/B Testing Fundamenta...
🟢Get all my free data science interview resources
www.emmading.com/resources
🟡 Product Case Interview Cheatsheet www.emmading.com/product-case...
🟠 Statistics Interview Cheatsheet www.emmading.com/statistics-i...
🟣 Behavioral Interview Cheatsheet www.emmading.com/behavioral-i...
🔵 Data Science Resume Checklist www.emmading.com/data-science...
✅ We work with Experienced Data Scientists to help them land their next dream jobs. Apply now: www.emmading.com/coaching
// Comment
Got any questions? Something to add?
Write a comment below to chat.
// Let's connect on LinkedIn:
/ emmading001
====================
Contents of this video:
====================
0:00 Overview
1:26 Data Scientists' Roles
1:48 Data Peeking
3:40 Multiple Testing Problem
8:49 Lack of Statistical Power

Опубликовано:

11 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 19

@Max-yv8lw 2 года назад

Hi Emma! I really like your videos. They are very insightful and helped me a lot when I was preparing my DS interviews:)

@alexandragrishechko6618 2 года назад

Your videos are super helpful, thank you!

@thegreatlazydazz 3 года назад

sublime video, emma. I get the feeling you will explain every useful thing in the Hippo book, thus saving us all the bother of reading it.

@FariborzGhavamian 3 года назад

Your videos are gold! Thanks!!

@Gavindsa 3 месяца назад

Emma your videos are amazing! Keep doing what you do. Looking forward to more such content

@kylechen4774 3 года назад

thx for sharing Emma, i found these very helpful and actually thats what i encountered in real life work. mkt ran some wrong a/b testing without actually understanding it then asked me to analyze the result. maybe i should share your channel to them lol

@snowguo1786 2 года назад

Thank you! this is very helpful! Im done with this video. All the content are noted!

@janeli2487 3 года назад

Hi Emma, thanks for keeping producing such high quality videos. Do you mind explain how to deal with multiple test problems in other scenarios that you mentioned, much as sliced data into multiple segments?

@PikaChu-th7nb 3 года назад

Great video. Very insightful. I would be interested to hear your thoughts on A/B tests set up to ensure that changes do not break anything. For example if an e-commerce website begins to place ads on the website, they want to make sure that adding ads onto the website does not cause people to buy less. What would be a good way to think about tests like this?

@saishastech8023 3 года назад

Good explanation 👌 keep going 😊

@arunvaibhav5059 3 года назад

You're really great!!! Please upload Facebook Data Scientist interview experience. Thanks!

@xueyingding697 2 года назад

Thanks for your videos! For using tiered sig. Lvls to check multiple metrics, how can we calculate the sample size? Should we calculate based on the most important metrics or should we calculate for each metric and choose the largest needed sample size?

@Ayy12366 2 года назад

Hi emma, this is very helpful! Thank you for making those videos, just a quick follow-up questions, I think testing multiple metrics in an A/B test is common; like usually we will pick one metric as the main metric and the couple other will just serve as support metrics; so if I just make whether or launch decision based on the significance of this one key metrics, it's fine, right?

@Ayy12366 2 года назад

The reason why I ask this is sometimes we got like tradeoff type of questions: you key metrics goes as expected but one supporting is going conflicts, will you launch, then we talk about short term and long term benefits something like that

@zhuyanshu8941 2 года назад

Can you clarify: multi hypothesis problem arise by testing a segment? Control vs. only Web segment? Or multiple treatment group such as :Control vs. Web vs. IOS?

@user-uc3cn1jj4n 2 года назад

Cannot find the material on p119 in the hippo book, the whole chapter talks about ethics.

@jasdeepsinghgrover2470 2 месяца назад

Good explanation but I think the last error is incorrectly handled.... Imagine you run an experiment and it is significant (you haven't checked the observed power yet), if you accept it then it is wrong but if you rerun it you just nearly doubled the p value. We should be only looking at the rerun or let the experiment have significant power (probably more than our threshold)