I recently failed google ds interview in AB Testing ; Feels really bad + next attempt would be after a year which makes me even sad. So yeah if sitting for FAANG always look into AB testing and statistics
Here is a simple version. You do not necessarily need a subquery or a temp table to solve this question: SELECT w.url, COUNT(a.event_id) AS click_count FROM google_search_activity a JOIN google_search_websites w ON a.website_id = w.website_id WHERE a.event_type = 'clicked' AND a.creation_dt BETWEEN '2022-01-01' AND '2022-06-01' GROUP BY w.url ORDER BY click_count DESC;
Here is a simple version. You do not necessarily need a subquery or a temp table to solve this question. SELECT w.url, COUNT(a.event_id) AS click_count FROM google_search_activity a JOIN google_search_websites w ON a.website_id = w.website_id WHERE a.event_type = 'clicked' AND a.creation_dt BETWEEN '2022-01-01' AND '2022-06-01' GROUP BY w.url ORDER BY click_count DESC;
Hey Dan! This is exactly what I wanted. Not just the template but going in details of each step and explaining why we chose something vs the other. This is amazing! I am going to subscribe to practice more of these.
The variance you've computed here is from Bernoulli and I would like to know how it follows the normal as n increases. isn't it the binomial distribution which does so?
for the second question, why can't i use: SELECT DISTINCT email, prime_joined_dt FROM amazon_users JOIN amazon_transactions USING (user_id) JOIN amazon_products USING (product_id) WHERE category IN ('clothing', 'jewelry') AND prime_member = 1 ORDER BY prime_joined_dt ASC; am I missing something?
Can the alternative hypothesis not be revenue being higher in the target group? And do a one tailed test? It doesn't make sense to change the algorithm for a business to make the same amount of money
airbnb: the problem its trying to resolve is two sides, for the hosts side, it allows hosts to make money from places that they are not living at, from the customers side, it allows customers to find cheap housings; airbnb targets the kind of users that have such needs, the onboarding process is the signup process for the 2 sides; user journey: customer side: they search for an accomodation, they land on the main site, they input the dates and destinations/other filters, they are taken to the lodging page, they can scroll, read, and go to multiple hosts sites, then they can check the reviews, location, description etc, the site rewards users with loyalty, it retains users through email notifications/voucher campaigns etc, it grows users through marketing events, referral programs, promo offers etc
ChatGPT can answer all the question related to code. So I don't think interviewer should ask those questions anymore. They would focus on how we solve the problems by ideas
I think the candidate was nervous and spike before thinking many times in the product case. Maybe if she had a couple minutes to compose her thoughts, she coulda done better.
Both the interviewer and candidate were pretty bad here. Candidate was low energy, and the interviewer didn't really dive deeper... just asked surface level questions and moved on
@datainterview Wouldn't the click through rate or number or products purchased per user per day be a more effective success metric? If it is revenue per user per day, what if the control group purchases only one high priced product whereas treatment group purchases multiple low prices products, the recommendation system works but the revenue would be higher for the control group
1. Looking at the past history of the same airline, the preliminary cause of the flight delay might be operational issues. 2. Geography 3. Weather 4. No of runways - if they are too less then if one flight gets delayed then the chances of others getting delayed increases
Good video, but I think you forgot to ask the most important question: how much money the business can save with your model and whether it is worth implementing it in the first place. In your case, you want to create a model that would predict delays in real time and for passengers that are already in the airport and their departure should be in the next couple of hours. But in this case, either your prediction won't be important at all (if it is 10 min or 1 hour delay), since passengers can't do anything with this information, or your delay prediction is so large (5+ hours) that users will ask for extra compensation (since you didn't inform them earlier) and complain why didn't you inform them a day before that the plane is delayed by 5 hours. In your case, you assumed it is for people who are already at the airport, but I would argue that the actual end users are people who's flight is in a day or so. Since if the company would inform users a day before that their flight is 5 hours delayed, users can make adjustments and might not complain so much and request for a refund, compare to when you predict that the flight will be 5 hours delayed when the user is already on the airport. It's more important to predict whether a flight will be delayed 12+ hours before departure, so that the passenger would decide for themselves when to leave home and whether to make other plan changes, rather than a couple of hours before the departure, when most probably everyone is at the airport, and the delay can be easily calculated using a simple math. And when people are already on the airport, it's already too late to tell them that their flight is 24+ hours delayed. In this case you don't need to have a real time prediction. Next point, I don't agree with 95% cutoff. Yes, there might be some bugs, but I would not cutoff like that, since those flight delays might be actually the most important ones (which have 24+ hours delay)
Question on metric tested in the hypothesis test - avg. # of clicks per user. Why wouldn't it be avg revenue per user per day? I believe the user could click the "buy now" button and still exit, or they may buy but spend less money with the new button,etc....
Excellent video. Should the candidate list the guardrail metrics way before step 7 (i.e., step 1) so those are known & agreed upon before the test starts ?
This is a neat solution but can be made 5x more efficient if you map the sum of the two rolls to an outcome on the seven sided die: For e.g Sum Number on 7 sided die (2,5) -> 1 (3,4)->. 2 (6)->. 3 and so on (you will need to remove one of the 6 cases which give 7 though). This way, we will be using 35 out of 36 possible outcomes instead of just 7
What if we have same user making multiple purchases across the day? Does it make sense to have metric as revenue-per-session / revenue-per-search if we have multiple purchases by same user in a day, and metric is revenue per user per day, wont we have to change the randomization unit?
Hey Dan, can you talk about the decision to pick revenue-per-user-per-day? Doesn't using this metric require the delta method or bootstrapping if randomizing by user? Thanks!
