Hi Mikko, thanks a lot for the great video and detailed explanation! Maybe you can help me out on a question? I'm currently workink on a project involving DiD in Stata where I consider several covariates - which have pre- and post-treatment levels. So far I was not seperating the covariates via indexes for the two time periods, and I got the criticism that including post-treatment levels for the covariates into the regression would lead to endogeneity. Can I thus only control for pre-treatment levels of the covariates if I want to avoid such endogeneity in my DiD? Cheers, thanks in advance!
Depends a lot on what the variables are. I do not think that say generally that including covariates from the second period is a bad idea. The basic idea of DiD is to justify the parallel trends assumption by looking at past trends and you would thus not need any covariates. But if the parallel trends assumption cannot be justified and if the treatment and control differ systematically on the covariates, then controlling for the covariates would be appropriate. I personally would not frame such analysis as a DiD analysis any longer, though, but would present it as a regression model instead. But all this depends on the context.
Thank you for explaining the DiD intuition. I would like to ask what is the suitable approach when one knows in fact that time trends between control and treatment group are not parallels, are there DiD techniques designed for these situations?
DiD would not be the right technique in that case. What would be the right technique depends a lot on the context. It is also possible that you cannot estimate a causal effect of the treatment. For example, consider the following: You are testing a medication and a) let people choose between being in treatment or in control, and there are more sick people in the treatment group than in the control group. b) Some people will naturally recover from the diseases but this natural recovery rate is unknown and this causes the trends of health over time to be different between treatment and control (more sick people initially = more natural recovery over time). If you do not know the natural recovery rate, it is not possible to estimate the causal effect of the treatment. There are a number of strategies to address this scenario. If you have prior information on the natural recovery rates, you could implement that in your model. You could also try to use instrumental variables that correlate with the selection to treatment vs control but do not influence recovery. Or you could estimate the model as it is and then try to quantify the bias. Morgan and Winship (Counterfactuals and Causal Inference or something like that) discuss different causal analysis strategies.
I'm reading a paper using methods of time-varying DID. This method is quite new and I haven't found any explanation on youtube. Maybe you could consider to update a video aboout that~ (Just a polite small request which can be neglected if you don't want to) @@mronkko
@@GuaGua000 I have advanced DiD on my list of things to do. I might do it next fall when I might need it for a course. There are so many things that I could talk about and the priorities depend on what I need for in-person teaching or for a research paper.
Thanks for a great video. Can you recommend any papers that uses the empirical test you mention at 12:28, in regards to testing the possible violation of the parallel trend assumption.
i cannot come up with any specific examples. However, if you search for "parallel trends" "Difference-in-differences" in google scholar, you should find lots of examples.
Hi Mikko. Thanks for a great video. I was wondering if treatment needs to be as-if random in a diff-in-diffs? Or does the common trends assumption solve this?
It does not need to be "as-if random". If it was, we could just compare the two groups post treatment. But because the assignment is not random, there are pre-assignment differences. DiD assumes only parallel trends (which itself is a strong assumption).
Perfect video! Super helpful for the methodology of my thesis. Would this be the right approach to determine the impact of COVID on venture capital activity? 2018-2020 vs 2020-2022. Thanks again for the video
Thanks. I do not see this technique as immediately applicable because there are no clear treatment and control groups in the case of COVID. I would take a look at quasi-experimental designs.
Hello Mikko! I am currently writing my master thesis on the influence of ESG on stock returns during the pandemic crisis. I am using a dif in dif for studying the causality between ESG score (treatment) and the current pandemic (time dummy). Is ESG as a dummy (one if the company qualifies in the top quartile, ESG score is last measured in 2018) qualified? I am worrying about self-selection bias. Can DiD fixed effects be a solution? Thank you in advance!
If you think that future performance correlates with selection after controlling for current performance, then DiD will not solve that issue. I am not sure what ESG stands for, but if it is a continuous variable, I would treat it as such instead of creating a dummy. DiD is really for natural experiments where the variable of interest is a dichotomy. Without knowing the specifics of your study, my off the cuff comment would be: Just regress performance during pandemic on ESG score, controlling for past performance and other relevant controls. (See my video on lagged dependent variables)
Hej Mikko! Thank you for a very good and interesting video! I’m wondering should one include individual/time fixed effect into equation since (did) is automatically panel data? Or should one test it Alex. Haussman test?
You need to use cluster robust SEs. Time dummy is included in the design. Individual level dummies cannot be included because they would be perfectly collinear with the treatment assignment dummy. I assume this is what you meant by fixed effect. If you mean the concept more generally, you can add fixed effects of covariates and probably should do that too. (I.e. use control variables)
@@mronkko kiitos nopeasta vastauksesta! Vaihdan suomeksi, niin minun voi olla helpompi avata! Huomasin kun lisäsin individuaali kiinteät vaikutukset niin interaktiotermi (estimaatti) (post_toimenpide*koeryhmä) muuttui positiivisesta negatiiviseksi! Muodostuuko tässä ongelmaksi siis se, että tuo (individual kiinteät vaikutteet) korreloi suoraan koeryhmän kanssa, joka on osa tuota interaktiotermiä? Ja ymmärsinkö oikein että malliin tulisi lisätä kuitenkin vaikkapa sukupuoli jolla on mahdollisesti vaikutusta tuloon esim. Eli normaalisti kiinteät vaikutteet olisi varmaan hoitanut tuon, mutta nyt tuokin tulisi lisä kontrollimuuttujana (jos relevantti)?
@@Allu-oe6ih Siis jos lisäät jokaiselle yksilölle, joka on siis mitattu kahdesti, dummy-muttujat, niin malli ei ole identifioitu eikä sitä pitäisi pystyä estimoimaan regressiolla. Yleensä tilasto-ohjelma "ratkaisee" tämän ongelman heittämällä yhden dummyn pois, mutta tämän jälkeen koeryhmä indikattoria ei oikein voi enää tulkita koska sen tulkinta riippuisi siitä mikä dummy heitetään pois.
Hi Mikko, very interesting this video! I have a question that is very important for my thesis. Once I have found the average treatment effect, how I can obtain the individual treatment effect for each element in my treatment group? Thanks in advance!
Individual treatment effects cannot be estimated in DiD. Their estimation is in most cases impossible. Google: "fundamental problem of causal inference"
That really depends on what you want to model and regression might not be an ideal technique. I suggest that you start by looking at my video on longitudinal analysis.
Amazing video! I was wondering if you know a rule where I can decide what number of sample to use. I am working on a problem where the sample size of the control group is about 400 people and after the treatment there is only 37, How do I know if this is valid?
Yes, at least if you can conceptually argue that there is an underlying latent variable. For example, if we have binary variable "below freezing temprerature", that depends on an underlying continuous variable.
@@mronkko thanks-! That is unfortunately not the case. I have public opiniom survey data from the Eurobarometer and one item asking about European identity vs. National identity (coded as a dummy). I want to analyze whether an EU policy has an impact on European identification. Therefore my plan was to resort to quasi-experimental methodology/DID (to see whether receiving the treatment/policy has an effect). According to your statement, that wouldn‘t work?
@@kasberge7164 I do think it works. I do not think of identity as a dichotomy but a continuum. We feel a degree of identity a (continuous latent variable) and are forced to make a binary choice (a realisation of a measurement process.) I would do a normal DID using linear regression.
Thanks so much!!! In principle, would an ordered response model or logistic regression model also be feasible? I can‘t find anything on this and am pretty new to the subject area and econometrics overall.
I like the DiD chapter in Little, T. D. (Ed.). (2013). (Vol. 1). Oxford University Press. but what is the best book depends on your background knowledge. There are also many good recent articles on DiD with varying levels of technical complexity. For example Athey and Imbens have written on this topic.
Such an enriching video with particular focus on the endogenity and violation of independent assumptions, which not any academic papers have dealt with. I just wanted to ask, can we use DiD as an approach to see the impact of any specific policy implications on an economy across various firm characteristics (probably performance, risk etc) of listed companies.
I would not use DiD for that. DiD requires that you have a treatment group and a control group. What would be the control group be in your case? You could consider the study to be a discontinuous time series design. See ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-I3U2qmsY1xI.html and doi.org/10.1016/j.leaqua.2019.101338
@@mronkko Can not we consider the years prior to the date of intervention as control group and year after the date as experimental group. For expansion support any policy intervention has happened in 2016 c so can the year before 2016 taken as '0' and after 2016 as '1'
Randomized controlled trial would be the best research design. But it experiments are not feasible and you need to work with observational design, the answer really depends on what kind of data you have and what alternative explanations need to be ruled out.
I don't understand why we can't conduct a difference-in-differences analysis without the parallel trends assumption for treatments & controls? For example, to model pre- and post- medical cost trends in treatment and control cohorts D=1 and D=0 for time periods T=0 and T=1 over continuous time X (let's say measured in days), we have: E(Y) = beta_0 + beta_1*D + beta_2*T + beta_3*X + beta_4*D*T + beta_5*D*X + beta_6*T*X + beta_7*D*T*P=X. Then: ATE = E(Y|D=1) - E(Y|D=0) = (beta_0 + beta_1 + beta_2*T + beta_3*X + beta_4*T + beta_5*X + beta_6*T*X + beta_7*T*X) - (beta_0 + beta_2*T + beta_3*X + beta_6*T*X) = beta_1 + beta_4*T + beta_5*X + beta_7*T*X. The ATE at T=1 is then beta_1 + beta_4 + (beta_5 + beta_7)*X. I can see one problem is that the ATE is not a constant, but changes over time if the trends are not parallel - you'd have to use the average value of X in the T=1 time period. If we compare average Y values in the T=0 and T=1 periods we don't have to worry about parallel trends since we're using a binary time category.
The parallel trends assumption means that both the control and treatment groups would have developed similarly had the treatment been applied. If the treatment group had developed differently regardless of the treatment, we cannot say that the treatment caused the difference. In the video I talk about the basic DiD with two time periods. If you have more time periods available, you can relax this assumption to some extent. There is quite a lot of recent work available that addresses this issue:. e.g. doi.org/10.1177%2F0962280218814570
@@mronkko If there are only two time periods (pre and post) then the average pretreatment outcomes have to match fir DiD analysis, correct? But for 2+ pre-treatment time periods the trends have to be parallel but intercepts can be different? I think I understand why we need parallel trends, but shouldn't intercepts match too? Otherwise the pre-treatment populations don't match - there could be different distributions of measured (or unmeasured) covariates. Also issues with how to measure "trend"? If we measure trend as % growth then over time trends will naturally diverge if the intercepts are different. Are they still considered parallel? If intercepts differ we can match on pre-treatment outcomes, but then there may be regression to the mean effects from pre to post time periods biasing ATT estimates. Maybe match on outcome z-scores instead?
The same way you apply it to medical data. 1) You justify the parallel trends approach based on theory and empirical checks of pre-intervention trends and 2) You estimate a DiD model using regression or some other technique depending on the number of pre- and post-intervention periods.
Hey, I kind of understand diff-in diff, now I am dealing with a problem, what if the control is on way larger levels than the treatment Lets stay Control before: 100, after: 200 = 100 % increase, Treatment before: 5, after 9. If I calculate the DID efffect using the standard table so like the diff between differnces i get in this case 100-4= 96!... So the conterfactual state of the world would in the case of treatment be 105 ? !, that does not make sense no? Even the R with OLS gives me these results. What am I doing wrong? Thank you!
Depends on your research question. If you really think that the parallel trends assumption holds, then your DiD estimate is valid. If considering relative changes makes more sense than absolute change, then you can use logs as you suggest.
