Difference in Differences Estimation in Stata 

The purpose of this video is to demonstrate the basic technique of differences in differences estimation. You can certainly add controls to the basic model, but that is outside the scope of this video. You can search my channel for other videos on that.

Can you help me to do my diff in diff other way I will miss my year we can talk about price

@@fgghdfg8638 I'm sorry that I just see your message now. Hope you are doing fine with your dissertation:)

Can you help me in this technique?

Hello can you help me?

Yes, you can.

You don't have to do that, but you might want to look into fixed effects models for that kind of thing.

You can interact a time variable (linear trend, or quadratic, etc.) with a treatment dummy variable.

Yes, you can use a twoway graph to do that.

You can only do this if the categorical variable is binary (eg. married and not married). Assign a 1 to married and 0 to unmarried. We now have a linear probability model (see my video on binary choice models). The interpretation of the diff-in-diff is now the difference in probability of being married.

At the beginning of the video, I show the data browser and scroll through the data. You can see some observations are before and some are after.

I wanted to keep things simple and focus just on the diff in diff technique. However, you can certainly add more variables to the regression as controls.

I have some other videos on general panel data methods.

Usually, this is done informally by comparing the dependent variable movement across groups in an extended period of time before and after the treatment goes into effect. You need a lot more data than I have in this example.

@@sebastianwaiecon thank you!

The actual source of the data shouldn't matter here, whether it's from a survey or not.

I face exactly the same situation, can you please reply to the above question?

I haven't, but a Google search turned up some resources. Best of luck with it.

You can still make a variable indicating before and after treatment. You might also want to think about a fixed effects regression.

1. I have no idea, but it sounds like you have enough data to make that determination yourself. 2. You should think about this on a case-by-case basis. Think about what you're trying to accomplish and whether or not interactions would help with that. 3. Time dummy variables are an important component in FE. I have some videos on FE and panel data on my channel.

Most likely what happened is that you made a mistake creating your dummy variables. Click the magnifying glass button to look at your data to check what went wrong.

Yes, you can take the natural log of the dependent variable to get an approximation of a percentage change.

If you want to know about p-values, I suggest taking a look at my video on hypothesis testing: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-lhoqZjQHHjk.html

For a binary variable, you will get the same result just putting the variable in or using the i. structure. If you have a categorical variable with more than two possible values, then you need to use i.

@@sebastianwaiecon Thanks a lot!!

I don't use coefplot, but I don't see why it would show results any different from your regression table.

I don't see any reason why not. However, with only 2 years of data, you have no idea of how the outcomes have been trending over time, and you may have a hard time justifying your counterfactual.

thank you so much.

I did that in the video. The variable nearinc is the dummy variable for the treatment group.

I don't know what you mean by "run the data" here.

Yes, I forgot to mention that in the video. You can add controls to the diff in diff regression as in any other.

Thanks. Another question would be, it is not necessary to tell Stata we have Panel Data when we have already created the dummy variables that differentiate the control and treatment group, and the pre and post periods? No need to run a fixed effects regression too, I guess. I'm just learning about the subject :)

For a simple DD like this, you don't need to use xtset, if that's what you're asking. You can actually think of a DD as a very simple sort of FE model that only has two groups and two periods. If you want to see more about FE, I also have a video on it.

I really appreciate your responses. Keep helping us!

Yes, you can do it. Generate a new variable for the difference, then regress the difference on a dummy variable for the treatment group.

Thank you very much! What about the interaction dummy between year and dummy? Given that my dataset is a balanced panel of 400 firms observed in both 2008 and 2013? Thanks again

With the wide dataset, there's no interactions as you've already built it in by taking the difference ahead of time.

@@sebastianwaiecon Just to follow up on this, if you do have the same units before and after, the paired difference test gives a different result than the regression you discuss in the video: Y = b1 + b2*treat + b3*time + b4*treat*time, which assumes independent samples, does it not?

I believe the estimate would be the same, but the standard error would be different.

I don't understand the question. What I showed here is the most basic version of diff in diff, with the bare minimum amount of variables needed. Even if I had added more variables, that would not have created any bias -- bias happens because you left variables out.

@@jackgandhi Thank you for the fast reply! Sorry I meant the covariate data structure. I recently did an DiD setup making use of this video's datastructure - and got the criticism that, since I included covariates with a time index for the post traetment period in the regression - these were endogenous and would thus impose bias.

@@ssjvegeto4ever What you are describing is a common and valid criticism of time series analysis. The purpose of diff in diff is, if the data allows, solving this problem using a control and treatment group. The "post" dummy (y81 in the video) is not enough to establish a causal relationship. This is why we have the interaction term (y81nearinc in the video). In this video, y81 controls for effects over time that are constant across groups while nearinc controls for group effects that are constant over time. The interaction pulls out the estimated effect. This is not to say this method is perfect as there could still be endogeneity due to variables that are constant neither across groups nor across time, so you still may need to think about controls. The diff in diff method is just one tool in the analyst's toolbox.

The number you gave estimates the difference between the treatment and control group before the treatment. We need to use the coefficient estimate for the interaction term to get the DID estimator.

Unfortunately, we can't do it with this dataset, since we don't have extra data on either side of the change.

Create an interaction term by multiplying the two variables you are interacting.

You might want to look at my videos on heteroscedasticity.

I can't comment on specifics as I've never combined all of these myself. However, both diff in diff and propensity score matching can be done with dichotomous dependent variables. You just need to be careful about the issues inherent in linear probability. See my video on binary choice models for details.

The most important thing for diff in diff is to identify a control and treatment group. In your case, that might be countries that were part of NAFTA and countries that were not.

@@sebastianwaiecon Enjoying your video.. But I neend help.. I have 25 countries and data from 1960-2020... How can I specify only one time 2012 while comparing it 2010-2016.. please help me

@@amnashaukat7827 A fixed effects model may be more appropriate: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-H95BHswbT3w.html&ab_channel=SebastianWaiEcon

This question is specific to data I don't have any experience with and is therefore outside the scope of this video.

It's all relative when it comes to standard errors. You could say an SE of about 8000, as it is here, is large, but the estimate is -20,000. Standard errors are always going to be big numbers when dealing with things like the prices of homes, which are in the tens of thousands. All other things being equal, larger standard errors mean less precision in the estimates. Here, we can still be quite confident the incinerator did decrease property values.

You could also consider including interactions between y81 and neighborhood 2 and y82 and neighborhood 1. Once we get into more than 2 periods you should also be thinking of this as a fixed effects model. You may find my video on that helpful.

@@sebastianwaiecon thank you very much! I will give a look to the video!

Reg is the basic regression command and xtreg is used for panel data methods such as within estimation and random effects. If you don't have the same units across years (pooled cross section), then you probably want to use reg.

Do you mean you have multiple periods before and after the change? It functions the same as this, but you need to define your "post" variable to include all periods after the change.

@@sebastianwaiecon So if I have 2000-2010 data, and the policy happened in 2005. I need to set 2000-2004 equal to '0', and 2005-2010 equal to '1'?

That would be the simplest way to do it. I'm not promising this is the perfect solution as you may need to think about more sophisticated ways to handle your specific data, but it is a good starting point.

The principles which drive DD -- controlling for time trends and cross sectional trends -- are still useful for logits (and probits also). However, you need to be careful about the coefficient interpretations, as it's not as clean as in the least squares DD. I would suggest looking at my video on binary choice models for details.

For the code, yes, you can change "regress" to "logit" and it will run.

Thank you!

Unlike FE models, diff in diff does not necessarily have the same cross-sectional units across time periods. In my example, it's not the same houses in '78 and '81. As such, ID-based FE won't work. Here, the nearinc variable plays the same role as the FE. Your time dummy is already in there in DD.

Yes, I get that. I have unbalanced panel data and I want to conduct a Difference-in-Differences with id and time fixed effects. Is // xtreg DepVar i.treated##i.during controls i.month , fe cluster(id) // the correct model to achieve that? Or do you think that it would be better to exclude the fixed effects?

If I'm understanding what you're trying to do correctly, I think you can include the fixed effects.

Firstly thank you for your video which is very helpful. As you have mentioned in your comment it was not the same house in '78 and 81', does that mean your treatment and control group are not the same pre and post-treatment ?

The criterion for being in the control or treatment group is the same in both years, but the specific houses aren't the same.

In this dataset, that is y81 -- a dummy variable with a 1 for 1981 and 0 otherwise. I have another video with some examples of how to create dummy variables: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-DuAhUpM-56E.html

You don't "fix" it, it's just the result you got. It tells you that you can't reject the hypothesis that your treatment had no effect. Now, it could be that you have some endogeneity that you need to control for, but statistical significance, or lack thereof, is not (by itself) a problem to be fixed.

@@sebastianwaiecon Hi, is the interaction term is insignificant, will adding more variables help us getting the result significant? Since, in the results show that the constant term is highly significant, which means that there is an omitted variable bias. I guess, adding more controls can help solve the problem for the insignificant interaction term.

​@@consultingfaqs It bears repeating that the treatment not being significant is not a "problem" to be be solved unless you think this is because of an omitted variable. Tinkering around with different models with the explicit purpose of finding a significant effect is not an ethical use of data. The constant term being highly significant is also not evidence of omitted variables. I'm not sure where you got that idea. Adding more variables might or might not result in existing terms being more significant. It all depends on the direction of the bias, if there is one.

nearinc indicates whether the house is within 3 miles of the incinerator. There is a variable called "dist" which is the distance from the incinerator in feet. To create the dummy, we would use the command: gen nearinc = dist

You are correct. It's a pooled cross section. It would be very unlikely for the same houses to be on the market in both years.

@@sebastianwaiecon Good point and thank you so much for the quick reply! I am working on a thesis and realised that I was supposed to be doing DiD when I had done a different methodology for the many few weeks. Your video is incredible. Big thanks from Australia!

Happy to help!

You can add control variables, but you'll have to run a regression rather than using the collapse method.

@@sebastianwaiecon Thanks :)

I get, that I can solve this problems by working with log-level model. But isnt this problem always with level-level dif in dif? What Am i missing?

You can do diff in diff with the dependent variable in logs. That's no problem as long as you are careful with the percentage change interpretation.

First of all, "autocorrelation" is a very specific term, which you are using incorrectly. In time series data, this refers to a variable correlating with itself across time. In any case, you've pointed out that an interaction term is correlated with one of the variables you are interacting. This is true by definition. There isn't anything you do about that -- it would be strange if it were not the case. In a more general sense, there is nothing wrong with two variables in a regression being correlated with each other. That is completely normal and probably the case in most regressions.

Thank you for pointing out my problem. You are right, it was my fault in using the term "autocorrelation". What I really mean is the "multicollinearity" but there was a mistake in typing. Anyway, according to the data in the video, the truth is "multicollinearity" really happens in the regression result because the coefficient of correlation between " nearinc" and "y81nrinc" variables is 0.5776. Usually, in the case of encountering "multicollinearity", we usually omit one of the two variables out of the model. However, it is impossible to omit any variable of these two variables due to the requirement of "Difference in difference" method because they must be included together to show the effect of the construction of the incinerator. That is why I asked the question "what should we do in this situation". And this problem not only happens in this example, but it also occurs in every "DID" model because we usually create a "did" variable by multiplying the "time" and "treated" variables (did = time * treated). And the consequence is there always is "multicollinearity" in "DID" model. Can you help me to solve this issue?

Multicollinearity is not a big deal. Getting into the practice of dropping variables because they are correlated with another variable in the model will lead you quickly into omitted variable bias. There is a simple test where you regress the one variable you are concerned about on all the other explanatory variables. If the R-squared is under 0.9, don't worry about it. As I explained previously, it is mathematically impossible for a variable and an interaction term involving it to be uncorrelated. The interaction term is absolutely key to a diff in diff regression.

@@sebastianwaiecon Thank you very much for the explanation.

@@sebastianwaiecon Hello, I found this video very helpful. However, when running my model, my DID variable keeps getting dropped because of collinearity. Is there a fix to that?

You may want to consider a fixed effects model instead.

And how come the interaction variable is all zero?

You can do DID if you set up a dummy variable to indicate when the treatment went into effect. Once this is in place, you can create the interaction term.

Thanks much for your reply sir

It's all in the video.

This is the dataset KIELMC.dta from the Wooldridge econometrics textbook. It is widely available online.

@@sebastianwaiecon thank you. also for kindly reply my message. God bless. stay safe stay healthy

I'm not aware of any changes in recent versions of Stata that would change anything in this video.

1) If what you're after is measuring the treatment effect, this doesn't matter. 2) I don't know what you mean by "take care," but R squared is not particularly relevant in DID estimation.

According to the original paper, construction took place from 1981-1984.

The idea will be the same -- create dummy variables for treatment and time and an interaction, then put those in a regression.

Mechanically, you can do it with dummy variables (see my fixed effects video). While I am not aware of a specific reason you should not do so, I don't know enough to definitively tell you one way or another.

The dataset is KIELMC.dta that comes with the Wooldridge econometrics book. You should be able to find it online.

Yes, it would. See my video on natural logarithms for how that would work.

or, like in your example, incinerator would have been installed for one neighborhood in 1981, for another in 1985 etc, for another in 2005... y81 time dummy won't work anymore, so what do we have to interact?

You'll need a dummy variable that "turns on" from a 0 to a 1 once the treatment is active. You won't be able to do this by building an interaction term, as it's more complex than that now. I'm not sure there's a better way than putting in the 1s on a case by case basis.

You can simply add control variables to the DID regression, if you want.

@@sebastianwaiecon I'm a STATA beginner, so can you explain a little bit more about where to put this part?

For that, you might want to look into a full fixed effects model. I have a video on that, as well.

It's KIELMC.dta, which comes with the Wooldridge econometrics textbook. You should be able to find it online.

You are always allowed to add controls if you think the DD method did not eliminate endogeneity.

And if we have 5 years of period 2007 to 2011 and the policy is announced at 2009, how to set the year variable?

I always recommend you go to the data browser and take a look at the values. Presumably something went wrong in your variable generation.

I also have this problem. My interaction term is omitted due to collinearity, does anyone know how you can fix this?

I have the same issue - what sort of issue in the variable generation can result in this?

@@bencaplan4565 for the time dummy, the control group also gets a 1 even if it is not treated at all

Yes, it is. In this example, you could imagine there might be a difference in the variance of prices with and without the incinerator.

@@sebastianwaiecon Then to solve this, would you add the vcerobust command at the end of your regression?

I can't think of a theoretical reason why you couldn't do that. To be honest, I think most people just use robust all the time and don't really think about it.

Yes. You would do this after running the collapse to get all the averages. The "classic" diff in diff graph has the outcome on the vertical axis and time on the horizontal axis. There are three lines: the treated group, the untreated group, and a counterfactual with the same starting point as the untreated group but the same slope as the treated group. See my video on graphing for how to use the twoway command.

@@sebastianwaiecon Thank you will do!

I recommend you ask your professor for help - it's what they're there for!