Some researchers have answered the question in their papers, but I was wondering how one would proceed to estimate the impact of college lockdown on undergrad 1st year students performance, if we consider some constraining assumptions.

Let's assume that we can only gather data from one college (due to time and administration constraints) through a survey (so not public data, which could result in a small sample size if the response rate is low) and that students are exceptionnally diligent, i.e they attended classes and seminars all the time before the lockdown and they keep the same behavior for online classes (Zoom).

Let's say that lockdown is the "treatment". The thing is, lockdown affects all the students in a given college, and everyone has to switch from a face-to-face setting to an online one. How should we proceed to measure the effect of lockdown, as there is no apparent control group?

The "control group" would be a group of students who are somehow not affected by the lockdown. To solve this conundrum some researchers used an interesting strategy :

Hansen et al. (2021) consider that students who don't attend the lecture/exercise session/tutorial form the control group. It seems to make sense, as they basically study on their own for the final exam. Then when the lockdown happens, well they keep studying on their own. So in a sense they're not affected by the lockdown.

However, given the assumptions made before, we wouldn't have students who don't attend in our sample. I thought of a specification where Top-quartile students could form the control group, the reasoning being that they should be really versatile in the learning method, so the lockdown shouldn't have any (negative) effect. But Zhang et al. (2021) found that top students see their math scores increasing with an increase in online watching time, so the lockdown definitely has an effect on top students. Hence top students can't form the control group as they would be positively affected by the treatment.

Therefore I'm at a loss for solving the issue of the control group. Is there a research methodology that is appropriate for this (I roughly understand the basics of synthetic control method, but it requires lots of data which isn't feasible according to the assumptions), or a nice idea to identify students "not affected by lockdown" ?

  • $\begingroup$ How about using first year students of a pre-Covid academic year for a control group? $\endgroup$
    – Giskard
    Oct 23, 2021 at 21:32


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy