I hope this question is not too general in nature; I just wanted to ask the members of this community their opinion on the following question: How can one choose between the Bayesian approach and the Frequentist Approach? In what context is one approach better than the other? The reason I ask this is because whenever I attempt to answer causal empirical questions, I immediately think of trying to estimate Treatment Effects (for instance using difference in means, Difference-in-differences or matching estimators). On the other hand, when I wish to forecast, Ive heard that Bayesian Model Averaging Techniques often outpeform linear projections. On what basis can I choose between the two approaches? Thanks!
This is a more "practical" answer, vs a deeper theoretical one, or even a specific one.
Take it as "a broad answer to a broad question."
Also, since it is a "Bayes vs frequentist" issue, at least the last few suggestions must be taken tongue-in-cheek.
- Step 1: Use simple models you understand to start. Lowering researcher error due to misapplication should always be a first-order priority.
- Step 2: Learn more about each approach as much as possible. Implement the same estimator from both perspectives if at all possible, for purposes of really wrestling with what is going on at a deep level. Again, start simple. replicate things you already know work and are well established. (Can you write a basic regression from scratch using both approaches? This is much deeper than it may appear.) Get some good introductory texts. Don't be afraid of "lowly undergrad" texts for introduction. The "puppy book" on Amazon is good here, as is "Think Bayes" from Green Tea (both for initial intro to Bayes).
- Step 3: Hopefully the earlier efforts will start revealing that some problems are more naturally approached in one framework vs. the other. When you start getting a good feel for this, re-examine the problem you want to address. Which approach seems appropriate to you now?
- Step 4: realize that if you are using likelihood-based methods, Bayesian and frequentist are two sides of a similar coin. (Or to quote the ever-entertaining Cosma, "Bayesian estimation is a likelihood-based method in which the impact of facts and experience is blunted and smoothed by prejudice.," haha.)
- Step 5: realize that there are even more varieties of estimation which don't neatly fit into previously-understood categories -- non-parametric, semi-parametric, semi-non-parametric -- and begin to despair that you will ever know or understand all there is to know about estimation, and which is the absolute best method for any one particular question.
- Step 6: Enlightenment. Use the tool that you believe is best for the task, in your own honest opinion -- even if it is "best" based only on Step 1, which is simply "I use this because I understand it the best and I know I am not accidentally doing something extremely wrong."
So there you have it, "a broad answer to a broad question."
All that said, I really do like both Bayesian and frequentist approaches. A lot of things goes into deciding which to use. Some problems lend themselves well to one vs the other. Some audiences lend themselves to understanding one vs the other (recall that often effective communication of a concept can be very important). It's up to you to learn enough about both the methods and the problems you are trying to address to choose well.
Do you want something backed up by solid theory, and can accept a numerical approximation as the answer? Bayes
Are you working with open systems, and need rigour? Bayes.
If you don't mind that it doesn't stand up in theory, but do need it to have an analytic solution? Frequentist.
You don't care about rigour, you want a quick and dirty answer, and you can bear to use the justification: "lots of other people have done it this way previously"? Frequentist
Are you working with closed systems with well-defined unknowns, and no informed probabilistic priors? Frequentist or Bayes. (usually frequentists will have an off-the-shelf recipe to do it quicker and more efficiently, but you should get the same answer either way)