I hope this question is not too general in nature; I just wanted to ask the members of this community their opinion on the following question: How can one choose between the Bayesian approach and the Frequentist Approach? In what context is one approach better than the other? The reason I ask this is because whenever I attempt to answer causal empirical questions, I immediately think of trying to estimate Treatment Effects (for instance using difference in means, Difference-in-differences or matching estimators). On the other hand, when I wish to forecast, Ive heard that Bayesian Model Averaging Techniques often outpeform linear projections. On what basis can I choose between the two approaches? Thanks!
$\begingroup$
$\endgroup$
2
-
1$\begingroup$ Whole books have been written about this. It's far too broad. $\endgroup$– 410 goneOct 7, 2015 at 12:44
-
3$\begingroup$ Looking at the body of the question, I suggest that you change the title into something like "Criteria to choose between the Bayesian / Frequentist approaches in empirical applications". I believe this will help avoid gathering here answers that are just declarations of philosophical creed on this "scientific civil war", which thankfully becomes more and more outdated, as researchers understand the imperfections and limitations of both approaches, together with their individual merits, and start seeing them as complementary sails to keep the ship afloat in this uncertain sea we live in. $\endgroup$– Alecos PapadopoulosOct 7, 2015 at 18:08
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
This is a more "practical" answer, vs a deeper theoretical one, or even a specific one.
Take it as "a broad answer to a broad question."
Also, since it is a "Bayes vs frequentist" issue, at least the last few suggestions must be taken tongue-in-cheek.
- Step 1: Use simple models you understand to start. Lowering researcher error due to misapplication should always be a first-order priority.
- Step 2: Learn more about each approach as much as possible. Implement the same estimator from both perspectives if at all possible, for purposes of really wrestling with what is going on at a deep level. Again, start simple. replicate things you already know work and are well established. (Can you write a basic regression from scratch using both approaches? This is much deeper than it may appear.) Get some good introductory texts. Don't be afraid of "lowly undergrad" texts for introduction. The "puppy book" on Amazon is good here, as is "Think Bayes" from Green Tea (both for initial intro to Bayes).
- Step 3: Hopefully the earlier efforts will start revealing that some problems are more naturally approached in one framework vs. the other. When you start getting a good feel for this, re-examine the problem you want to address. Which approach seems appropriate to you now?
- Step 4: realize that if you are using likelihood-based methods, Bayesian and frequentist are two sides of a similar coin. (Or to quote the ever-entertaining Cosma, "Bayesian estimation is a likelihood-based method in which the impact of facts and experience is blunted and smoothed by prejudice.," haha.)
- Step 5: realize that there are even more varieties of estimation which don't neatly fit into previously-understood categories -- non-parametric, semi-parametric, semi-non-parametric -- and begin to despair that you will ever know or understand all there is to know about estimation, and which is the absolute best method for any one particular question.
- Step 6: Enlightenment. Use the tool that you believe is best for the task, in your own honest opinion -- even if it is "best" based only on Step 1, which is simply "I use this because I understand it the best and I know I am not accidentally doing something extremely wrong."
So there you have it, "a broad answer to a broad question."
All that said, I really do like both Bayesian and frequentist approaches. A lot of things goes into deciding which to use. Some problems lend themselves well to one vs the other. Some audiences lend themselves to understanding one vs the other (recall that often effective communication of a concept can be very important). It's up to you to learn enough about both the methods and the problems you are trying to address to choose well.
-
2
-
3$\begingroup$ (+1) As a trained frequentist I could offer that "Frequentist estimation is in reality a sub-case of the Bayesian approach, where our prior belief on the unknowns is that they are constants and so they don't have a distribution". $\endgroup$ Oct 7, 2015 at 19:43
-
$\begingroup$ I kind of like the idea that the Bayesians consider a distribution of parameter estimates instead of point estimates. I'm not sure, but are ='Bayesian Treatment Effect' estimators common in the literature? $\endgroup$– ChinGOct 9, 2015 at 13:58
-
1$\begingroup$ @ChinG: I googled the term you suggested and found some good-looking results from top econometricians and statisticians: "Treatment Effects: A Bayesian Perspective," by Heckman et al., a paper by Gelman. Those may be good places to start. $\endgroup$– CompEconOct 10, 2015 at 19:01
$\begingroup$
$\endgroup$
2
Do you want something backed up by solid theory, and can accept a numerical approximation as the answer? Bayes
Are you working with open systems, and need rigour? Bayes.
If you don't mind that it doesn't stand up in theory, but do need it to have an analytic solution? Frequentist.
You don't care about rigour, you want a quick and dirty answer, and you can bear to use the justification: "lots of other people have done it this way previously"? Frequentist
Are you working with closed systems with well-defined unknowns, and no informed probabilistic priors? Frequentist or Bayes. (usually frequentists will have an off-the-shelf recipe to do it quicker and more efficiently, but you should get the same answer either way)
-
$\begingroup$ Accepted. But structural models (arguably well grounded in theory) exclusively use OLS techniques. I have yet to come across a paper that use Bayesian methods in an empirical setting (for example Labor Economics Journals, Development Economics Journals). The only times I have seen Bayesian techniques being used in conjunction with frequentist ones is forecasting. $\endgroup$– ChinGOct 7, 2015 at 12:56
-
1$\begingroup$ I think that's my 4th point, then; the defence of: "lots of other people have done it this way previously". If you want the rigorous theory, then reading ET Jaynes is one place to start. Or get onto one of David Draper's rather magnificent training courses. $\endgroup$– 410 goneOct 7, 2015 at 12:59