Python vs R (vs Stata): the old battle revisited

Question

I am an avid Python user. I know Stata but I'm not a pro. I don't know R. I do econometrics (mostly time series, but also cross-section and panel), and statistics, and Python seems quite sufficient in meeting my needs.

I know that economists (at least old schoolers) are mainly using Stata and statisticians mostly R.

I read here and there that 1) R's libraries are superior to Python's, and 2) when it comes to visualization nothing can beat R's ggplot.

My questions are:

a) What are examples of econometric/statistical analyses that can be easily done in R but not (as easily or at all) in Python?

b) Is visualization in R really superior to Python? In what sense: less coding, ease of coding, more intuitive, or the quality of the final output?

c) Where will you put Stata in this comparison?

Answer 1

I use all three programs.

Python can do everything that R can do and R can do everything that Python does, but I must say R is superior to Python when it comes to the packages. For that reason for most econometric analysis I usually default to R. I find also producing nice standard statistics graphics with R easier (but for maps I prefer Python).

However, Python is far superior for web-scraping, numerical analysis and sentiment text analysis (although R has some good packages for that as well). Also, I prefer to use Python when I need to set up my own program as programming in Python is more natural (if that makes any sense) than in R, unless the program can be build easily from ready made functions from various packages.

I always recommend to people around me to learn both Python and R - the difference between them is not that big and with R you don’t really need to invest heavily into programming skills but just basics and then use packages.

Also, Jupyter Notebooks that can accommodate both R and Python make using both of them easier.

When it comes to Stata I use it only for a educational purposes (I teach econometric tutorials at university). To be honest I don’t like stata for several reasons:

1. Stata is not a freeware and i don’t think that the price tag is justified given that it’s inferior product compared to free ware programs like R and Python. If you can get it for free from uni then you probably don’t care about that but still it’s something to keep in mind.

2. Stata is a program not a language so if you want to create a new complex function you need to separately get and learn Mata (statas programming language).

3. Stata has some serious limitations on matrix sixes even in the most expensive edition the max mat size is 11000 which is serious limit when you work with panel data and have to run some iterative model with large number of variables. You will be routinely forced to run for example panel LR heteroskedasticity test on random subsamples even with the most expensive edition.

4. Creating beautiful graphics in Stata is nearly impossible - now don’t get me wrong you can make some decent graphics in Stata but it pales in contrast to what you can do with Python or R.

5. Stata is clunky with time series analysis. If you look for easy to use program for time series analysis it would be EViews. For example Stata can’t handle if you have quarters that are expressed in date format it will think you have gaps in your time series and won’t let you run basic time series commands until you create new time series variable. Also the offer of time series models is quite low and programming your own is painful.

6. While technically possible webscraping or numerical analysis in Stata is hell - if you need that for your work don’t use it.

However, Stata has also few advantages:

1. It’s more user friendly than R or Python, and can be even used without coding through interface (that’s why we use it for tutorials for students as a first program they see so they don’t get overwhelmed).

2. Doing some adjustments to datasets creating dummy variables etc is easier compared to Python or R but that’s mostly because in both Python and R you can have various types of data, lists, data frames etc.

3. With purchase to Stata you get access to Stata forums - it’s something like stack exchange but they actually pay professionals to give you answers there and often beside support you can get there very good advice even on econometrics, and you will usually get advice really quick.

I personally would not use Stata for my own scientific work. Python and R are superior to every aspect of Stata - but it’s very good starting program for students.

Answer 2

I use Stata and Python heavily. I have dabbled in R, but won't pretend I know it well enough to comment on it. Stata and Python complement each other nicely and I am a big fan of both. You can run Python code in Stata and Stata from Python. The key distinction is the Stata is purpose-built for data management and regressions, it focuses on causal inference, and all its commands are maintained. Python can do a whole lot more but isn't as good if your focus is causal inference. For either, you really have to roll up your sleeves and learn the language for it to be practical.

Python

• Free
• Object-oriented language. If you used to it and understand classes, it gives you great power.
• You can do a lot more with Python. I use it primarily for machine learning, GIS manipulation (geopandas/folium), building interactive analysis sites (Flask/Django), dealing with API requests, web-scrapping, and working with databases (SQL Alchemy).
• Writing functions in Python is ridiculously intuitive.
• No GUI.
• Lots of web examples and tutorials
• Documentation of package varies quite widely.
• You have to remember the names of each package you want to use.
• You have to worry about compatibility of packages and versions. The language is not backwards compatible. The whole environment needs to align perfectly.
• No GUI or drop-down menus (a double edge sword)
• The ability to deal with multiple dataframes, variables, lists, etc.
• Better ability to dealg with multiple dataframes, variables, lists, etc.
• Supports many more data structures (JSON, GeoJSON, etc.)
• Much better for machine learning.
• Not so good for causal inference and standard errors. To be fair, the main package for it, statsmodels, has been catching up. But most people learn to do regression in Scikit learn, which doesn't even give you standard errors.
• Lots visualisations packages. Seaborn is quite impressive, but the interactive visuals cufflinks, plotly, etc. really pop and can be loaded into any website.
• It's a production language. Meaning I can spin up a server with Python installed and have it receive requests (via website, API) and endlessly run jobs as needed.
• Lots of IDEs which can help with coding. Most people learn it in Juyter Notebooks but if you really want to use PYthon you'll eventually move something like Spyder, Atom, or VS code.

STATA

• Not free
• Functional programming. It follows a grammatical structure. Once you learn your key verbs (commands), it becomes very intuitive but you have to invest the time to learn the language.
• It's better for data management of data in tables. I had a few native Python analyst learn Stata inside/out and they now do most of the data management in Stata and switch to Python for what Python does best.
• It has a GUI. You can use a drop-down menu to find what you want to do and it will print out the code for you. It's a plus and minus since it allows people to use Stata without really learning to code.
• Better ability to browse the data.
• It's backward compatible. No need to worry about the version, a deprecated commands, or whether the two packages will work together.
• No need to remember and load packages. If you type a command, Stata loads the code.
• Code tends to be more concise in Stata, which has a lot of built-in helper functions. Anything I write in Python, I can write in 1/2 to 1/3 the lines in Stata.
• It is supported (versus exclusively user-written). This means you have standardized documentation, and the equations, and citations. Documentation in user-written languages can vary a lot depending on the package and rarely does it include the equations and citations. Howeover, the document is in the program or in a series of very lengthy (but searchable) PDF volumes that come with it.
• Surprisingly poor web documentation and fewer online tutorials
• It is the defacto language if you are focused on causal inference and want to be sure your standard errors are right. For panel regression, nothing that I've tried beats Stata. In contrast, in Python, some of the most common and popular data science packages - scikit learn and keras - don't produce standard errors because they focus is on prediction.
• Writing functions (programs) is less intuitive and clunkier. Once you learn it well, however, you can write very powerful user-written commands (functions) in Stata. If they are good, they are often published in the Stata Journal and become part of the base code, eventually.
• It's clunkier when dealing with multiple datasets (dataframes), variables, and arrays.
• Graphics are easy but the out-of-the-box default options are very basic looking. You can write your own scheme to make the visuals pop, but few people do so.
• Very good ability to produce automated word, PDF, and Excel reports.
• Machine learning hasn't been fully incorporated. In specific, it currently lacks ensemble methods (Random Forest Regressors, XGBoost), and neural nets.
• It's not a production language, meaning I cannot spin up a server with Stata and use to receive requests and produce analysis/reports. If you only using it inside and organization, it's fine, but if you need the analysis to interact with a website good luck (there are workarounds, but it's a pain).

Answer 3

I'd rate my experience: Stata 9/10, Python 7/10, R 3/10

on a) There are many econometric approaches specific to a certain field for which packages have been developed for R and Stata, but not (yet) for Python. One example is the Heckman Selection approach in Labor Economics, which I had to do myself in Python. There are numerous other examples.

on b) Possibly, ggplot is the best combination in terms of flexibility and intuition. Matplotlib in python is pretty much of a pain, I do bokeh as often as possible.

on c) In my view, Stata beats both others in terms of ease of coding. Something like a bar chart with two nested catgories can be done extremely quick and intuitively. The Stata output though is so far mostly limited on publication graphs. So quality of output is inferior. And the data handling before you can put them into a graph is also likely to be more painful in Stata (only one dataset) comparing two the other two (multiple parallel dataframes).

Answer 4

In my experience, I think Python is better for econometrics than R and Stata for the following reasons:

a) In real applications, get and transform data is 60% of the work. For this tasks Python is better.

b) To select the best model and features it's necessary to use loops. Loops in R are difficult but in Python are easier to use.

c) Object oriented programming is easier in Python. This means that we can develop our own objects and libraries easier than in R.

d) Python is a Swiss knife. It can be used for econometrics, for web scrapping, machine learning, ETL, quantitative finance, among other applications.

If you want examples of Python applied to econometrics, you can check on this book https://www.amazon.com/dp/B08KJ1322G that have several examples of python applied to econometrics.