The non-mathematical answer I believe is well described in Straub & Werning's last paragraph:
"In quantitative evaluations it may well be the case that one finds a
zero long-run tax on capital, e.g. for the model in Judd (1985) one
may set $\sigma < 1$ ($\sigma$ is the reciprocal of the intertemporal elasticity of substitution), and in Chamley (1986) the bounds may not bind forever,
depending on parameters. In this paper we stay away from making any
such claim, one way or another. We confined attention to the original
theoretical results, widely perceived as delivering zero long-run
taxation as an ironclad conclusion, independent of parameter values.
Based on our analysis, we find little basis for such an
Just above their Proposition 2, there is also another illuminating passage: S&W write:
"(...)This shows that the solution cannot converge to the zero-tax
steady state. Indeed, it actually proves the solution cannot converge
to any interior steady state, since, we argued, the only possible
interior steady state is the zero tax steady state."
In other words, it appears that Judd and Chamley did not fully solved their models, but provided results conditional on the parameters being such that the steady state will be an interior one. S&W argue (I have not checked mathematical correctness), that, depending on the parameter values, the optimal solution may lead to a corner steady state in some aspect (see below for an example) -and in a corner steady state, the capital tax-rate will be positive. This needs checking because Judd explicitly considers different values for $\sigma$ (so a plain mistake may be involved after all).
Now if you ask me, this mostly indicates that the tools used may not be well suited after all (or we have somehow "misused" them) to solve the specific theoretical problem, and so they are unreliable to affect policy one direction or the other (which, by the way, reminds me of my question...)
Because, exactly of what value is (for either theory or policy) to, say, empirically determine that the elasticity of intertemporal substitution is lower than unity, and then declare "the optimal taxation solution for a government that cares only about the workers entails positive tax on capital and zero consumption of workers" at the steady state? (see Proposition 3, p. 11 of S& W). Who is going to contemplate seriously such a proposition for purposes of real-world policy?
$\sigma >1 \Rightarrow$ the intertemporal elasticity of substitution ($1/\sigma)$ is lower than unity. $\gamma =0\Rightarrow$ the planner puts $0$ weight on capitalists' utility. $c_t$ is the workers' consumption, $C_t$ is capitalists' consumption, and $g$ is governments own consumption (i.e. it is not the part of government budget that goes to the workers as transfers).
Again, I have not checked the mathematics here. Apart from what kind of economic content and relevance one could conceivably provide for workers consumption going to zero, another issue here is that, if the limiting tax rate is going to unity as government own consumption goes to zero, then what happens to the Tax revenues (which are not given to the workers as transfers because then their consumption could not go to zero?)