I'll try to translate the argument of Hansen's original (1985) paper
Let $h_t = \alpha_t h$ be aggregate labour supply at $t$.
First define aggregate leisure $\ell_t = 1 - h_t = 1 - \alpha_t h$ which gives:
$$
\alpha_t = \frac{1 - \ell_t}{h}
$$
Then the instantaneous utility function can be written as:
$$
u(c_t) + \alpha_t (v(1-h)-v(1)) + v(1) = u(c_t) + v(1) + \frac{v(1-h)}{h} - \left[\frac{v(1-h)}{h}\right]\ell_t
$$
Here the choice variable $\alpha_t$ changed to $\ell_t$. We see that this function is linear in $\ell_t$ which gives an infinite elasticity of substitution between leisure in different periods.
I'm not a macro person, so I might be totally wrong here. However, for another way to see the issue is to look at your Bellman equation:
$$
V(k_t) = \max_{c_t, \alpha_t, b_t} u(c_t) + \alpha_t v(1-h) + (1-\alpha_t) v(1) + \beta\left[\alpha_t V(k_{t+1}^g) + (1-\alpha_t) V(k_{t+1}^b)\right]
$$
where:
$$
\begin{align*}
&k_{t+1}^g = (1-\delta) k_t + w h - (1-\alpha)b_t - c_t\\
&k_{t+1}^b = (1-\delta) k_t + b_t - (1-\alpha)b_t - c_t
\end{align*}
$$
Here I'm assuming the premium to be based on the market risk ($\alpha$) and not on the individual risk $\alpha_t$, so $\alpha$ is fixed for the individual that is optimizing this bellman equation.
Let us show that 'in general' aggregate frish supply will be $h_t = h$ or $h_t = 0$ except for a particular value of $w$. So let's go with the assumption that $\alpha \in (0,1)$.
Assume that we already solved for the optimal values of consumption and insurance.
To find the optimal value of $\alpha_t$ take the first order condition of $V(k_t)$ with respect to $\alpha_t$:
$$
v(1-h) - v(1) + \beta \left[V(k_{t+1}^g)- V(k_{t+1}^b)\right] = 0
$$
As $k_{t+1}^g$ is increasing in $w$ and the rest is independent of $w$ there can be at most one value of $w$ for which this condition holds.
If $w$ is bigger than this value, it will be optimal to set $\alpha_t = 1$ so aggregate demand will jump to $h_t = \alpha_t h = h$. (I'm hand-waving here as the optimal values of $c$ and $b$ may also of course depend on $w$).
If $w$ is smaller than this value, it will be optimal to set $\alpha_t = 0$, so aggregate demand will jump to $h_t = \alpha_t h = 0$.
This means that aggregate elasticity is infinite.