Okay so, in the book they looked at the first order taylor series of $\dot{k}$, so as @denesp said, we need the time here too! So rather here we should look at the taylor series around $\dot{y}$
$$ \displaystyle \dot{y}(y) \simeq \left[ \frac{\partial \dot{y}(y)}{\partial y(k)} \bigg|_{y=y^*} \right] (y(k) - y(k^*)) $$
$$ \frac{\partial \dot{y}(y)}{\partial y(k)} \bigg|_{y=y^*} = \left( \frac{\partial \dot{y}(y)}{\partial k(t)} \bigg|_{y=y^*} \right) \left( \frac{\partial k(t)}{\partial y(k)} \bigg|_{k=k^*} \right) $$
First we look at:
$$ y=f(k) $$
$$\Rightarrow \dot{y} = \frac{d}{dt} f(k) = \frac{d f}{d k} \frac{dk}{dt} = f'(k) \dot{k} $$
We know the key equation of the Solow model is:
$$ \dot{k}(t) = s f(k(t)) - (n+g+\delta)k(t) $$
$$ \Rightarrow \dot{y} = f'(k) \left[ s f(k(t)) - (n+g+\delta)k(t) \right] $$
We take the derivative of this with respect to capital:
$$ \frac{\partial \dot{y}}{\partial k} = f''(k)\left[ s f(k(t)) - (n+g+\delta)k(t) \right] + f'(k) \left[ s f'(k) - (n+g+\delta) \right]$$
The value $k^*$ is the golden-rule level of the capital stock so:
$$ s f(k^*) = (n+g+\delta)k^* $$
And hence
$$ \left( \frac{\partial \dot{y}}{\partial k} \bigg|_{y=y^*} \right) = f''(k^*)*(0) + f'(k^*)\left[ s f'(k^*) - (n+g+\delta) \right] $$
Next we use the hint (not much of a hint, and rather misleading to begin with in my opinion)
$$ \left( \frac{\partial k(y)}{\partial y(t)} \bigg|_{k=k^*} \right) = \left( \frac{\partial y(k)}{\partial k(t)} \bigg|_{y=y^*} \right)^{-1} = f'(k^*)^{-1} = g'(y^*) $$
Plugging both of these in to our first order partial derivative:
$$ \frac{\partial \dot{y}(y)}{\partial y(k)} \bigg|_{y=y^*} = \left( f'(k^*)\left[ s f'(k^*) - (n+g+\delta) \right] \right) * \left( f'(k^*)^{-1} \right) $$
$$ = s f'(k^*) - (n+g+\delta) $$
since around the balanced growth path $s = (n+g+\delta)k^*/f(k^*)$ and putting
$$ - \frac{\partial \dot{y}(y)}{\partial y(k)} \bigg|_{y=y^*} = \lambda $$
$$ \lambda = (n+g+\delta) - \frac{(n+g+\delta)(k^*)*f'(k^*)}{f(k^*)} $$
With $\alpha_k = \frac{k*f'(k)}{f(k)} $ being the elasticity of output with respect to capital we obtain:
$$ \lambda = (n+g+\delta)(1-\alpha_k(k^*)) $$
So $y$ converges to its balanced growth-path value rate $\lambda$ the same as $k$ converges to $k^*$