The foundation for my exposition comes from Mas-Colell's examples in Ch. 6.
The maximization problem for your specific question can be generalized pretty easily. Consider the case with one risky asset and one riskless asset. Let $\beta$ be the wealth invested in the safe asset, normalized to 1 dollar per dollar invested. Let $\alpha$ be the wealth invested in the risky asset, which has some random payout $z$ such that:
$$\int z \ \text{d}F(z) > 1$$
So that the mean return is greater than the riskless asset. So we express our maximization problem as:
$$\max \ \int u(\alpha z + \beta) \ \text{d}F(z) \\
\text{s.t.} \quad\alpha + \beta = w$$
You can take advantage of the fact that $w - \alpha = \beta$ $\implies \alpha z + \beta = w + \alpha(z - 1)$ and find first order conditions. If $u$ is concave (risk averse) then Kuhn-Tucker FOCs which combine to make:
$$\int u'(w + \alpha(z - 1))\cdot(z - 1) \ \text{d}F(z) = 0 \quad \text{iff} \quad \alpha \in (0, w) $$
So for the generic case, you can do the same setup with $N$ risky assets and one riskless asset that is better than whatever other riskless assets are out there. Let's normalize it again to payout of 1.
So the maximization is now:
$$\max \int u(\alpha_1 z_1 + \cdots + \alpha_N z_N + \beta) \ \text{d}F(z_1, \cdots z_N) \\
\text{s.t.} \quad \alpha_1 + \dots + \alpha_N + \beta = w$$
Notes:
If you have already done the easy case, this general case should not be so bad, just some more work. For other readers, I'll state the definitions of constant absolute and relative risk aversion below, respectively.
$$r_A(x) = -\frac{u''(x)}{u'(x)} = n \quad \forall x$$
$$r_R(x) = -\frac{x \cdot u''(x)}{u'(x)} = n \quad \forall x$$
