The price elasticity of demand is $$-\frac{dQ}{dP}\frac{P}{Q}.$$
(Note that here and elsewhere in my answer I have defined the elasticity so that is positive when demand is downward sloping.) For a linear demand curve, the slope ($dQ/dP$) is constant and so the size of the elasticity is proportional to $P/Q$. Since the demand curve is downward sloping, when $P$ is higher $Q$ is lower and so when $P$ is higher $P/Q$ is also higher. Hence the elasticity is higher when $P$ is higher.
The proportion of expenditure on the good is not relevant since we are doing a partial equilibrium analysis. In particular, there are no income effects from price changes. (Alternatively the linear demand curve could come from assuming a quasilinear utility function over two goods so that all the income effects fall on the other good.)
More generally, the budget share is relevant for the price elasticity if there are income effects. The Slutksy equation in elasticity form is:
$$\varepsilon_p=\varepsilon^h_p+\varepsilon_Ib $$
where $\varepsilon_p$ is the standard price elasticity of demand for the good, $\varepsilon^h_p$ is the elasticity of the Hicksian (compensated) demand, $\varepsilon_I$ is the income elasticity and where $b$ is the budget share.
For a normal good, the income effect is positive ($\varepsilon_I>0$) and so the larger the budget share, the larger the price elasticity of demand (all else being equal). Thus your intuition is correct for the case of a good with positive income effects. For a good with no income effects ($\varepsilon_I=0$) the second term in
the equation disappears (only the substitution effect is relevant: $\varepsilon_p=\varepsilon^h_p$).
