The set need not be bounded. To see this, just take $U$ to be constant. Then the set will either be empty or equal to $\mathbb{R}_+^n$.
It is also possible that no minimum exists. Let $n=2$, $p=(0,1)$, $v=1$, and $U$ be given by $U(x)=U(x_1,x_2)=x_1\cdot x_2$. Clearly, $px=0$ is only possible if $x_1=0$, which would lead to a utility of $0$. But for every $\epsilon>0$, the bundle $(1/\epsilon,\epsilon)$ has utility $1$ and a price of $\epsilon$. So no expenditure minimizing bundle exists. Of course, there is also no expenditure minimzing bundle if there exists no $x$ at all such that $U(x)\geq v$. This can happen if $U$ is bounded above.
However, an expenditure minimizing bundle exists if it holds that prices are strictly positive and there is at least one bundle $x^*$ such that $U(x^*)\geq v$. To see this, note that minimizing expenditure on the set
$$\{x\in\mathbb{R}_+^n\mid U(x)\geq v\}$$
is equal to minimizing expenditure on the set
$$\{x\in\mathbb{R}_+^n\mid U(x)\geq v, px\leq px^*\}.$$
The latter set is closed and bounded.