Growth as is meant here "must" be nothing in particular. It is a specific metric, the percentage change in yearly GNP/GDP, and it is what it is.
In Blanchard and Fischer 's "Lectures on Macroeconomics", in the introductory chapter 1, page 2, Figure 1.1, the logarithm of USA GNP 1874-1986 is graphed: and it is impressively linear , bar a disturbance around World-War II (a dive before it that was roughly equally compensated immediately after). But this means that
$$\ln Y \approx at \Rightarrow Y \approx e^{at}$$
(for the US Economy, $a \approx 0.030\;\; \text{to} \;\;0.037$ for the period).
It is the data that told us that "growth was exponential" during this period.
(Note that "exponential growth" usually includes the concept of constant growth rate, while in informal language, "exponential" may also refer to exploding paths, paths with increasing growth rate).
And so economic models were deemed relevant if they could replicate to a respectable degree the observed data.
The question "can this go on forever?" is an altogether different issue, starting with the meaning of the word "forever".