It is stated that saving equals investment. However, suppose the interest rate rises and people save more. However, businesses will borrow less so investment decreases. Hence investment is less than saving now. What happened?
Kenny LJ and FooBar already did a great job of clarifying the broader conceptual issue here: in equilibrium, prices don't just increase or decrease without a cause (a shift in supply or demand), and any deviation from the equilibrium price will result in a surplus or deficit.
But since you're talking about interest rates, I assume that you're interested in what happens when a central bank changes the interest rate. This is legitimately very confusing, because at first glance it doesn't seem like the central bank is affecting either the supply of savings or the demand for investment - and hence, by changing the interest rate, it seems like the central bank will break the equality of savings and investment.
This isn't true. But it took me years to wrap my head around this, and I suspect that the vast majority of economists would get it wrong if asked to explain it (as evinced, perhaps, that someone as smart as FooBar misfired a little on this point). Indeed, John Taylor - an extremely well-known macroeconomist - recently fell victim to a fallacy here, as Miles Kimball pointed out (see point 7 here; as further proof of the difficulty of this topic, I'm not sure that Kimball got it quite right either!).
To clarify, let's split into two cases.
Case 1: flexible prices.
Note that what really matters for savings and investment is the real interest rate, which is the nominal interest rate minus expected inflation, while the central bank sets the nominal interest rate. Suppose that the central bank increases interest rates, and at first suppose that inflation expectations remain constant. Then as you point out, desired savings will increase while desired investment falls.
This means that there will be a glut in the economy: there will be more goods supplied than demanded. How is this resolved? In a world where prices are flexible, the price will fall until inflation expectations rise exactly enough to offset the central bank's increase in the nominal interest rate, leaving the real interest rate ultimately unchanged (so that desired savings and investment balance again). For instance, if we have a certain expectation of where prices will be next year, then when prices fall today, that fall creates more expected inflation over the next year.
This is the neoclassical, flexible price case, and it resolves your paradox in the simplest possible way - by saying that expected inflation will adjust such that real interest rates, and hence the desire to save and invest, are unchanged. But as you can probably guess, this is fairly unrealistic: prices don't always adjust that quickly, and it's counterfactual to say that higher nominal interest rates result in an equal rise in expected inflation.
Case 2: sticky prices.
Let's suppose that prices are "sticky", meaning that they don't change in response to events very quickly. This is the kind of assumption made in "New Keynesian" models (and, to some extent, "Old Keynesian" and monetarist models before them).
Indeed, for our purposes, let's suppose that prices are so sticky that expected inflation is a constant, so that a change in the nominal interest rate translates exactly one-for-one to a change in the real interest rate. (This is not a bad assumption empirically, according to Nakamura and Steinsson!) This means that we can't use the dodge from before: when the central bank pushes up the nominal interest rate, the real interest rate that matters for the savings/investment market rises too, and it seems like we should get some kind of disequilibrium.
Indeed, if not all sides of the market react right away, something like this might happen. Suppose that in response to an interest rate hike, consumers quickly increase their savings, cutting back consumption, but producers are slow to respond and keep producing the same amount of goods. (Let's ignore the response of desired investment to interest rates, which complicates the story but doesn't change the basic lessons.)
Then there will be a surplus of unsold goods, which will accumulate as inventories. This inventory accumulation is a form of investment (e.g. line 14 in the US national accounts here), so $S=I$ will continue to hold - but it will hold in a weird way, where consumers' saving is matched by producers' unintended inventory accumulation.
Alternatively, suppose that producers react right away. Then when consumers try to save more and consume less, producers will produce less, and there will be less income earned by workers and capitalists. How will consumers deal with this fall in income? Partly, they'll save less - because it makes less sense to save when your earnings are low relative to their usual level. Partly, though, they'll consume even less - which means that producers make less and incomes fall again, and so on and so on.
This is just the Keynesian multiplier, and the process ends when income $Y$ has fallen by $\Delta Y = \Delta S/MPS$, where $\Delta S$ is the initial increase in desired savings and $MPS$ is the marginal propensity to save out of income. At the point, the decrease $MPS\times \Delta Y$ in desired savings from the $\Delta Y$ fall in income precisely offsets the initial increase $\Delta S$ in desired savings arising from the higher interest rate. (If there is an initial fall $\Delta I$ in investment in response to the higher interest rate too, then this becomes $\Delta Y = (\Delta S - \Delta I)/MPS$; and it becomes more complicated still if investment changes in response to income, etc..)
In short, $S=I$ continues to hold because desired savings depends on income - which you can't see from the simple, partial equilibrium diagram showing S and I relative to the interest rate. Consumption (and therefore income) will eventually fall enough to cause a decline in savings that restores equilibrium.
Are prices exogenous?
You assumed an exogenous shift in the interest rate. Typically, prices are endogenous in Macroeconomics (and the interest rate is nothing but the price of capital). Absent frictions, prices adjust such that supply equals demand.
That is exactly why your thought experiment doesn't make much sense in the very stylized model: Suppose we somehow raise the interest rate, then supply does not equal demand anymore (unless there is storage, see below). Hence, the Walrasian auctioneer will decrease the interest rate until supply equals demand again - the original point.
Can there be exogenous shocks to prices?
In a more sophisticated world, exogenous factors such as monetary policy affect the interest rate. Why? Well, because actually savings needn't be investment. For example, if households just hold cash at home and do not spend it, that is savings (defined as income minus consumption), but it is not invested. Similar stories hold for banks that hold cash (or store their money at their central bank).