There is a mistake in your premise that this inflation is supply driven which is widely understood to be false by economists already for some time. Initially there was some controversy about this (e.g. the Krugman vs Summers debate), but you would be hard pressed to find any mainstream economist now calming that inflation is just supply driven.
Especially in UK current inflation is predominantly demand driven. According to OECD estimates more than half of current UK inflation is purely demand driven (almost 4.5%), whereas about 1% is ambiguous and only about 3% can be attributed to supply side factors.

Moreover, there are other issues with your premises. You seem to claim that central bank cannot do absolutely anything about supply side inflation, but that is simply not true. Although central banks have much less control when it comes to supply side inflation because of course they cannot 'fix' supply shocks or somehow manage natural level output, they can still manage inflation expectations that determine inflation even in presence of supply shocks. For example if you take basic Philips curve given by;
$$\pi_t = \pi^* -\beta(u_t-u_n) + \epsilon_t$$
where, $\pi_t$ is current inflation, $\pi^*$ inflation expectation, $u_t$ current unemployment and $\epsilon$ vector of shocks, then even in presence of negative supply shocks that CB cannot do anything about inflation can be lowered by lowering inflation expectations $\pi^*$. Hence to an extent that people will expect lower inflation when central bank increases interest rate it could still help at least a bit even if it couldn't bring down inflation in line with CB's mandate.
So to sum up, BoE likely increased its interest rate because:
- more than half of current inflation in UK is demand driven. In principle, sufficient interest rate hike could lower current >8% inflation by about 4.5%. Of course BoE might not want to go pedal to the metal on hiking the interest rates for other reasons, but in principle this is how much inflation could be for sure reduced by sufficiently aggressive policy.
- While BoE cannot fix issues causing about 3% of current inflation they could still possibly bring inflation at least little bit further down by lowering inflation expectations.
- 1% of inflation is ambiguous, in case this 1% in reality relies mostly on demand factors higher interest rate can bring inflation down even further.
Taking the three points above one could reasonably argue that BoE could in worst case scenario cut the current inflation in half and in best case scenario bring the inflation down to something around 2.5% which would be quite close to its target.