As regards
QUESTION 1
The concept of "inverse" is usually used for a function. Here we are dealing not just with two functions, but with two optimization problems: "Duality" is, in a sense, the analogous concept of "function inverse", for optimization problems.
If two optimization problems are dual (in a more or less rigorous sense), then the solution of the one is, in a sense, the "inverse" of the solution of the other.
But what is a "solution" in an optimization problem? When the analysis is abstract, the solution does not take on a specific numerical value, but it is a value function, i.e. a function that will give us the solution for any values of the inputs on which the solution depends.
For the Utility Maximization Problem (UMP), the value function is the Indirect Utility function, $v(\mathbf p,m)$. Abusing notation (but we gain something from this abuse) we can write
$$v(\mathbf p,m) = \bar U \implies m = v^{-1}(\mathbf p,\bar U)$$
For the Expenditure Minimization Problem (EMP), its value function is the Expenditure function, $e(\mathbf p,\bar{U})$, and
$$e(\mathbf p,\bar{U}) = m \implies \bar U = e^{-1}(\mathbf p,m)$$
Looking at the two we see that
$$v(\mathbf p,m) = e^{-1}(\mathbf p,m),\;\;\; v^{-1}(\mathbf p,\bar U) =e(\mathbf p,\bar{U})$$
So, I wouldn't think that it is correct to say that "they are inverses for a single point". Stretching the concept of "inverse", for any given set $(\mathbf p,m)$, the "inverse" Expenditure function is the the Indirect Utility function, and for any given set $(\mathbf p,\bar U)$ the "inverse" Indirect Utility function is the Expenditure function.
Note: When the concept of "inverse" is treated in the proper manner (i.e. for univariate functions), then we examine the property point-by-point to conclude that it holds.