First of all your example is incorrect. This is how the process of money creation looks like on the balance sheet of a bank (see McLeay et al 2014):

So it is simply not correct that when banks create new money they just create asset loan and liability deposit with equivalent amounts.
Most examples I've read stop here and show how the bank has effectively created money from nothing
I do not know what sources you read, but careful researchers would not say that private bank can create money from nothing. That is something that only central bank can do. Some people misinterpret the current excess reserve situation as private banks being given the same privilege central bank has but that is wrong interpretation. Careful scholars, such as Rendahl & Freund (2019) still argue private banks can't do it from nothing despite of the excess reserves in the system.
Citing from their article:
From an economic viewpoint, commercial banks create private money by transforming an illiquid asset (the borrower’s future ability to repay) into a liquid one (bank deposits); they would quickly be insolvent otherwise. In addition to bank solvency representing a constraint on private money creation, banks require access to liquid reserves in order to be able to engage in money creation.
but how is that deposit considered money if it cannot be withdrawn due to lack of actual money at Bank A?
Money is not cash in circulation. Money includes even things such as treasury bills, bonds other liquid assets. Deposit account is money, by definition of money in economics.
In economics money is everything that fulfils (to a considerable degree) the following three roles:
- Medium of exchange
- Unit of account
- Store of value
The deposit account clearly can serve as a medium of exchange, people, generally will sell you stuff even if you just transfer them money.
Next, the deposit account is denominated in your local currency (e.g. if you live in Euro-zone it will show how many euros you have) which is generally agreed upon unit of account since sellers and buyers quote their prices in euros not in terms of pies or berries or something else.
Finally, you can use your deposit as a store of value. If your deposit account has some money in it then it will be valuable in future (despite inflation it will keep at least considerable amount of its original value).
Hence deposits are money by definition since they tick 1, 2 and 3 requirement reasonably well.
Does this not refute the claim that banks are not reserve constrained?
You seem to have some personal definition of being reserve constrained that does not correspond to its regular use in economics.
For bank to be reserve constrained the amount of lending they can do has to be limited by their reserves. When serious economists say banks are not reserve constrained, they do not mean to say banks can lend money without having reserves, they mean by that the system has so much excess reserves that for all practical purposes supply of reserves is unconstrained.
If bank can always get extra reserves from central bank or other private banks no question asked then it is valid to say the banks are not constrained by reserves.
Before 2008 banks typically were reserve constrained because there were almost no excess reserves (see FRED data), that is reserves in excess of what bank has to have by law (e.g. with reserve requirement of 10% for every \$100 deposit bank has to keep \$10 as a reserve and with rest can do as it pleases). Banks kept small insignificant buffer but whenever they wanted to lend more they would have to get more reserves somehow (e.g. get more deposits, inject its own capital or beg the Fed for more - which in past used to carry bit of a stigma for a bank). However, after 2008, due to changes in Fed policy, US banks started to drown with excess reserves. The situation was similar elsewhere.
When banks literally drown in so much reserves that bank can satisfy virtually any (realistic) demand for loans then the banks can be said to not be constrained by reserves. However, if the reserve requirements are in place they still have to have reserves, there is just so many of them that they are not constraining bank lending decisions.

Additionally, why is it said that commercial banks create money with loans (which create deposits, or "money" in these examples) if that money is meaningless without cash behind it?
Money is not meaningless without cash. Total cash in the US economy is now about 10% of all money in the economy (see FRED data on total cash and M2). Yet people still can go on and do commerce without that cash, just transferring money digitally from one account to another.
Even though it is impossible for everyone to withdraw the money from their deposit accounts (this is why bank panics sometimes happen and why fractional reserve system can be sometimes fragile), the digital currency in form of your account is still money equivalent to cash (except I suppose cash is more useful for black market as its harder to track).
What am I missing here?
What you are missing is that deposits do not have to be fully backed by cash reserves. If reserve requirement is 10% then even with \$100 cash economy could end up with as much as \$1000 of money. There is no need for banks having enough cash for every single person if most people just keep their money in their account and transfer them around.
Furthermore, nowadays also reserves are not just cash, but also digital accounts at the central bank. Central banks don't just store piles of cash, although they can always order more cash to be printed (in case of ECB directly in case of Fed indirectly through the treasury).