How did interest rates work under the gold standard?
Overall same way as under fiat system. Supply and demand interaction on loanable funds market determined the interest rate. Also from more micro perspective there is more than one interest rate but each market has its own supply and demand of funds.
Say under the gold standard \$1=1 gold unit. And the interest rate is 10%. Does this mean if I deposit 1 dollar at the bank at the end I would end up with \$1.1 which means in essence I would actually end up with 1.1 units of gold?
Yes you would end up with that if that was the interest rate.
So would the limit to interest rates under a gold standard be defined by how much new gold could be added to the financial system at a time?
Not directly only indirectly and only nominal interest rate would be affected. Real interest rate would be independent of amount of gold, it does not depend on amount of money in the economy. Nominal interest rate would be affected through the fact that amount of gold in such economy is linked to inflation/deflation.
However, there would be no hard limit on nominal interest per se. Debt is what we owe each other, any interest payment circulates in an economy and can be used to pay more interest payments to other people. One gold coin or banknote backed x amount of gold can circulate infinite number of times through the economy.
Moreover, people can also always default on their debts even under gold standard. In addition, under gold standard central bank can at any time change the conversion rate between banknote and gold. As long as a note is explicitly backed by any amount of gold country is technically on gold standard even if the amount is extremely small as long as it is not zero.