It was also really strange for me at first. I asked from many people who might happen to know the answer. However, they were all not 100% satisfactory. I think, Kenny LJ's answer is 99.5% correct but i feel that it's still missing something. My explanation is to add to his/her answer something i think very very subtle, but hopefully important, one.
Think you have an organization and record the purchase in your balance sheet. You will purchase the good as an asset with an debit entry. Also, you will credit your cash with the same amount.
When these two entries come to the BOP, the asset will be in CA and the cash will be in FA. Why the money should be recorded in FA? When a country starts to hold a foreign currency /it was well before and independently of your purchase/, it's recorded in BOP as an investment to the foreign country. Now when you pay with the money of the foreign, you stop your investment and make it a physical good. So your stake with the other country doesn't change, but its form. So the balance is not affected in the end.
If you work with other examples, it'll become clearer and clearer. Let me give some.
Imagine you export something to a foreign country. How will it be 0 in BOP? An export debits your goods and increases the CA. If you get a foreign currency from your foreign partner for the payment, it'll be credited as investment to the other country as you hold its currency which will credit or decrease the FA.
Imagine you borrow from a foreign country, it'll be an increase in FA as it's a foreign investment to your country. But in which currency will you hold your investment proceedings? You may start holding that country's currency. This will be recorded as your country's other investment to the other country which will be decrease in FA.
If you sell above investment money into your country's central bank and your central bank holds it as cash, your country's other investment's increase will be undone which will result in positive FA from the investment /it'll look like the foreign country invested positive amount in net/. But the official reserve account of the country will increase with the same amount which will also make the BOP 0.
I hope it helps.