There are a lot of examples of legal operations with trade balance and capital flows. Nonetheless, I've failed to find a clear explanations of how to interpret illigal (read, criminal) foreign currency (cash) transfers in terms of Balance of payments. Is such operation capital inflow or outflow, and why is it so? How should it be reflected in real Balance of payments, if this illigal transaction is revealed by foreign state authorities?
This answer starta out with how illegal transactions are normally treated in the national accounts. Transactions that are just attempts to evade currency controls are a special case, discussed later.
If the illegal transaction was somehow measured, there would be two entries: the entry for the illegal good/service on the trade account, and the cash flow in the capital account. That is, they are interpreted in the same way as legal activities. Therefore, in response to:
Is such operation capital inflow or outflow
Currency leaving the country is a capital outflow, and a capital inflow if the currency was coming into the country.
why is it so?
To keep the published balance of payments balanced.
The problem is: how do we measure these flows? In particular, we generally have no idea where non-electronic currency (e.g., $100 dollars bills) is being held.
If illegal activities were disguised as something else ("money laundering"), the cash flows should end up in whatever accounts the activity was disguised to be.
Countries tend to only include illegal transactions in the national accounts if the statisticians believe that they have a way of tracking them. For example, Statistics Canada was able to estimate the extent of tobacco smuggling (to avoid taxes) based on the disruption in other data. The underground economy behaves differently in each country, and so attempts to measure its effects varies across countries.
To what extent illegal transactions are just a measure to evade currency controls (and not for trafficking in illegal goods) they would once again show up in the form that the transaction was registered. One typical way this is done is to adjust invoices in trade, so that the amount of money exchanged is greater than the true value of the goods. This shows up in the national accounts as an excessively large imports/exports (depending on the direction of the flow).