Smith’s invisible hand concept is not related to the idea of comparing an economic system, as capitalism for instance, to other possible economic systems.
But it is an image that has become famous as it synthetizes the idea that the economic system has, in itself, the capability of self-adjustment and spontaneous coordination of economic activities, toward a ‘desirable’ state, traditionally identified as full employment of resources.
As if there were a ‘hand,’ a hidden agent, underlying the economic system and coordinating individual actions.
This was a core question of economic analysis, beginning with nineteenth century
classical economists, and Smith was the most important classical economist who tried to explain this self-adjusting feature of the economic system, whereas other classical economists as Malthus and Marx hadn’t this belief.
Smith’s invisible hand is, in a sense, a continuation of a philosophical tradition, exemplified in Mandeville’s Fable of the Bees, that states that ‘private vices’ as egoism and the pursuit of individual interest, can become ‘public virtues’ in the overall functioning of human activities.
For Smith (he wrote about it in his The Theory of Moral Sentiments) egoism was not a vice also from a private point of view, provided that it is applied to the appropriate sphere of activity, that is production and exchange:
The centrale aspect of that thesis is therefore
that egoism is not a disrupting factor of the society, but it can
become an element of order and development. […] The Wealth of
Nations represents the attempt to explain in which way the free
realization of individual forces on the economic ground gives rise to
the establishment and development of economic society.$^1$
In this sense, Smith confirms liberalism, and sees economic activity as the foundation of society, assigning to the State the function of ensuring the orderly functioning of economic activity.
In contemporary term, the issue about Smith’s invisible hand could be formulated in terms of the capability of self-adjustment of the economic system and, on the contrary, the existence coordination failures.
The contemporary literature about coordination failures considers Keynes as its most famous exponent, and there are other well-known economists who dealt with this approach, as Clower and Leijonhfvud, or economists who dealth with non-walrasian equilibria.
Many economists therefore think of depression as being a state of
coordination failure; a state in which market forces have failed to
coordinate the millions of transactors that interact daily through a
web of interconnected markets. What Smith called the ‘invisible hand,’
or Mummery and Hobson (disparagingly) called the ‘automatic machinery
of commerce,’ has not
guided them to a state in which markets clear. Instead, people are
somehow led to act at cross purposes, failing collectively to take
full advantage of potential gains from trade. As Keynes put it, the
system is not ‘self-adjusting'.$^2$
$^1$ Napoleoni C., Smith, Ricardo, Marx, Boringhieri, (1973), p. 46 ( my translation).
$^2$ Howitt P., Coordination failures, (2001), p. 1,
https://www.brown.edu/Departments/Economics/Faculty/Peter_Howitt/publication/Coordination.pdf