A thought experiment
Let us for a moment suppose that all of the tractors, trucks, fertilizer plants, irrigation systems, shovels and other production equipment (capital goods) used in the US agriculture instantly disappeared and that no food could be imported from abroad. Under these hypothetical circumstances, do you think that all of the expert ballet dancers, sushi chefs, pole vaulters, computer scientists and basket weavers would continue to practice their niche talent? A more likely outcome is that they will revert to subsistence farming like everyone else. Now suppose that all tractors, trucks, fertilizer plants, etc. became twice as efficient, allowing food to be produced at even lower costs. Do you think this will result in an increase or decrease in the number of people who can make a living pursuing their niche talent?
The point is that a country with a large amount of capital goods has an easier time satisfying the basic needs of more people. This allows these people to engage in other activities. We can therefore rephrase your question in the following way:
"Is there any logic to the market economy that makes the accumulation of capital goods likely, or even inevitable?"
As an illustrative example, suppose that you and a couple of friends are stranded on a deserted island and are forced to survive by catching fish with your bare hands. If one day you happen to catch so many fish that you have enough for a week, you may decide to eat them while you construct a fishing rod. This "investment" will allow you to improve your "fish productivity". If you catch so many fish that you have more than you can eat yourself, you might trade some of them to a friend if he promises to build a hut for you. Maybe you loan out 10 fish to another friends so he can build a fishing rod of his own. In return he promises to give you 15 fish at some later date. If the project is a success you might sell off your old rod and fashion a musical instrument out of some coconuts. You can entertain your friends in exchange for fish.
Saving and investing resources in capital goods and voluntarily exchanging goods and services allows you to solve your most urgent problems and to raise your standard of living. You can get away with living from playing coconut music because valuable consumer goods are made readily available by this process.
Now suppose that I declare myself Chief of the island and mandate that fishing rods can only be owned if form 1304-b is filled out and delivered to my hut atop the volcano before summer solstice. Also, they may only be constructed by and bought from my best friend who holds an exclusive patent. Furthermore, anyone catching more than 20 fish per day is required to hand 35% of the excess over to me for my personal use. The rest will be handed to those who did not catch as many that day (no matter how hard they tried). Moreover, to protect hut-builders from exploitation, it is forbidden to pay them less than 100 fish per day to construct housing. Charging "interest" for loaning out fish is also exploitative and forbidden. To enforce these rules, some islanders who could have been engaged in production are recruited to my court and paid with part of the fish-tax I collect. Anyone caught engaging in illegal "black market activities" are put in prison for at least one month.
The example is admittedly cartoonish, but hopefully illustrates why Adam Smith might believe that an unhampered market has an easier time facilitating the build-up of capital goods than one subject to intervention.