How can a free market be operated in a way that significant social and environmental costs are intrinsically covered?

Question

More precisely, is there any practical way to operate a free market so that external costs, namely negative social and environmental impacts, become naturally (intrinsically) covered (paid for or mitigated) by the market itself?

History suggests that capitalism inherently incentives production processes which externalize as much of the cost of production as possible.

This economic tendency to externalize costs wasn't a big problem in a world of seemingly infinite resources. More recently, however, globalization appears to both intensify and conceal the externalization tendency, as large international corporations seek to increase profits and/or stockholder earnings through any available means, including unethical and/or illegal means which remain concealed by the sheer size and complexity of the global marketplace.

[The next two paragraphs are probably not necessary for the question, and may be inflammatory.]

But the damage being done by externalized costs is also global, and is apparent to us all. We call them "wicked problems". We are becoming like the laboratory rats that, under conditions of severe crowding, go crazy and start eating their young. For all of its incredible efficiency and historical success -- creating a huge increase in global wealth -- free market capitalism in its current form is like a cancer, sacrificing the host in its pursuit of unlimited growth, both environmentally (e.g. global warming) and socially (e.g. the middle class shrinks while the financial sector expands).

There is hope, and there is still time. Perhaps all that is needed is a tweak to our economic system. At the local level externalized costs can be identified, controlled and mitigated by government regulation. But at the national level, regulatory mechanisms are expensive and inefficient, and regulators will always be playing catch up to corporate ingenuity. The regulatory challenge is even greater at the global level.

It would be far more efficient and sustainable if the tendency to externalize costs were managed by market mechanisms that, ideally, were built into the economic system, rather than imposed from the outside.

Answer 1

As far as we can tell, no, there is no way to do it using only a free market.

Coase postulated that within a free market, agents would negotiate private contracts to internalise the externalities.

In reality, this didn't happen. It turned out that humans aren't perfectly informed hyper-rational beings. And as Coase himself had inferred, the transaction costs are typically too high.

There are well-documented ways in which markets have been adjusted for externalities: sometimes taxes or subsidies are sufficient; other times, prescription or proscription has been necessary.

There are even well-documented ways in which the commons have been protected, meaning that the destruction of them is no inevitable tragedy: Elinor Ostrom won the Nobel Economics Prize for her work showing how empowered stewardship by a group of custodians had repeatedly protected the custodians' commons.

So the relative efficiency of market mechanism versus planned mechanism is nowhere near clear enough to claim that "It would be far more efficient and sustainable if the tendency to externalize costs were managed by market mechanisms ... ". Each of the alternatives has inherent inefficiences: planned mechanisms have dead-weight costs. market-mechanisms have transaction costs and failures due to imperfect information and bounded rationality. There's a lot of evidence, particularly when it comes to the wicked problems you describe, that the market inefficiencies are far worse than the planning inefficiencies.

Answer 2

We could create new institutions, combine them with free market and maybe account better for social and ecological costs. For example, when every commodity gets a CO2 impact label and every person will have an personal budget of CO2 tickets, which is limited to a sustainable level, a new market for such tickets is created. To put such an label on every commodity and to account the personal budgets are costly tasks, but it would not be impossible today. Informing customers about environmental impact is a strong signal for an ecology conscious business and could make this idea interesting for trading companies, even if it will limit the consumption.
Another way already exists are rating agencies for goods and services. It is realized as an smart phone app. A customer scans a bar code and augment the product packaging with information about negative or positive effects caused by producing the product.
These are only two simple ideas, which both partly exists today.
Processes like these will lead to more conscious producers. Consumers alone can't change the economic world, but we can help to make it more social and environmental friendly.