I have to intervene to say that market failure and externality are not the same thing. So I do not think it is at all correct to define market failure as
when "the production or consumption of a good or service causes additional positive or negative externalities on a third party not involved in the economic activity".
Externalities are but one example of market failure. Market failure is more properly defined as any situation in which a market, left to operate without any intervention, fails to produce the efficient (welfare-maximising) allocation.
Sources of market failure include
- Externalities: if there is a negative externality then there will tend to be too much of an activity from a social perspective—resulting in inefficiency.
- Market power: if the market is not perfectly competitive then firms will tend to increase price above marginal cost to increase their profit. This results in consumers not buying the good even though they are willing to pay more than its cost of production—which is inefficient.
- Information asymmetries: If one party in a transaction has an informational advantage over the other then s/he will try to exploit it to the counterparty's detriment. This, in turn will lead to transactions taking place where it would be efficient for them not to (or to mistrust and the failure to realise efficient transactions).
- Missing markets: sometimes efficient trades don't occur because the market simply doesn't exist. For example, there is no market to insure against the risk that an unborn child will be born disabled and requiring a lifetime of care even though many parents and their children would like such insurance (an argument often used to for the existence of state-provided social security schemes).
To address your actual questions:
"Don't all activities produce externalities"? Yes, but many of these externalities are priced. For example, if I buy an apple then you can no longer consume that apple, which is an externality. However, this does not result in a market failure because the price mechanism in a competitive market ensures that I get an apple and you don't only if I am willing to pay more for that apple than you are. So the apples go to the people who value them the most, which is the efficient thing to do. Since we are doing the efficient thing, there is no market failure.
So, when should we worry about externalities? We should check whether the net effects can cancel each other out. For example, suppose that the private benefit of some action was lower than the social benefit, but that the private cost was also lower than the social cost by exactly the same amount. Then the net effect would be that MPB=MPC at exactly the same quantity where MSB=MSC. The private individual would then take the socially optimal action and there would be no market failure. A market failure only occurs if the externality is such that MPB=MPC at a quantity different to that where MSB=MSC. Only then will the behaviour of the private individual (whose optimal action is to equalise private marginal benefit and private marginal cost) differ from that which is socially optimal.
A note on marginal benefit and cost:
When performing this kind of analysis, We typically assume that the objective is to maximise to total social welfare (green line), which is defined as the difference between the total accumulated benefit of the activity (blue line) and the total accumulated cost (red line):

The marginal social benefit is the benefit society gains if we increase consumption by one unit. In other words, the MSB is given by the slope of the TSB curve. Similarly, the MSC (defined as the extra cost bourne by society if consumption increases by one unit) is equal to the slope of the TSC curve.
Now, we observe something interesting: the total welfare curve obtains its maximum at exactly the point where the slopes of the TSB and TSC curves are equal:

In other words, welfare is maximised when MSB=MSC. This is not a coincidence for this particular graph, but rather is a far more general property.
This is actually quite intuitive. Suppose that MSB > MSC. If we increased consumption by one unit then society would get MSB units of extra benefit and MSC units of extra cost. Since MSB > MSC, this results in an increase in total social welfare. Similarly, if MSB < MSC then we could reduce consumption by one unit and society would save more in costs than it would loose in benefits. So neither $MSB>MSC$ nor $MSB<MSC$ can be consistent with maximised social welfare. Only when MSB=MSC do we find that there is no way to increase welfare by increasing or reducing consumption.