Take S&P for example. In the long run it just keeps climbing. Same for the market as a whole. Why would this happen? Why the market keeps going up (in the long term) all the time?
Roughly, people produce "stuff" by spending their time working (labour) and by employing machinery/tools/land/etc. (capital).
If you have more workers or more machinery then it should not come as a surprise that more goods and services get produced.
But this is only part of the story. Even if we divide by the number of people to get the size of the economy per-capita, we still find growth:
So we clearly can't fully attribute growth to an increase in the population of workers.
For some time in the early 19th century, economists thought that growth was due to an increase in the use of machinery during the industrial revolution. If each worker has more tools—went the reasoning—then each worker can produce more and per-capita output will increase. But in 1957, Robert Solow found that only 12.5% of US growth in the first half of the 20th century was due to an increase in the capital / labour ratio. That still leaves 87.5% of growth unexplained.
If growth is not due to an increase in the availability of labour and capital, then it must be driven by the existing capital and labour becoming more productive. Why might that happen?
Education levels increase over time so that an average worker in 2000 produces more value than did an averager worker in 1900. Indeed, in 1900 the average worker was a farm hand or low-level industrial worker who produced a few thousand dollars worth of goods each year. Now, the average worker is a skilled industrial worker or office worker who produces good or services worth tens of thousands of dollars per-year.
Similarly, technological advancement makes tools and machinery more productive over time. For example, a \$200 modern computer is several orders of magnitude more powerful than a \$1million computer was in 1970. That means that we can now produce anything created by (or with the aid of) a computer in quantities many orders of magnitude larger than was possible 40 years ago.
So, in sum, more skilled workers who have access to better tools produce more stuff. Since a country's GDP is just the value of the stuff it produces in a year, education and technology cause GDP to increase over time.
The Unit of Measurement keeps growing.
Prices are measured using money, and the total quantity of money in the economy is continuously growing due to the operation of the banking system, as you can see here for the USA:
As a consequence, so are any and all statistics that rely on any form of monetary derived data, such as share prices.
Stock indexes not only make no correction for this, but also suffer from "survivor bias" as less successful companies are removed from the index. Other economic statistics make a partial but inadequate correction by factoring in the inflation rate - however, this doesn't account for changes in production which also influence price.
The correct way to handle price data mathematically would be to normalise it for the growth in the money supply, which would at least remove the 'known' unknowns. If you do that however, a lot of economic statistics start looking very different.
On a personal finance note, it can be useful to know the monetary expansion rate for whatever country you're living in (it varies), since it makes it much easier to make sense of comparisons of price related data between different periods.