First, that is only a rule of thumb, exact number of clusters you need in any case is context dependent.
Second, actually as pointed out by Cameron et al (2008):
A practical limitation of inference with cluster-robust
standard errors is that the asymptotic justification assumes
that the number of clusters goes to infinity.
Consequently, clustered standard errors only provide correct inference asymptotically. However, if you ever took any statistics class you might already know that the sufficiently large samples for asymptotic results to hold are typically samples with sample size of 30-40 observations. For example, for standard regression a typically a sufficient sample size to justify its asymptotic properties is 30 observations per independent regressors (see Verbeek A Guide to Modern Econometrics pp 36).
For a clustered errors a rule of thumb is 30-40 (clusters i.e. groups don't confuse this with observation count) although most authors go with the upper value of 40 or even Angrist and Pischke went for 42 (see MHE pp 319*). Now the rule is arbitrary in the sense that of course every problem is unique and sometimes 29 or 31 clusters might be sufficient and sometimes you might need even 45 clusters (the other answer is correct in pointing out more is always better).
However, the number is not arbitrary in a sense its completely made up. The number of 30-40 cluster is actually based on examination of how different errors affect statistical inferences from Monte-Carlo simulations such as the one below (table taken from Cameron et al 2008).
You can see that as number of clusters increases the rejection rates (and their standard deviation) drops. Already with 30 clusters you can see that clustering errors are doing pretty well but depending on a setting more clusters might be needed still.
Minimum of 40 clusters is very reasonable rule of thumb for an applied econometrician. If you would want to get to exact number you would have to do some Monte-Carlo simulations for your problem and most likely you would find that the result is very close to the rule of thumb. Consequently, if you are willing to tolerate some (very) small possibility of having incorrect inference minimum of 40 clusters is reasonable rule of thumb (again more is of course even better). If you are willing to tolerate larger but still small possibility of having incorrect inference you can even use 30 as a rule of thumb.
* Althought that was partially for the lolz since they constantly reference Douglas Adams The Hitchhiker's Guide to the Galaxy