The question about using mathematics to model economic phenomena is a rather old one dating back to at least the marginal revolution (19th Century). A couple of the key figures of this era were William Stanley Jevons and Leon Walras. I don't aim to provide a complete, or terribly accurate, history, but let's just focus on Walras.
In the preface to his Elements of Pure Economics, Walras paints a picture that suggests mathematics is a necessary input (basic ingredient) of economic modelling. He says:
As for those economists who do not know any mathematics, who do not
even know what is meant by mathematics and yet have taken the stand
that mathematics cannot possibly serve to elucidate economic
principles, let them go their way repeating that "human liberty will
never allow itself to be cast into equations" or that "mathematics
ignores frictions which are everything in social science" and other
equally forceful and flowery phrases. They can never prevent the
theory of determination of prices under free competition from becoming
a mathematical theory. Hence, they will always have to face the
alternative either of steering clear of this discipline and
consequently elaborating a theory of applied economics without
recourse to a theory of pure economics or of tackling the problems of
pure economics without the necessary equipment, thus producing not
only very bad pure economics but also very bad mathematics. (Extract from preface to fourth edition).
The above quote marks the problems that arise when mathematics is not used in economic modelling.
Walras makes a further statement with regard to mathematical language, offering a critique to his predecessors. He says:
As to mathematical language, why should we persist in using everyday
language to explain things in the most cumbrous and incorrect way, as
Ricardo has often done and as John Stuart Mill does repeatedly in his
Principles of Political Economy, when these same things can be stated
far more succinctly, precisely and clearly in the language of
mathematics. (Extract from Lesson 3).
Alfred Marshall expresses a similar view with regard to mathematical language being a more efficient tool than words. He says:
The chief use of pure mathematics in economic questions seems to be in
helping a person to write down quickly, shortly and exactly, some of
his thoughts for his own use : and to make sure that he has enough,
and only enough premises for his conclusions. (Extract from preface to first edition).
However, Marshall provides a more muted opinion about mathematics being a necessary output of economic modelling:
But when a great many symbols have to be used, they become very
laborious to any one but the writer himself. And though Cournot's
genius must give a new mental activity to everyone who passes through
his hands, and mathematicians of calibre similar to his may use their
favourite weapons in clearing the way for themselves to the centre of
some of those difficult problems of economic theory, of which only the
outer fringes has yet been touched; yet it seems doubtful whether any
one spends his time well in reading lengthy translations of economic
doctrines into mathematics, that have not been made by himself. (Extract from preface to first edition).
If we now fast-forward to the present day, we see that mathematics is everywhere in economics. Thinking of doing a PhD in Economics? Well, mathematics (and statistics) is going to be served up first - no question about that.
So, what can go wrong when using mathematics in economic modelling? The main problem, I believe, is not being able to see the woods for the trees. This point is illustrated in Klemperer (2003) Using and Abusing Economic Theory. In this paper, Klemperer gives the following example:
So graduate students are taught the elegant mathematics of affiliation
and whenever, and wherever, I give a seminar about auctions in
practice, I am asked a question along the lines of 'Haven't Milgrom
and Weber shown that ascending auctions raise most revenue, so why
consider other alternatives?' This is true of seminars to academics.
It is even more true of seminars to policy-makers. Thus, although a
little knowledge of economic theory is a good thing, too much
knowledge can sometimes be a dangerous thing.
The point is concluded pointing out that too much detail (i.e. advanced mathematics) can distract from what's really important in the wider context:
In short, a little graduate education in auction theory can often
distract attention from the straightforward 'undergraduate' issues
that really matter.
Lastly, the OPs question could also be addressed from the viewpoint of econometric methodology, but I'll omit that aspect because there are numerous controversies in econometric modelling. Names to look up would be: Clive Granger, David Hendry, Edward Leamer, Lawrence Klein, Christopher Sims, to just scratch the surface!