There are several issues with this approach. One is that any changes in the quantity of money - including those considered 'acceptable amounts of inflation' - act to distort the price signal being communicated to all economic participants which is an extremely undesirable side effect.
The other is a little more insidious. Governments can only print cash or asset money. The vast majority, over 98% in most modern banking systems of money being used is liability money - money that is represented as deposits in the banking system. Were the government for example to print physical cash and deposit it in a commercial bank the book keeping operation would be [debit cash, credit deposit]. In almost all cases, it is the deposit money that is actually spent as the government pays salaries etc.
So if a government prints money and its banking regime relies on a framework where assets act as a regulatory control on lending (and consequent deposit creation), and there is no other regulatory control, the result is hyper-inflation. The problem is not just the money created by the government, it is the consequent multiplication that results from banks increasing their credit/money creation. (This form of regulation is generally referred to as the reserve requirement in economic literature.) Typically the resulting inflation then leads the government to print more money, and the result is a rapid spiral that quickly destroys the usefulness of the currency concerned for any economic transaction.
Most modern banking systems use a combination of reserve requirements and capital requirements, and this is why the quantitative easing interventions have so far had no inflationary effect on the economies using them. Even though the US government printed a huge amount of money for the TARP intervention, the capital controls intervened to prevent the runaway hyperinflation that would have occurred under earlier regimes. However it's still a dangerous thing to do, and the long term behaviour of banking systems that rely on basel capital controls is poorly understood.
Finally, it's worth noting that the alternative available to any government besides borrowing, is raising taxes or controlling its' expenditure. Ultimately the government's use of social resources has to be regulated by something, and keeping a more or less balanced budget isn't a bad place to start.