Since you keep stressing the public good part of the argument..
The Export-Import Bank of the United States (Ex-Im Bank) is the official export credit agency of the United States federal government. Operating as a government corporation, the bank finances and insures foreign purchases of United States goods for customers unable or unwilling to accept credit risk. According to its charter, the Ex-Im Bank does not compete with private sector lenders, but rather provides financing for transactions that would otherwise not take place because commercial lenders are either unable or unwilling to accept the political or commercial risks inherent in the deal.
So the service at hand is lending to higher risk customers.
In economics, a public good is
a good that is both non-excludable and non-rivalrous in that individuals cannot be effectively excluded from use and where use by one individual does not reduce availability to others.
Lending is very much excludable. To the extent that funds of the ex-im bank are limited, it is also rivalrous. Hence no, the services of the ex-im bank are not a public good.
Second order public good
To quote you:
that might result in what you might call a Public Good i.e. improved export conditions, job growth, tax revenue,
This part is irrelevant. Either you provide a public good, or you don't. Providing a private good, that may result in benefits for society doesn't make you a special provider that warrants special benefits.
Any producer of consumption goods creates "improved export conditions, job growth, tax revenue,". That doesn't warrant the government sponsoring them, because they create something "akin to a public good".