Regarding Forecasts in General
Economic forecasts as any forecasts can be sometimes wrong, however that does not mean they are always wrong and many times people just don’t really understand how to properly evaluate if forecast was wrong. For example if mean forecast of GDP growth is 4% but 95% confidence interval is 2%-6% you cannot say forecast is wrong unless it’s outside the bounds and unless further forecasts from the model give wrong predictions significantly more than 5% of the time. Normally in literature to evaluate forecasts people compute some forecast accuracy metric like root mean squared error (or even more fancier ones) which check how the actual forecasts performed against real observations and while still imperfect many forecasting models are good enough so that you can reasonably take them in account when making some (policy) decisions.
Furthermore, also many people confuse forecasts with predictions. For example, even if we wouldn’t be able to forecast how UK GDP changes after brexit we can make predictions based on our understanding of the economy. For example, based of our understanding of international trade a country seceding from a trade arrangement and not replacing it with a better one will experience some negative shock - we need forecasting to estimate the size of the shock, but just to predict there will be a shock requires just knowledge of general economic models and their implications.
Regarding Brexit Forecasts
Forecasting effect of something like brexit is very difficult because effects of brexit will depend on many contingencies like UK’s post brexit trade arrangements. If post brexit UK will become some beacon of free trade and somehow would manage to get the same access to EU market as it had before then the impact would be of course completely different than if UK will become some protectionist island.
Hence in case of brexit you get different forecasts depending on what assumptions you are willing to make about UK post brexit trade arrangements as well as its macroeconomic and monetary policy which can try to accommodate brexit shocks more or less successfully.
Because of that, adding to the uncertainty, most serious organizations or scholars making forecasts for brexit are working with different scenarios. For example, OECD which is well respected and non-partisan world organization estimated that depending on various scenarios brexit would in terms of GDP cost UK about 3% in short term and in long term between 3-7% (see here) However, those are just mean estimates taking some baseline scenarios. For a nice overview of brexit predictions by different organizations as well as further explanation why it’s hard to forecast effects of brexit you can look at this article from the economist.
However, the problem is that you can’t really evaluate how accurate the OECD or other organizations’ brexit forecasts are in advance. To really properly evaluate their accuracy we have to wait for brexit materializing and then making post mortem analysis because it’s a one-off event. A developed country seceding from economic union is totally unprecedented in modern history.
It would be possible to provide some measures of forecasting accuracy of the parts of brexit models in their previous non-brexit use but that would very laborious task. For example, the forecasts impact of brexit on UK GDP also depends on how brexit will affect UK trade openness which according to estimation of Fournier et al. (2015) could be between 10-20%, then you have separate forecasts what would happen to labor supply etc. As you can imagine doing survey on forecasting accuracy of all these partial components would be very daunting task (hence the reason why I did not do it here even though it was part of your question). If you really want to go that deep into weeds then you can just check all the sources for those individual models cited in the reports and then try to find on google scholar some evaluation or comparison of those models in different situations. Or if you are good with R or Python you can just check some basic forecasting evaluation criteria and then laboriously estimate all models and check their out of sample performance compared to some benchmark like random walk.