Keep in mind that in a natural setting cacti don't grow in pots - they grow in great big chunks of soil called 'the ground'.
Cacti often flower well when they're stressed, and root-binding helps with this; and some species, especially epiphytes, are adapted to growing in tight crevasses and are thus more comfy in smaller pots. Many species though are naturally surrounded by more soil than we plant them in, when we put them in pots.
One of the major reasons for not oversizing a pot is that the plant might not be able to suck excess water form the soil before it dries naturally, hence leading to rot. If you're careful with the amount and frequency of water that you supply to a plant, this should not be too much of an issue, especially for a golden barrel. And by the sounds of it you have decent soil. I guess it all depends on whether you want to bugger around with the roots, and with the hassle of repotting again.
I find these days that I am inclined to use larger pots than are conventional for terrestrial cacti, simply because my plants grow into them so quickly. With my epis I am a little more restrictive, but even then I prefer to let the plant grow into its pot and then settle in for the squeeze as it increases in size.
I wouldn't worry too much about it. Just be a little careful not to overwater, and it should be fine. And if it's a terracotta pot, your 'problem' - such as it is - is lessened just that little bit more.