The credits for My Hero Academia add "plus Ultra" after the series name. Occasionally All Might shouts "plus Ultra!" as sort of a battle cry when he is about to strike a foe. From context, I guess it means "extra power" or "boost", but "plus ultra" seem so random. Is there more meaning to the two words than this? Is this just something which does not translate well from Japanese?
"Plus Ultra" is actually a Latin Phrase. It translates to "Further Beyond." In My Hero Academia, it has been adopted as the school motto of U.A. High. It's uncertain in-universe whether or not the motto was around before All Might, or if he's the one who came up with it, and his alma mater adopted his personal motto.
In-context, it is an extremely gung-ho statement that tells you to push past your limits and go further than you think you can.
It also happens to be the national motto of Spain, previously the personal motto of King Charles V. Which he adopted as a rejection of the words inscribed on the Pillars of Hercules "Non Plus Ultra" (Nothing Further Beyond) after the discovery of the Americas.
Even IRL, it carries that same sort of gung-ho "You told us there was nothing beyond this point...let me prove you wrong!" sort of sentiment.
The fact that it sounds cool to Japanese Ears by using a pair of English-adopted words that they like is bonus points. But this phrase is not, in-fact, "English" gibberish as anime/manga occasionally produces on account of their writers not being entirely fluent in English. It's a Latin phrase with real-world history.
Even in Japanese they say "Plus Ultra!". It is more a reference to when All Might saved many people despite being tired, while having a smile in his face, because he always goes beyond his limits. Plus Ultra is (as far as I concluded from Season 1) when he does something with all his power and more. When he gives his all doing something and even a bit more is when he shouts it. Going beyond his limits. The others adapted to it as their favourite slogan from their almighty hero. That's why they always shout it too. It reminds them about All Might and gives them courage to push forward or to give that little extra more that's needed. But yeah at the end of the episode it can be compared to fanboy/fangirl behaviour.