In Dragon Ball Super in the battle of gods saga when Vegeta fights Beerus, Beerus says fighting him was more fun than fighting Goku super saiyan 3. In the movie Battle of gods, master Roshi says Vegeta's power has finally surpassed Goku, and in the manga Vegeta could power up super saiyan 2 to a level above super saiyan 3. So everything indicates Vegeta enraged SSJ2 was able to surpass the power of a SSJ3. But what about Gohan's mystic form? Did Vegeta SSJ2 against Beerus surpassed Gohan's mystic form?
Well, Gohan was the strongest unfused character at the end of the Dragon Ball Z saga. Also, 6 months after the Buu saga, the memories of Majin Buu was erased using the dragon balls. The Battle of Gods arc was 4 years after. We know there was a massive time jump because Gohan and Videl were just classmates at the end of the Buu saga but Gohan was married with Videl and also was a (Going to be father), at the start of Super. Let us assume the mystic gohan multiplier is an immense multiplier > than the SSJ3 multiplier. Base Gohan wouldn't have trained during the 4 year period up until the battle of gods and obviously Goku and Vegeta would've trained a lot during that period. So I'd say at that point in time, SSJ3 Goku was possibly as strong as Mystic Gohan or at least stronger than Super Buu and almost as strong as Mystic Gohan. Vegeta on the other hand, might've been as strong as Kid Buu in his SSJ2 form or maybe slightly stronger. Unfortunately, when Gotenks fought Beerus, he was in his base form so we cannot use Beerus fight with Gotenks and Vegeta's initial fight with Beerus to gauge Vegeta's power level. I believe during the battle of gods arc, Ultimate Gohan (Dragon Ball Z) > SSJ3 Goku > Super Buu > Vegeta SSJ2 > Ultimate Gohan (Dragon Ball Super)> Kid Buu. I believe Gohan became didn't train and was a lot weaker than he was in Z.
This brings us back to your question as to whether Vegeta's rage SSJ2 transformation surpassed Gohan's mystic form in Z and my answer to the same would be a yes! I believe SSJ3 Goku wasn't as far behind Mystic Gohan(Dragon Ball Z) during the battle of gods arc. Goku definitely wouldn't go four years without trying to surpass or at least approach the power level of someone stronger. So it's very likely SSJ3 Goku might've even been stronger than Mystic Gohan , however for argument sake lets say he was slightly weaker. Vegeta in his rage transformation was a lot stronger than Goku in his SSJ3 form and I'd say very likely surpassed Gohan's Mystic form in Dragon Ball Z.
Note: The list I made with the power difference was with the worst case scenarios. Vegeta and Goku might be a lot stronger than where I placed them on the list. It might even be possible that SSJ3 Goku was at the level of Mystic Gohan during the battle of gods arc and Vegeta might've been on the same level as Super Buu or possibly even stronger. Vegeta and Goku would've definitely gotten a Zenkai boost after their fight with Kid Buu and the two of them would've obviously trained and gotten a lot stronger post Buu saga.
Depending on what you mean, there is no way to know for sure.
There are two difference interpretations to your question. However, no matter what, there are a lot of assumptions we must make. Indeed, Dragonball Super often ignores consistent power scaling.
Comparing Vegeta vs Mystic Gohan at his prime
Feat-wise, we know that Mystic Gohan at his prime was significantly stronger than Gotenks SSJ3, and Super Buu absorbed. However, he was not stronger than Super Buu with Gotenks absorbed. Gotenks is roughly equal in power to Super Buu.
Super Saiyan 3 Goku was, according to him, stronger than Fat buu, and even possibly Kid Buu.
The relative strength of Super Buu and Kid Buu is unknown. However, it is likely that Super Buu absorbed is roughly on the same level as Kid Buu if not a bit stronger.
This means that Goku in Buu Saga is roughly as powerful as Super Buu.
If we assume that absorption increases Super Buu's power by the absorbee's power, then Super Buu with Gotenks absorbed is roughly twice as powerful as he originally was.
This means that Mystic Gohan was less than twice Super Buu's Power, and thus less than twice the power of Goku.
Now, The time between Buu Saga and Battle of Gods was 4 years. The relevant question is whether or not 4 years of training made Goku twice as strong. It is like that it did, and as such, it is likely that Vegeta did exceed Mystic Gohan at his prime.
Comparing Vegeta vs Mystic Gohan at battle of God
Considering that Gohan was extremely weak in Resurrection F due to lack of training, and that Resurrection F was 1 year after Battle of Gods, while Battle of Gods was 4 years after Buu saga, it is likely that Gohan at Battle of Gods was way weaker than even Buu saga goku.
As such, it is very likely that Vegeta was stronger.
Pretty true, but in the ToP saga, Piccolo said in the manga "he's managed to grow stronger than ever even during this fight", which would make him way above Vegeta's SSJ2 in the Z anime if you were comparing Z to ToP.
1Please do not plug your YouTube channels on this website.– F1KrazyJan 15, 2021 at 13:04