I've been listening to more Japanese music now because I'm learning the language and I added 始まりの歌 by ryokuoushoku shakai to my playlist, which translates to "Hajimari no uta", but one of their other songs, "Shout Baby" sounds familiar. I couldn't tell if it was from the show or not.
Here is Shout Baby on 緑黄色社会's channel: https://www.youtube.com/watch?v=CMvep1V6rZQ
and you can hear it is indeed the same song as the MHA S4 ED2: https://www.youtube.com/watch?v=Ei0LbNH02-Y (Funimation channel)
始まりの歌 is also on 緑黄色社会's channel here to confirm it's the same artist: https://www.youtube.com/watch?v=cjf9c6TwH6o
(始まりの歌 pretty much translates to "(the) first song" BTW)