The Rou Kaioshin says that you can only fuse once. But the way it's worded it could mean that the person you once was can't fuse with another because you won't be yourself anymore. I.E. Goku can only fuse once because afterwards he won't be Goku anymore.