Supposedly Overhaul needs to touch an object or body with his hands to activate his quirk. But in episode #14 of the 4th season of the anime, at the begining of the episode he's seen reconstructing his own body without touching anything with his hands. Does Overhaul need to touch with his hands to activate his quirk? Or how does his quirk work?
Yes, he needs his hands to perform his Quirk.
You are probably refering to the part when he makes his arms/feet big with the help of the other bodies that he absorbed. If you watch close, you can see him performing some moves before - like clenching his fists or other hand movement - which indicates that when used on himself, he can touch any part of his body with his hands for the Quirk to activate.
That is why after the battle both his hands are cuffed and Shigaraki cuts off both to prevent him for using his powers ever again
He touched the big guy's body and turned that big guy into the matter. So, since Overhaul did that, he has a matter that he can use to repair himself with. So, he can gather matter by touching someone else's body, but as for himself, he doesn't really need to touch his own body except when he is healing himself or others.