I just rewatched the beginning and it makes is surprising clear. At that time everyone knows who Zoro is, it is implied Zoro could kick Luffy's ass, and everyone looks down on Luffy until they get confused by his devil fruit power.
Zoro is a demon.
Zoro is however a very honorable demon. He signs (figuratively) a contract to torture himself to protect someone he views as innocent from someone he views as completely dishonorable. He follows that contract twice: once to honor the efforts of the girl he was protecting after the vermin mistreated her; once when the vermin clearly broke their contract.
He, while powerless, signs a contract with Luffy. As long as Luffy frees him and supports (or more accurately does not get in the way of), Zoro's dream to become the worlds greatest swordsman, Zoro will be part of Luffy's crew. Even when Zoro finds out that Luffy doesn't really have a crew at all, he shows a ting of apprehension but no willingness to break his contract.
As time continues, Zoro increasingly grows to respect Luffy above and beyond the requirements of his contract. He will not leave Luffy, however, unless his contract is broken. This would have likely occured if Luffy was not strong enough to proceed into the New World and provide Zoro the opponents he needed.
I guess when you make a contract with a straight shooting demon, expect him to follow through in incredible ways.