During the Impel Down Arc it is seen that luffy already had a straw hat when Ace and Luffy both have flashbacks of the past when they were kids. Didn't Luffy only get the straw hat when he was older from Shanks?
In the timeline of One Piece you can find under "12 years ago" the explanation. Luffy met Shanks before his grandfather Garp took him away.
Red-Haired Shanks, then twenty seven years old, arrives at Luffy's hometown and decides to stay there. Luffy accidentally eats the Gomu Gomu no Mi, giving him rubber powers and an inability to swim. Higuma is eaten by a Sea King in an incident involving Luffy and Shanks. Shanks leaves Luffy's hometown after staying there for almost a year. Shortly after Shanks' departure, Luffy is taken by his grandfather Monkey D. Garp to train with Curly Dadan and meets his adopted brother Portgas D. Ace. Luffy soon forms a brotherhood with Ace and Sabo in desire to becoming pirates.