In-universe, no reason is given apart from the fact that every time I can recall this happening they were near trees and thus a chunk of wood was convenient. The actual source is never addressed, and we never see trees falling over because they lost a chunk of themselves.
For the real-world reason...It's a Japanese trope based on the actual legends built up around historical Ninjas (probably also by said Ninjas, for whom misinformation was a great tactic). "Kawarimi" was a supposed technique possessed by the ancient Ninja that used misdirection and split-second timing to swap places with something that could easily be mistaken for the Ninja.
There may be some historical grounds for this, but not to the extent we see in Naruto. Instead, this could easily be a trick used by historical ninja to trick and mislead enemies, and build up the mythos they had constructed for themselves. As an example, a Ninja may have previously prepared a wooden dummy that was about his size and dressed it in the same clothing as he was wearing. This dummy could be hidden somewhere convenient for later use. Then, when fleeing pursuers, you head back to where you hid the dummy and in a moment where your pursuers lose track of you (either with natural obstacles or a smoke/flash explosive set off), you set the dummy in your place and go the other direction. You could even set it up to go over a cliff. While your pursuers are distracted by the dummy, you slip away. Thus was built up the legend that Ninjas could 'trade places' with things, and since wooden (or straw) dummies were convenient for those purposes, they likely got used the most.
Over time, this morphed in Japanese culture along with a lot of the other myths of the Ninja. For example, the myth of Ninja dressing in all black growing out of Kabuki Theater (Stage hands dressed in all black, and so theater goers were used to completely ignoring them...so when one of them jumped out and drew a sword, it was as if they had appeared from nowhere...allowing the theater to have 'super-stealthy Ninjas'). Over this time, the idea of the Kawarimi morphed and changed until it landed on them simply swapping places with a log. Even the particular appearance of the log is somewhat standardized in this trope: a short round chunk of wood of about the same thickness, either completely smooth or with a single 'stump' branch coming off the side.
It turns up in quite a number of places, from other anime and manga to a bunch of video games. In short, in Japanese culture, the "kawarimi log" is simply the standard thing for a Ninja to swap places with, without regard for where it came from. It's the same idea as how, in American culture, if something is blindly tossed out a window, there will be a cat down there that yowls and knocks over a bunch of stuff over...or how no one questions that saying "Well, at least it can't get any worse" will instantly make things worse...or how someone can put on a pair of glasses, and that is considered a solid disguise.
So, while there is no in-universe explanation...the out of universe explanation is simply "it's a standard narrative tool in Japanese culture."