Are the two connected or just close by? In the video games it always seems like they are two separate buildings. But in the anime it seems like it takes no time to fly from one to the other.
No they are not.
Kami's Lookout is in a Geo-Stationary orbit directly above Korin's tower but are not connected, directly. The Lookout - Wikia
They used to be connected by the Power pole before Master Roshi gave it to Grandpa Gohan, who later gave it to Son Goku. Goku used it as a weapon but its primary purpose was to reach Kami's Lookout.
Here is what Wikia has to say about this. Power Pole - Wikia
The Power Pole is a magical, length-changing staff that is owned and wielded by Goku for most of the Dragon Ball series. Before Master Roshi gave the Power Pole to Grandpa Gohan (who later gave it to Goku prior to the start of Dragon Ball), it was owned by Korin. Although Goku uses it primarily as a weapon, its true purpose is to use its size-changing ability to travel from Korin Tower to Kami's Lookout.
Referring to What happened to Goku's Nyoibou and Kinto'un? for the details about Power pole in DBZ. It is seen again in the end credits of DB GT.
The Power Pole is later seen extending from Korin Tower to the Lookout several times during the course of Dragon Ball Z, and Goku is seen with it one final time in the Japanese ending credits of "Until We Meet Again", the last episode of Dragon Ball GT.