We wrestled with the same question, and that led us to the decision to be snowbirds. We haven't moved anywhere yet, but have a condo up north near the grandkids (2 1/2 yrs and 10 mo), and have a house in TV.
We would not have purchased in TV if I couldn't assure my wife that anytime we missed the little ones so terribly we could just go north and I am pleased with the idea that any time we get sick of the kids we can go south!
I agree that in a few years the little ones will barely say "Hi Grandpa" as they bolt thru the house to play with their friends, but in the meantime they are very enjoyable! Once they've grown, I'd bet we'll become full timers in TV.