Two things, The Villages wasn't built yet and baseball players didn't make large amounts of money like they do now. The Yankees winter home was St. Petersburg until '61 and then the Mets took their place in '62. I use to go and watch the Yankees practice at Crescent Lake in St. Pete when I was 10 to 11 years old. The field is still there, it's now called Huggins-Stengel Field. Last time I went I stood about 10' from Mickey as he played catch on the other side of the fence.
The field is still there, 13 blocks north of Central Avenue.