Since the 1970's I have had close friends who reside in Santa Fe, NM, in a neighborhood to the south and east of the Plaza, not far from the state prison. When an escape occurs or a riot
New Mexico State Penitentiary riot - Wikipedia, as at Attica in NYS
Attica Prison riot - Wikipedia, they get notified to not answer the door and so on. Frankly, it has never been a major problem for them or their neighborhood which is quite upscale and costly, by the way.
As for Coleman?
United States Penitentiary, Coleman - Wikipedia
Looks like Whitey Bulger is in there!
Frankly, in The Villages, I would worry more about a sinkhole opening up under my house, my house being struck by lightning or a drunk driver running into my car or golf cart than living near Coleman.