I sent this (long) post to The Villages Media on 9/4/2012. I also participated in a small group where the survey was discussed and we were told that USF would publish results, etc. To date, I am not aware of any peer reviewed, journal articles from USF or any other credible entity. So, I believe that my original concerns are valid.
"The design, analysis and evaluation of results of these surveys leave a lot to be desired. The Health survey had a sub-tagline as "America's Healthiest Hometown" in the caption when it was distributed, before the data was collected and results compiled. This a priori assumption seems to have been a significant factor in the survey construction and subsequent analysis, evaluation, and reporting of results.
Any competent student of survey methodology can easily point to issues that call the reported results into question. Both surveys are biased, as respondents self-selected their participation. The sample is biased. The published reports mix the number of residents with the number of respondents with erroneous or miss-leading results. Although the actual number of respondents to the Resident survey has not been given (to the best of my knowledge), one reported result was that 100% of Residents visited the town squares weekly. Perhaps 100% of the Respondents so reported, but it is difficult to believe that 100% of the 89,000 residents do so. Poor analysis, poor reporting, or both?
The September issue of the Villages Voice boldly proclaims "Villagers Happiest And Healthiest in U.S." Based upon what metrics, one asks? The first sentence points to the answer- Villagers have rated themselves so! Based upon another self-selected sample, with its inherent biases. So, it must to factually true, right? And, what metrics of health were compared, with other populations, before the conclusion was reached and announced? Surely something besides the answers to Questions 21 and 64 in the survey were used to reach this conclusion!
I have been a resident in The Villages for over 9 years. I believe that I have been a respondent to each annual Resident survey. Each year I have anticipated some improvement in the methodology, analysis, and reporting. And each year I have been more disappointed. Responses to open-end questions seem to be ignored. Statistics about participation and responses to the actual questions are not provided. Selected (favorable?) information is presented in dribs and drabs, and negative results (there must be some!) are noticeably absent.
Unfortunately, some of these same issues seem to be emerging from the USF/Villages Health survey. Results from a biased sample of some 33,00 are being extrapolated to the entire 89,000 population. No statement of quantitative or qualitative metrics (there are a lot of healthiness metrics that could be used) is given; no specifics are given about comparable studies for comparison; and selected results are being dribbled out, as opposed to publication of total results, complete with data sets and methodology. One mark of a competent study is the ability for it to be replicated by others. Maybe even suitable for publication in professional journals. Thus far, this study is woefully lacking in so many ways. In fact, I am surprised and dismayed that USF links its name to such a pedestrian approach. Basic reliability and credibility questions need answers I do not see enough results to support the a priori assumption that Villagers are happiest and healthiest in the U.S.
Yours for reliable and credible studies, improved methodology and evaluation, and full/complete disclosure of all results (at least via Internet) to all participants and other interested parties."
BTW, I believe that a diligent search will find that TV applied for a FL a trade/sales mark or something similar about the time the survey was launched. I have not idea as to its status.
|