Last week we provided Adele Ferguson with excerpts from the 2013 McCrindle Baynes Village Census with 5,000 resident respondents. We offered to discuss the results with her, which verifies the wide range of research results generated by the sector over the past 10 years. She did not contact us.
However she did write an article in the Business section of the Sydney Morning Herald/The Age on Saturday querying the research technique with the headline: “Retirement residents are happy… probably, perhaps maybe”.
In the article, she called on two unnamed ‘experts’ to review the information we provided on the 2013 McCrindle Baynes Village Census process and results.
Apparently she took our letter to two ‘experts’. One was “far from impressed” and the other “with decades of experience” queried the structure of key questions. Adele did not name either expert.
They concentrated on this question: “Would you make the same decision to move into this village again?” Their consensus was the five answer options did not have a midpoint:
One expert said “the survey combined ‘yes, definitely’, ‘yes, probably’ and ‘yes perhaps’ for a total yes of 91 per cent”.
He or she recommended “next time they might also offer a five-point scale that says ‘very satisfied’, ‘fairly satisfied’, ‘neither satisfied nor dissatisfied’, ‘fairly dissatisfied’ and ‘very dissatisfied’ or some other variation that has two positive points on the scale, two negative points, and a neutral mid-point”. (These answers don’t relate to the question by the way).
Mark McCrindle of McCrindle Research who conducted the Census provided this response:
“There has been much discussion over the last week about the 2013 Villages Census and particularly the response to the question: “Would you make the decision to move into this village again?”
As the lead researcher I was struck by the commitment and diligence of retirement village residents nationally to participate in this survey. The number of lengthy handwritten responses on many questions gave an indication of the interest respondents had in getting their views across.
The more than 5,000 completed surveys we received back on what was a significant, 46-question printed questionnaire gives another indication of the value residents placed in this opportunity to h ave their voices heard.
An important element of research is to measure not only an answer (“yes” or “no”), but the strength of response (a passive “yes possibly” is of different quality to a passionate “yes definitely”). The fact is that in total 91% selected “yes” and 9% selected “no”.
Now the 9% who would not make the same decision again to move into their village is not insignificant, and provides many opportunities for case studies, but so does the quantitatively larger 91% who, if forced to decide one way or the other, would make the same decision again.
This question also measured the strength of this response and it found that the biggest response was of those who held this decision with conviction. The response “yes definitely” was the option for 48% of all residents compared to “yes probably”, 27% and “yes perhaps”, 16%.
This question therefore allows us to set a higher threshold of “definitely” or “probably” to show that three in four residents would most likely make the same decision again.
Additionally, respondents were filtered based on how long they had been residents of their village, and the 1,012 recent movers who completed the survey, who had been living in their village less than 24 months, were even more likely than the average resident to state that they would make the same decision again (“yes definitely” 54%, “yes probably” 28%).
In recent commentary, an unnamed ‘expert’ questioned the conviction of residents in choosing “yes” but the question was devised to do exactly this – to provide varying “yes” options to measure the strength of views held.
Someone else who was consulted in recent media coverage mentioned that the language was “too vague”, yet each option was supported with a statement to ensure a clear understanding. Any discussion that patronises what the data shows to be an educated, intelligent and informed sector of our society is misplaced.
An additional comment was that equally balanced options should be provided for each question. While this twelve-page survey included numerous five-point ‘Likert Scale’ questions, for certain areas of analysis, such structures are nonsensical. In addition to “yes possibly” should an option be “no possibly”?
What does such an option even mean? And how can a “no probably” be separated from a “yes possibly”?
The arguments over question structures and answer options can continue endlessly but the main point is that robust national research was conducted of the retirement sector by an independent research team which I was proud to lead. The survey instrument itself was reviewed by a research advisory panel.
The reason that independent research is conducted is that independent researchers have no vested interest in the sector that they research. Their guiding principles therefore, whether it be in the survey design, the response analysis, or the reporting are accuracy, clarity and validity.
It would neither be proper for operators in the sector to conduct the research, as they have an interest in seeing results come in as positive as possible, nor would it be valid for journalists to run the research, as there exists for them an interest in seeing results that are as negative as possible.
Sometimes it is best to just let the numbers, and the bulk of the residents, speak for themselves.”
Mark McCrindle
Social Researcher
The 2013 McCrindle Baynes Village Census was a confidential pen and paper plus online survey. It consisted of 46 questions distributed to residents at 236 villages across Australia. 5,218 village residents responded.
23 village operators participated.
