The December Effect for Compliments in Aged Care

Gathering feedback from aged care consumers, representatives, visitors, and staff is key to improving the quality of care offered by aged care services. While constructive criticism is important it is also helpful to collect compliments as those can identify areas where needs are being met, provide a frame of reference for complaints, and give staff the recognition they deserve.

In this article I analyse two quality indicators collected by MOA Benchmarking members that describe the number of compliments received by aged care services. They are, “the number of compliments by care recipients, representatives, or visitors” and “the number of compliments by staff”. In particular, I investigate the phenomenon that the average number of compliments received is higher in December than the rest of the year. I refer to this as “the December effect”. The effect can be seen from a time series of the average rate of compliments per service over time in Figure 1, among MOA members. In this analysis I hope to provide an estimate of how large this effect is and confirm the result using a statistical model.

Figure 1: Compliments by care recipients, representatives, or visitors. The values corresponding to december are highlighted by a circle.

There are differences from year to year that may distract from the seasonal effect so in Figure 2 I have plotted the indicator from 2015 to 2023 with the yearly mean subtracted¹. The graph reveals a peak in 2020 around April, for compliments by staff, which may have been related to the COVID-19 pandemic. There was a peak in 2021 of similar magnitude, for compliments by staff, which occurred around September. These peaks appear to be one-off occurrences, unlike the December effect which is visible across years.

Figure 2: The monthly rate of compliments less the average for the year.

Both graphs visually confirm that there is a December effect. However, there may be significant heterogeneity in the number of compliments received by aged care services. The distribution of the rate of compliments across services is shown in Figure 3. To get an accurate estimate for the December effect it is important to model this heterogeneity. I achieve this by using a Poisson mixed effects model with random intercepts based on the service and an indicator variable for December. For compliments by care recipients, representatives or visitors, the number of bed days is included as an offset. For compliments by staff the days in the month is included as an offset². Calendar year was found significant for compliments by staff, but not for compliments by care recipients. Therefore, I included calendar year indicator variables for the model of compliments by staff, but not for the model of compliments by care recipients.

Figure 3: The distribution of the average rate of compliments by aged care service. Only services with at least 12 datapoints were included.

Is the December effect real? Clearly - complicated statistics is not required to make this observation. However, the models provide some added precision. The fitted models summarised in Table 1 tell us that December is associated with 42% [95% CI: 34%-50%] more compliments by care recipients, representatives, or visitors, than the rest of the year and 49% [95% CI: 45%-53%] more compliments by staff. The results are statistically significant at a 5% level.

Table 1: Incidence rate ratios of factors in the mixed effects model for compliments to aged care services.

model	term	IRR	95% lower	95% upper	p-value
care recipients	isDecTRUE	1.42	1.34	1.50	<0.001
staff	year2016	0.89	0.85	0.94	<0.001
staff	year2017	0.89	0.84	0.94	<0.001
staff	year2018	0.95	0.90	1.00	0.051
staff	year2019	1.16	1.10	1.22	<0.001
staff	year2020	1.35	1.29	1.43	<0.001
staff	year2021	1.47	1.39	1.54	<0.001
staff	year2022	1.41	1.34	1.49	<0.001
staff	year2023	1.48	1.40	1.56	<0.001
staff	isDecTRUE	1.49	1.45	1.53	<0.001

The model can also be applied to the industry as a whole to get an estimate of the total number of compliments received by aged care services in December compared to the rest of the year.

Based on numbers from Australian Government Department of Health (2023), there are 225,216 residential places and 2,639 residential aged care services. Using these industry numbers and the fitted model, services across the industry can expect to receive a total of 2,454 compliments from care recipients, representatives, or visitors, and 10,029 from staff in a typical, non-December, 31-day month. Meanwhile in December the industry can expect to receive a total of 3,483 compliments from care recipients, representatives, or visitors, and 14,951 from staff³. That is an additional 1,029 compliments from care recipients and 4,922 compliments from staff. These increases may be attributed to “the December effect”.

This was largely an academic exercise, but it is nonetheless encouraging to see an increase in compliments during the holiday period, a period during which it may particularly be encouraging and important to recognise efforts.

References

Australian Government Department of Health. 2023. “Stocktake Data as of June 30, 2023.” 2023. https://www.gen-agedcaredata.gov.au/resources/access-data/2023/october/stocktake-data-30-june-2023.

Footnotes

Instead of plotting \(C_{Y,M}\), the rate of compliments in year Y and month M, the graph shows \(C_{Y,M}-\frac{1}{12}\sum C_{Y,M}\) where \(Y\) is in \(\{2015,2016,\ldots,2022,2023\}\). Note that \(\frac{1}{12}\) is replaced by \(\frac{1}{10}\) when Y=2023 and with \(\frac{1}{5}\) when Y=2015, due to fewer than 12 months being available.↩︎
Technically the log(bed days) and log(days) are the offsets, respectively, since the model uses an exponential link.↩︎
The random effect for all services in the data were used with beds based on most recent month available. Fixed effects were set to days=31 and year=‘2023’. Then the result was scaled using industry bed count for compliments from care recipients and number of services in the industry for compliments from staff.↩︎