Collecting tenant satisfaction measures solely online carries ‘heavy negative bias’

The data was collected ahead of the new TSMs, which all social landlords had to begin collecting in April. 

The 22 TSMs, finalised and published last year, form part of the Regulator of Social Housing’s new focus on consumer regulation. 

They cover five main themes: repairs, building safety, effective complaint-handling, respectful and helpful tenant engagement, and responsible neighbourhood management.

Of the total TSMs, 10 must be answered by social landlords, with the remaining 12 perception questions to be answered by tenants.

The latter must be collected through perception surveys, which generally produce lower satisfaction results than transactional surveys. 

It means many landlords may see satisfaction rates fall when the new regime comes in. 

Previous research by Housemark found that in the three years to September 2022, 60% of housing associations with 1,000 or more homes were using transactional surveys, which on average produce results 15 percentage points higher than perception surveys. 

However, on top of this, the method in which the perception surveys are carried can produce a more negative or positive result. 

Mr Cox said: “Collection method has a big impact, with online surveys in particular carrying a heavy negative survey bias, around 15 percentage points. 

“London councils who carried out their baseline survey entirely online typically reported overall satisfaction between 35% and 45%. 

“Had these councils carried out the perception survey by telephone, they would likely have received an overall satisfaction score of between 50% and 60%. 

“As such for this year’s regulatory survey, most London councils are planning to avoid online collection methods.”

He also said that online surveys are also “incredibly useful” as they can be “relatively quickly deployed, require little resource, and can provide really meaningful insight into where services are going wrong”. 

“Online is therefore a brilliant method for collecting satisfaction with transactional services such as repairs. 

“However, they are less effective for governance purposes as the inherently lower scores can lead to internal wrangling which may not necessarily be justified,” Mr Cox explained.

He added: “On the plus side, after years of declining satisfaction scores, we did notice an uptick in April from landlords who carry out monthly tracker surveys.

“Again it’s a little early to say if this will be sustained, but it could be a result of recent focus on the customer experience, alongside genuine improvements in performance around repairs in particular.”