By-election shock should be a reminder to focus on what it is actually like to live in Scotland

What lessons for public policy in Scotland can we take from the Hamilton by-election result? It appears that, in a close fight, it is the incumbent party which has been punished most. But which incumbent?

The local authority which covers Hamilton and Larkhall (South Lanarkshire) is Labour-controlled. The UK Government is obviously a Labour administration with an enormous majority. It is only Holyrood that the SNP controls. So what can we take from the fact that, in a result which doesn't exactly feel like a glowing endorsement of anyone, it is the Holyrood incumbent who is punished?

On one hand, that might be obvious enough – this is a Holyrood election to send someone to the Scottish Parliament and so issues under the remit of that parliament dominated much of the campaign. But there may be another factor which amplifies that effect.

Because this wasn't a national election, there has been much less focus on national-level issues in the campaign and certainly much of the local conversation will have revolved around locally-felt issues. That means it is an election that encouraged people to focus on the place where people live.

In the case of Hamilton, that is potentially tricky for the SNP. The most visible factors in Hamilton are to do with the town centre and public services. Hamilton is almost a case study in how to destroy your high street with an edge-of-town retail park. Hamilton town centre was once busy and thriving. It is now in a sorry mess.

Like everywhere else, getting a doctor's appointment is a chore, local authority services are threadbare and local infrastructure isn't in its healthiest state. While Labour control the local authority, a lot of this happened while the SNP was in charge, so it may not have been a focus on local politics so much as Scotland-level politics as they impact locally.

If this is correct (and there is plenty to suggest it is), there is perhaps a strong lesson that political parties should adopt. Talking to journalists, lobbyists, decision-makers and party activists is not the same as persuading the public that you're delivering a better quality of life for them. There is a very big difference between national-level statistics and how that feels in a given location.

The issues which are driving people just now are things like housing, crime, local social cohesion or the sense that things are 'going to the dogs round here', the ability to access public services, the ability to feel like you have influence over local decisions – and the broad sense that 'anyone is listening to me and my concerns'.

Bragging about having a 'record number of affordable homes' built at a national level doesn't help if a local housing market is saturated and private rental costs are soaring. A six hour wait in accident and emergency might well be an improvement on a seven hour wait, but it is not a positive experience. You can talk about how you're focussed on 'town centres first' but does it feel like that in your town centre?

Politicians have become 'managers of performance indicators'. They stand at a national level and explain how 'the data' shows that what they are doing is a success. Of course, often the response to data that shows things are not a success is to seek new data. But either way, that data may not tally with public perception.

Politicians need to get out of Holyrood and ask themselves what it actually feels like to live with their own policies. Lobbyists may provide the rhetoric to suggest that the housing situation is great, but do they produce the great, affordable houses people want to live in? A new out-of-town development may look like 'inward investment', but is it really disinvestment in towns?

In Scotland politics seems to have become a game played between commentators and politicians, the former assessing the latter from an advantaged position, the latter trying to placate the former using insider access. It all rather passes the public by – they live with whatever happens below.

This problem is getting worse every day. The SNP may like to hope that it will not face a by election near the shores of Loch Lomond any time soon given local opposition to Flamingo Land. Will activists in Galloway choose to punish the party that rejected their campaign for a National Park? What fate an SNP candidate standing in Torry given the likely theft of St Fittick’s Park?

The Scottish Government has hailed all of these decisions as data-driven and in the public interest. The public appears to feel differently. In the space between statistics and disillusionment lies a lived reality for most people. That should be the focus of politics again.