Leading workplace analyst David Coats argued that workers in the south of England find the combination of a long commute, rising interest rates and a job they don't like contributes to their "southern discomfort".

But Coats, the former head of economics at the TUC, also argued in a Fabian lecture that workplaces have actually become more secure and better paid for the most vulnerable in his Next Decade lecture for the Fabians. He threw down a challenge to employers and government to improve workplaces.

The future of work - David Coats

The most important point that I want to make this evening is that work, and more precisely the nature or quality of work must be seen as a political issue. After all, work is where we spend a good proportion of our adult lives. What happens to us at work can affect our health, life expectancy and life chances. Work is where we find stimulation, fulfilment, friendship and even romance. How we work, when we work and how long we work affects the productivity of the economy and national prosperity. It would be rather odd, to say the least, if the centre-left had nothing to say about work.

Let me start by recording that the challenges facing the government today are very different from those we confronted in 1997.

I do not want to dwell on the past decade, beyond saying that in general, there is a foundation of success on which to build rather than a legacy of failure.

The minimum wage has had a huge impact in the battle against low pay – although there is much unfinished business.

The UK now has a much fairer labour market than was the case in 1997 – new rights for individuals, wider access to flexible working, improved protection against unfair dismissal, new rights for trade unions.

And even if David Cameron does find himself as prime minister at some point in the (distant) future, he will have to accept that he cannot leave the mainstream of EU social policy – the social chapter is here to stay unless the UK leaves the EU.

However, Labour cannot develop persuasive answers to the challenges that we face just by reminding the electorate of our achievements.

As the Swedish social democrats say, we must be proud but not satisfied. There is still more to be done.

I am not going to talk very extensively either about individual employment rights or the reform of trade union law.

Indeed, many of the problems I want to discuss do not lend themselves to a regulatory solution.

My purpose therefore is to make the following arguments.

First, social democrats should care about the quality as well as the quantity of work.

Second, the UK, on recent evidence, has a job quality problem – and this may be contributing to the southern discomfort that John Denham's recent lecture sought to explain.

Third, even though government cannot legislate for high quality jobs or high trust in the workplace, there is still a role for the state in setting framework conditions and encouraging the application of best practice.

Fourth, New Labour now needs to address questions that have been off the agenda for the last ten years – including the role of trade unions and the link between employment relations and productivity.

And finally, the centre-left must develop a new story about work that takes account of the changing conditions in which we live. We must have a "day to day" narrative that speaks directly to the problems that people face in their working lives: long working hours; the difficulties of achieving proper work life balance; relentless change in the workplace; a lack of influence over key decisions leading to disempowerment and a loss of control.

More specifically, our political task is to offer compelling answers to the following questions:

• Is it true that globalisation presents the UK with an unparalleled set of problems that could further reduce job quality?
• Is migration having a negative impact on the conditions of work for employees in the UK?
• And what does all this mean for our prosperity in the medium term? In other words, what does our narrative have to say about economic growth, job quality and productivity?

Let's begin with my first argument. Why should the nature and quality of work matter?

For social democrats work has always been a source of self-respect, which as John Rawls reminds us, is the primary good.

It is no surprise then that we have always given a high priority to full employment. We know that work is good for us, and certainly better than worklessness.

We only need to look at Thatcherism's callous disregard for policy consequences to confirm that unemployment has a corrosive effect on both physical and mental health.

Even though full employment may be taken for granted by the electorate today, we should not underestimate the significance of the government's achievement. Getting people back to work has enhanced the life chances of the least advantaged. In other words, "work first" works. It is the only sensible policy for a social democratic government to pursue.

Work helps to make us people that we are. It shapes our identities, influences our loyalties, moulds our opinions and gives purpose to our lives.

And because we spend so much time at work, what happens there can have a profound effect on relationships with our partners, our families and our friends. Satisfaction at work shapes life satisfaction more generally. An unhappy worker is likely to be a stressed parent, an anxious colleague and a troubled friend.

Atomised relationships in the workplace can have devastating social consequences. Or, to put it another way, if we care about social cohesion we should care about the quality of work.

And work is also, for most of us, our principal source of income. Hence our desire to "make work pay", implement an effective national minimum wage and eliminate gender pay inequality.

As social democrats we are keen to ensure that individuals enjoy the widest possible life chances. We want to enlarge the sphere of human freedom so that citizens can choose a life that they value.

We should be worried about "bad work" because it can be devastating to life chances in much the same way as unemployment.

Simply expressed, bad work is associated with poorer health and shorter life expectancy.

It ought to be self-evident that we value a longer over a shorter life, assuming reasonable health in old age, and a healthier over an unhealthier life.

And social democrats should be troubled by the fact that the security guard at the front desk is likely to die younger and have poorer health than the supposedly highly stressed senior manager on the executive floor – in large measure because of their experiences at work.

At this point it may help to be a little more precise and offer a definition of "bad work". I am talking about jobs characterised by:

• Employment insecurity
• Monotony and repetition
• Limited autonomy and control
• An imbalance between effort and reward
• Procedural injustice in the workplace
• Weak social capital and low trust

We should note perhaps that while these are features of jobs held by the most vulnerable workers, we can also see them amongst the better paid and protected. Indeed, the best study of the work-health relationship used a representative sample of Whitehall civil servants – hardly a group that we generally see as exploited.

My case then is that Labour can neither renew itself in government nor win the next general election unless it understands these inequalities, recognises the impact on life chances and develops positive and popular policies in response.

Indeed, the Prime Minister accepted this in one of his valedictory lectures by placing great emphasis on the importance of high quality employment.

If all this is right – and I'm happy for you to disagree with me – then we need to move on to my second argument, that the UK does have a "job quality" problem.

Until relatively recently, the evidence was mixed.

Employment insecurity fell between 1998 and 2004, according to the Workplace Employment Relations Survey.

And job satisfaction appeared to be improving too. Things really were getting better.

Some commentators, writing in the first years of this decade, offered a less positive assessment, suggesting that there had been a decline in general job satisfaction, rising dissatisfaction with pay and working time, falling autonomy and control (despite the rhetoric of empowerment), the more widespread use of intrusive performance management systems and an intensification of work.

In other words, these analysts suggested that people were working harder and seemed to be increasingly frustrated in their jobs.

A more recent study, the 2005 European Working Conditions Survey, again found a high level of apparent job satisfaction, although a closer look at the detail confirms that the UK has a serious job quality problem.

For example:

• 55% of British workers say that their jobs are boring and monotonous.
• Almost a third say that their jobs never involve learning new things.
• Barely two in five report a high degree of autonomy.
• On a composite measure of job quality the UK is below the EU average and closer to Bulgaria, Lithuania and Estonia than Sweden, the Netherlands, Austria, Denmark and Finland, all of which have much higher scores.

So, despite the apparently high levels of job satisfaction in the UK, when it comes to job quality or job content we bear a closer resemblance to countries that languished under Stalinism for forty years rather than the Nordic social democracies.

The only way to explain this is to suggest that workers are displaying adaptive behaviour. They have lowered their expectations and have accepted that work, awful as it may be, is as good as it can be.

The widespread incidence of "bad work" may be contributing to the increasing number of Incapacity Benefit claimants presenting with mild to moderate mental illness (anxiety and depression). Obviously, not all mental illness is work related, but in a world of widespread bad work there is bound to be some impact on mental health.

We might also usefully ask whether this experience is contributing to southern discomfort? Is the brute fact of employment in a "bad job" (even if it is a decently paid job) amplifying the dissatisfaction caused by a dreadful commute to work, anxiety about house prices, rising interest rates and the difficulty of getting onto the property ladder?

And what about the "permanent revolution" afflicting many organisations – the endless reorganisations, perpetual restructurings, the continued pressure to improve performance, to beat the competition, to meet rising customer expectations?

You might say that there is no congruence between the "bad jobs" I have described and the discontents of the southern middle class. I'm not so sure. Richard Sennett in his Culture of the New Capitalism suggests that there is something distinctive in the employment policies of multi-national companies operating in highly competitive international markets. These are the organisations that are supposed to be networked, flexible, non-hierarchical. But they are also organisations where the pressure to perform can be intense, where "permanent revolution" is the order of the day and where workers believe that they are victims of change, not agents of change.

And many of the companies that Sennett describes are to be found in London and the greater south east of England.

Even if government is not being explicitly blamed for bad work, we might justifiably say that what happens at work is just another ingredient in the cocktail of wider social insecurity.

People are uncertain about their place in the world, concerned about the fraying of the ties of family and friendship, worried about whether they can really rely on their colleagues in a world of weakening workplace relationships, virtual teams, episodic encounters with people who rarely become their friends. For some people work becomes a substitute for community – they put all their energies into work – whereas for others work becomes the most significant barrier to community participation – they just don't have time to get involved.

This then brings us to the question what can and should government do?

Self-evidently, high quality jobs cannot be created by legislative fiat, nor can people be compelled to trust each other by statute. Some problems are beyond the reach of regulation.

But government can create the framework conditions for an improvement in the quality of work.

Government can, for example, invest in the skills of the workforce.

If people have appropriate skills then they are more resilient and able to cope with periods of intense pressure at work.

Similarly, government can impose obligations on employers to recognise trade unions, or to inform and consult representatives of the workforce if trade unions are not present.

The aim must be to give workers more influence, perhaps even more control over work organisation, management style and culture and difficult processes of change. In other words, government can, through regulation, establish an expectation that organisational decisions must be justified and legitimised.

Some of you might say that this sounds like a plea for what used to be called industrial democracy – and you would be right.

I make no apology for this and would simply say that it cannot be right for social democrats to believe that citizens lose most of their human rights when they cross their employer's threshold.

There is a reason why freedom of association and the right to bargain collectively are to be found in a range of international human rights conventions. These are our values. We should be proud of them, not apologetic or grudging.

A government supporter might say that enough has been done already. The Employment Relations Act and the Information and Consultation regulations are entirely consistent with the principles of economic democracy, legitimacy and justification. There is some truth in that, but it is by no means the whole story and I want to return to the issue in a moment.

Government also has a role to play as a repository and promoter of best practice.

The DTI has an array of business support services all focused on improving performance. ACAS and the HSE offer advice on employment relations, stress and health at work. It is not too ambitious to think that a judicious reordering of priorities could begin to have a significant impact on the job quality problem.

Government can also support workplace innovation through pump-priming funds – like the Partnership at Work Fund and the Union Modernisation Fund – but with a specific focus perhaps on improving job quality and productivity through, for example, effective information and consultation arrangements.

Finally, government can encourage the social partners (trade unions and employers) to work together more effectively to address these problems.

The Chancellor tried to do this in 2001 when he invited the TUC and the CBI to develop a joint approach to the productivity agenda. A report was produced and then, once the political pressure was off, the parties reverted to type, with the CBI becoming increasingly resistant to the whole idea of social partnership and the unions taking refuge in calls for the repeal of the anti-union laws.

Which brings me neatly to my fourth theme, that we need to enter forbidden territory if we are to renew ourselves and make progress.

Two questions are relevant here. First, what kind of workplace institutions, if any, do we believe are needed to ensure that workers have effective voice? Or, in cruder language, what about the unions, where do they fit in New Labour's story?

Second, what is the relationship between the workplace and the government's productivity framework? Is there a gap and if so where is it?

Answering the first question will help us to make sense of both the limits of government action and the need for institutions that can carry Labour's values into the workplace. If we really want to make progress on the job quality agenda then we need employees, through collective action, to start making demands of their employers.

The problem of course, in a mostly union free private sector, is that there is no countervailing force to the power of the employer.

It seems implausible to suggest that the government should reintroduce the closed shop – partly because this is inconsistent with the EU Social Charter and partly because it is hugely (and justifiably) unpopular.

Nor is it the government's job to recruit members into trade unions.

But there are some specific things that government could do that might help to rebuild effective voice institutions. The first would be to recognise explicitly that collective bargaining and collective representation can be good for both employers and employees.

The CBI often complain about the accumulation of "red tape" and individual employment rights but they fail to recognise that simply leaving workers unprotected is electorally unacceptable. Indeed, if progress had been made voluntarily and if bad employers behaved better then trade unions would be more reluctant to reach for a regulatory solution at every opportunity.

We might say that employers and government have a clear choice, more litigation or more negotiation. My suggestion is that we follow Lord Wedderburn's principle, first articulated in the 1960s:

Most workers want nothing more of the law than that it should leave them alone…a grievance settled in the plant or office is better than going to a tribunal.

He went on to say that for a long period this was a "fundamental feature" of British industrial relations. But I'm afraid that today it is a "fundamental feature" lost in the mists of history, hence the rising tide of employment tribunal applications that flowed from the decline of effective workplace trade unionism.

The second thing the government can do is make much better use of the instruments already on the statute book. Very little effort has been made to promote the benefits of information and consultation.

The DTI's budget is minuscule and both employers and trade unions could be forgiven for believing that these regulations do not matter very much. This must change.

Government reluctance to take an interest in what happens inside the workplace can also be seen in the Treasury's productivity framework.

You will recall that five "drivers" of productivity have been identified: investment (physical capital), skills (human capital), R&D (innovation), competition policy and entrepreneurship.

Six special reports have been published since 1997 and, while some progress has been made, the UK still has a significant productivity gap with our major OECD competitors.

Let me be clear. The framework is incomplete rather than wrong or misconceived. Other Anglo-Saxon or liberal market economies have enjoyed rather more success in integrating the workplace into their productivity discussions.

This is certainly true in the Republic of Ireland, where the National Centre for Partnership plays a central role and in New Zealand, where the Department of Labour has both a Partnership Resource Centre and a workplace productivity programme. We should learn from these experiences.

Another small example may reinforce the point. Investment in skills is all very well, and the Leitch report offers an ambitious prospectus for the future. But there is little value in investing in skills unless these skills are fully utilised. Leitch recognises the problem but dispenses with it in three paragraphs.

I would argue that a more serious treatment is required, not least because between fifteen and twenty-five percent of employees report that their skills are under-utilised.

There is no alternative but to recognise that the remedy lies in improving work organisation, job design and the quality of management. Indeed, we might say that Britain's real problem today is a management problem. That is not to say that all managers are useless. My argument is slightly different. We place too little value on the people management skills of operational managers. Some organisations expect people management skills to be acquired by osmosis. And there are still far too many examples of management degenerating into supervision, surveillance, progress checking and the correction of errors. Simply put, too many British organisations are over controlled and under managed.

A sceptic might say that this is all very well, but an agenda focused on job quality has no chance of success if the face of the threat of globalisation. Jobs are about to flood abroad as corporations seek low labour costs. India and China are, in Digby Jones's words, "about to eat our lunch".

And Digby has been joined by other, more respectable voices in predicting imminent catastrophe. Alan Blinder, the eminent US economist has suggested that millions of jobs are potentially "offshoreable" with devastating consequences for the economies of the developed world.

So it is said that the UK has no choice but to be a lean, mean knowledge economy with a relentless focus on skills development and cost reduction.

And, it is said, while the UK faces low wage competition internationally, wages domestically, particularly for the lower paid, are being dragged down by rapid inward migration.

Only one part of this story is true. Our prosperity in the future will depend on our ability to export knowledge intensive goods and services. As my colleagues at The Work Foundation have shown, the growth of the knowledge economy is one of the principal reasons for the government's economic success.

But the rest of the story is a rag-bag of myths, half-truths and total nonsense.

There is little evidence that international trade with the developing world is the principal cause of change in the UK economy. Technological developments and national policy choices are much more important – as is the structure of capital markets and the ease of merger and acquisition. This is where the pressure for restructuring has come from. Not the supposed competitive threat of India and China.

Perhaps we should also reflect on the sobering fact that structural change is not new.

"Creative destruction" has been described as the fundamental dynamic of capitalism.

Of course, we have to recognise too that there are winners and losers – and that unless we pay particular attention to the position of the losers then free trade and open markets will lack legitimacy.

In other words, government should devote more attention to explaining the benefits of globalisation and offer more support to those who find themselves temporarily displaced. "Let them find jobs at Tesco" is a rather inadequate reaction to labour market turbulence.

Amartya Sen elegantly makes the case for intelligent intervention as follows:

The appropriate response has to include concerted efforts to make the form of globalisation less destructive of employment and traditional livelihood, and to achieve gradual transition. For smoothing the process of transition there have to be opportunities for retraining and acquiring new skills…in addition to providing social safety nets for those whose interests are harmed – at least in the short run – by globalising changes.

I cannot improve upon these words, but would offer the following practical observations.

Legitimising the process at global level may require a more explicit linkage between trade and labour standards in international agreements.

In domestic policy a more sophisticated anticipation of change by the regional development agencies and the skills institutions would ease the pain of restructuring.

At the level of the organisation and the workplace anticipation is important too, as is the proper involvement of the workforce through the effective information and consultation arrangements that I mentioned a moment ago.

Trade unions of course have demanded a toughening of employment law as a response to globalisation.

Unfortunately, it is not at all clear precisely what this means. On one level it could simply be a plea for proper union involvement at an earlier stage in the decision making process; if so then there are no significant objections that might be raised.

But the government would be right to resist a demand for job protection that makes necessary change almost impossible.

There is a balance to be struck between flexibility and security. If we need stronger laws then they are in the field of information and consultation rather than job protection.

This brings me to my final theme, the position of the most vulnerable workers in the UK.

Aren't we witnessing a significant downward pull because of rapid inward migration? Aren't wages and conditions deteriorating?

The simple answer, so far as the formal economy is concerned, is a resounding "no". The minimum wage has been rising rapidly – and much faster than both earnings and prices. Forty per cent of known migrant workers are paid at or close to the national minimum wage.

As a result average earnings growth seems not to have been affected by inward migration, even in those areas with a high concentration of migrant workers.

This might be contrasted with the position in the United States where rapid inward migration and a falling federal minimum wage have had a depressive effect on wages.

The policy lesson for the UK is clear. Maintain the real value of the minimum wage and ensure that it is properly enforced.

This leaves of course the question of what to do about the informal sector where anecdote suggests that many migrants are employed. In part the solution lies in enforcing the new gangmasters legislation with the utmost vigour and looking to other enforcement agencies for a more enthusiastic regime of inspection across the full range of employment rights. Government might consider increasing the resources available to make detection more likely and increase the penalties that can be imposed.

Trade unions have a role to play too in ensuring that migrant workers are aware of their rights and have the confidence to take enforcement action when necessary.

Perhaps the bigger challenge is identifying how we can make further progress in eliminating low pay now that the minimum wage has matured and cannot be pushed much higher without some negative effect on jobs.

We need to recall that the majority of minimum wage workers are women and that the largest group are working part-time. Progress will only come when we place a higher value on what have traditionally been seen as "women's jobs" and eliminate the "pay penalty" associated with part time work.

Both issues are dealt with in the report of the Women and Work Commission chaired by Margaret Prosser. Full implementation of their recommendations must be treated as a policy priority.

But this can only offer a partial solution. More ambitiously, progress depends on an improvement in the performance of employers in low wage sectors. They need to be equipped with the wherewithal to pay wages above the minimum. In other words, they must make a conscious decision to abandon "low road" business models and focus on improving productivity and performance.

Government policy can help. Public procurement can be used to promote better employment practice. Dialogue between the social partners can be encouraged at sectoral level. And the co-ordination of enforcement activities could be significantly improved.

Once again though we are dealing with the limits of a regulatory solution. The minimum wage has taken us some distance towards the elimination of low pay, but we are still too far from our destination. A more ambitious portfolio of policies is needed if we are to move forward.

This lecture series is entitled "Labour's next decade - the debates we must have" and my purpose tonight has been to make the case that job quality is a political issue that cannot be ignored by the centre-left.

Job quality affects health, life expectancy and therefore life chances. The UK has an enviable record of getting people back to work but a rather less impressive performance in creating quality jobs.

The widespread experience of "bad work" is another ingredient in the cocktail of southern discomfort, which we really cannot ignore – and the experience of "bad work" can help to explain some working class disillusionment too.

However, we would be wrong to believe that a slew of new regulation is the right response.

As I have explained, regulation has its place, but real progress depends on filling the institutional gap left by the retreat of the trade unions – either with a revived, modernised and responsible trade union movement ("New Unions" perhaps?) or with effective information and consultation bodies.

We might even say that the process of re-regulating the labour market is reaching maturity. Government must seek an effective partnership with intermediate institutions because we now confront problems that are beyond the reach of the state. Indeed, we might say that the regulatory reflex is distracting us from the real agenda. We need to get beyond a narrative that focuses exclusively on the tension between supposed burdens on business and new workers' rights.

Perhaps the most important argument that I want to leave you with this evening is that the government cannot succeed unless it begins to look inside the workplace. Unless it does so we will not improve the life chances of the least advantaged, we will continue to have a large number of low paid workers and we will fail to close the productivity gap. The workplace may have been off-limits as a political question in the last ten years, but this must change if we are to build a distinctively British social democracy.

I would suggest therefore that we have a once in a generation opportunity to establish a lasting legacy in the world of work, to transform British workplaces, improve the quality of jobs, improve business performance and increase productivity. Just as with work-life balance, government has the opportunity to put job quality at the top of the agenda.

This would be a new, ambitious and distinctive offer to the electorate, recognising that job quality matters to both workers and business. I hope that I have outlined some possibilities tonight and that the debate will continue.

Back to the Future of Work