In his first major public lecture as Leader of the House of Commons, the Rt Hon Jack Straw MP made the case for politics as the force which can bond an increasingly atomised society.

Membership of political parties has halved since 1980. Spending by political parties – in real terms - has almost trebled in the same period. Turnouts – at 84% in 1950 - now hover around the 60% level. 2001 saw the lowest turnout since 1918, 2005 the second lowest.

Trust in politicians of all parties, never particularly high, has rarely been lower – despite the fact that Parliament has become more, not less, effective in one of its principal tasks, of holding the executive to account. All parties have to recognise that whilst their market share will vary from time to time, the market overall for politics has been shrinking. No party committed to values of liberal democracy has any partisan interest in this continuing. But as we seek to raise participation, turnouts and general involvement in party politics, we must also be aware that going back to the past is impossible. The world has changed profoundly, not just since I was young, but in the less than two decades since my own children – now in their twenties- first went to school.

In this lecture, I want to set out what I think has happened; offer some reasons why; and then some proposals for the way forward.

The 'what' in terms of direct political involvement is well illustrated by those figures I gave at the opening - a halving of political parties' memberships and of a drop in turnouts. Much other data parallels these trends – including a fall in local election turnout from around 50% straight after the war to 36% on May 4 this year. Trade union membership is also down by around half in the last quarter century – from around 13 million in 1979 to less than seven million now - though it is now levelling out. Other organisations which depend on active personal involvement and collective commitment face similar, seminal declines. The Christian churches are the obvious example. Figures from the Church of England show usual Church attendance on Sundays at 1.5 million in 1970. In 2004 it was 903,000, though recent figures have also shown a slight increase in weekly and monthly attendance figures.

Here public perception of what is happening accords with the reality. But in the second area I highlighted in my introduction, that of trust in democratic political processes and their current effectiveness, the available data is going in one direction, whilst public perception would appear to be going the other way.

The question of effectiveness is bound to provoke more argument than simple indication of voter turnout and parties' membership. But let me give nine examples to back up my point. Before I do, let me make clear that I am not claiming that the accountability of the Government to Parliament has reached some state of grace, a level near perfect and incapable of improvement. Certainly not. There are continuing criticisms which we have to take on board, and clearly scope for further reform. I just say, however, that scrutiny of government is far more substantial than it was back I the mid-seventies when I was working as a special adviser.

First, select committees. Before the then Leader of the Commons Norman St John-Stevas (now Lord St John of Fawsley) introduced his reforms in 1979, there was a far lower level of select committee activity and no permanent departmental select committees, with the notable exception of the Public Accounts Committee. Today, there are 18 departmental select committees ; they have sizeable staffs; their chairs are paid. They have shown commendable independence of spirit and action, regardless of party affiliation. In 2003/04 they published 152 reports, compared with 56 from equivalent committees in 1980/81, and around 20 in 1975/76.

Second, written questions. The number of oral questions answered in Parliament is naturally constrained by the availability of time on the floor of the House, but this is not the case for written questions. And the number of those tabled has been on an inexorable rise. In 1964/65, there were 8,270, rising to 17,468 in 1971/72. By 1985/86 that had risen to 31,808. The latest figures for this session – to the end of April – show that more than 66,000 have been tabled already.

Third, the introduction of Westminster Hall as a secondary Chamber has significantly increased the opportunities for backbenchers to raise a wide range of national and local issues.

Fourth there has been the erosion of that most opaque source of power, the Royal Prerogative. For example, before the late 1980s all three of Britain's intelligence agencies used to live in a strange world where their very existence was never officially admitted ("averred" was the word) and their legal base were Royal Charters which were never published. All that was changed by the 1989 and 1994 Acts. Now there's a committee of senior parliamentarians overseeing their work, and very specific duties on the ministers responsible.

Decisions to go to war used to be taken by the executive under the Royal Prerogative, with the lightest endorsement by Parliament in the form of a debate on the adjournment of the Commons. In contrast, decisions in respect of Iraq were agreed through explicit, substantive, voteable motions. This was key to establishing the domestic legitimacy of the specific decisions on Iraq. But the process has also established a precedent for the future, making it very likely that any similar decisions about military action would be taken with a Parliamentary vote.

Fifth, has been the considerable expansion in the role of the courts in supervising decisions of the executive. Applications for judicial review were relatively rare until the 1970s. By 1996, the numbers had risen to 3830. The trend since then has been a steady rise, to 5356 in 2005. This development of "judicial activism" gathered pace in the 1980s and 1990s but was reinforced by the Human Rights Act 1998. Sixth, has been the coming in January 2005 of the Freedom of Information (FOI) Act 2000. When I was putting this Act through Parliament in 1999 and 2000 it used to be claimed that it would do no more than make statutory the old FOI code. No longer. The FOI Act opens the executive up to scrutiny in a profound way. This act is driving a sea change in the culture of Whitehall in respect of openness and transparency.

Seventh has been the progressive move to ensure that National Statistics are independent and at arms length from the government. Gordon Brown is now consulting on legislation to complete this process, and in doing so help guarantee the integrity of official statistics, the key currency of debate in all advanced democracies.

Eighth is the fact that MPs are now overwhelmingly full time, with funds to employ three constituency and research staff: whilst the opposition parties, through the provision of "Short" money – up from £1.7m in 1998/99 to £6.1m in 2005/06 - have greatly expanded their research and constituency staff, many of whom are (properly) devoted to pinning down the government on one issue after another.

In contrast, twenty five years and more ago, many MPs, on both sides, were part time. Their connection with their constituency was often remote. The story of the North West MP who in 1978 complained that his local constituency party were harassing him for not having visited the constituency for nine months is not apocryphal. The Commons Post Office told me a few years ago that in the mid sixties, MPs were receiving an average of 10-20 items a week. Since then MPs have seen an exponential rise in post – and now email as well - from their constituents.

Ninth, there would appear to be an equivalent increase in the workload of ministers as they deal with the natural pressures which flow from this higher level of accountability. The evidence is anecdotal, but nonetheless powerful. Roy Jenkins, in "Life at the Centre", recalls that when he became Home Secretary in 1965 the Permanent Secretary of the day, Sir Charles Cunningham, reduced the relatively few official submissions to the Home Secretary to "two sheets of thick blue paper, boiled down to a few hundred words of lucid explanation, concluding in a clear recommendation, and boldly initialled 'C.C.C'". He went on: "No other course was outlined, there were no background documents from which an alternative could be devised, there was no indication whether or not there had been dissenting opinions as the file had made its way up through the various ranks of the hierarchy." In contrast, as Home Secretary in the late 1990s I used to have up to two full red boxes each night.

The evidence overall is therefore pretty strong that in both qualitative or quantitative terms, ministers are now much more accountable to Parliament, and government as a whole is much more open and transparent. But here, as I said earlier, while the data goes one way, perception goes another. A study being conducted by the Constitution Unit of newspaper articles featuring information gained by use of the Freedom of Information Act has found that rather than increasing trust and confidence in government they either reduced trust and confidence or made no difference.

It is sometimes argued that confidence in politics and Parliament remains low because of changes in the Chamber of the Commons itself. True, attendance in the chamber was often higher in the past, but that was because there were fewer competing demands on MPs, such as select committees and from their constituencies, and the Chamber was not televised. But it is a fallacy to believe there was some sort of golden age in the Chamber. Churchill's biographer Martin Gilbert observed that many of Churchill's now most remembered pre-war speeches were to empty houses: Churchill himself insisted that when the Commons was rebuilt after the war it should not be big enough to accommodate every MP – otherwise "nine-tenths of its debates will be conducted in the depressing atmosphere of an almost empty or half-empty chamber".

But the chamber is still the cockpit of British politics (as we saw with the Iraq debates). One of my tasks as Leader of the House is to look at sensible ways of encouraging attendance in debates, including looking to see how the committee load might best be spread to enable this to happen.

Better attendance is desirable but it will obviously not solve the problem of declining trust and involvement in the political process.So in the second part of this speech, let me offer some thoughts on why there has been such a significant decline in formal political activity and deterioration in scores in political trust.

On turnouts, we may have been suffering from a symptom of what the late JK Galbraith called "the culture of contentment" as more people become satisfied with their living standards and lifestyles. In contrast, it is no accident that in Iraq in 2005 turnout in three polls rose and rose again, to 75% in December, notwithstanding the palpable risk of being maimed or killed in return for exercising the right to vote. For Iraqis, starved of free elections for generations, voting was an existential issue.

For sure, voting matters here in the UK. When I am faced on the doorstep faced with another non-voter, I often ask whether he or she has any complaints or suggestions for improvements in their lives. If the answer is 'yes', as it invariably is, I suggest that they can help secure what they want by voting. It's a rational argument, but a subtle one. These folk may well vote about what is of interest to them at the time, in Big Brother or the X Factor, or in a "red button" TV poll on a current issue, but that is a different, much more quiescent activity from the civic duty of walking to the polling station or completing a postal vote. The question for politicians is how to make similar, more direct if not instant, links between voting and what it can achieve as in those TV polls.

But whilst voting today matters, it used to appear to matter more. The ideological divide between Marxism-Leninism and liberal western capitalism dominated all politics in the four decades after the war. However much their programmes may have converged in practice, parties of both left and right appealed to this divide, to fear of the "other" and to class loyalties. The choice at elections was often presented as one between competing whole life systems. No more. In the key ideological battle of the twentieth century, western liberal capitalism emerged the clear winner. A consequence is that whilst competing values and policies lie at the heart of any vibrant political debate, some of the argument now is more shades of grey, more technocratic, more about the means than ends – and this may appear to make politics less intrinsically exciting and political leadership harder.

There is also the timeless and prosaic factor which affects turnout, which is whether a contest is tightly fought (as in 1992) or appears a foregone conclusion (as in 1997, still more in 2001).

But alongside these trends there have been deeper factors at work which help explain both lower participation and increasing cynicism about politics.

British society in my early childhood in the fifties was, still, pretty rigid, hierarchical, class-ridden and deferential. The great thing about being a teenager in the sixties was that one could witness the age of deference crumbling underfoot. That process has continued. Deference means taking things, and individuals, "on trust". But it is not a great shift from not taking things on trust, to not trusting what one is told. I think that one of the explanations for the increase in cynicism about politics is that as the democratic system has become more and not less effective, it has been demystified, and made more transparent, so its continuing weaknesses, rather than its incremental strengths, are what is most revealed.

These trends have in turn been reinforced by the most profound change of all which has overtaken our society, in modern communication, and above all the internet. It means we have many more tools at our disposal in terms of delivering messages, but it also makes the carrying of those messages more complicated, the field for debate more crowded and the environment in which they are sent more competitive. Political parties' problems here are probably best illustrated by data from the world of the media. It is estimated that through television, radio, the internet and mobile communications, people now receive an average of 1,500 messages every single day. Mass television audiences are becoming a thing of the past: in 1995, there were 225 television programmes which were watched by more than 15 million people. In 2004 there were 10. Andy Duncan, chief executive of Channel 4 recently observed that "today's thirtysomethings are probably the last to share a common TV heritage" with "a collective memory of programmes that had impact, special meaning or resonance for them and millions of other people". Maurice Saatchi of M&C Saatchi comments that the under twenty-fives have been "rewired" – "This, apparently, is what makes it possible for a modern teenager, in the 30 seconds of a normal television commercial, to take a telephone call, send a text, receive a photograph, play a game, download a music track, read a magazine and watch commercials at times six speed. They call it "CPA": continuous partial attention". The result: day-after recall scores for television advertisements have collapsed, from 35 per cent in the 1960s to 10 per cent today." Indeed conventional media, especially newspapers, which themselves depend on reader integrity and loyalty, are in a surprisingly similar position to political parties. According to the media commentator Roy Greenslade, newspaper sales for the 10 national titles – together selling around 11 million copies a day – are the lowest in half a century. Between 1980 and 2002, the annual aggregate circulations of national daily newspapers fell from just below six billion to just below five billion. Regional evening papers suffered a similar decline, from an annual aggregate of around 2.5 billion to 1.5 billion. Go on the tube, see how few are reading a paid-for newspaper today, and reflect on how it used to be (and note that newspaper circulations seem to have tracked election turnouts).

Overall, our society is more prosperous, less class ridden, less deferential, and healthier too. All good things for which the Labour Party has fought. But one of the challenges of the fragmentation of modern communications is that as consumers have more sources of information so there is the possibility of an increasing sense of detachment, with the lack of a direct, active relationship between those who provide news or entertainment or even political messages and ideas, and those they are aimed at. The sheer bulk of information and messages has led to an increasingly passive relationship between those who provide and those who receive. As a result, I suggest we have become too much of a spectator society, watching not doing. I enjoy the benefits of consumerism, of choice, as much as the next person. But consumerism which is about me cannot be a substitute for citizenship which has to be about we.

Let me now turn to the third part of this lecture. In doing so I add a health warning: there is no magic shopping list waiting to be identified, which if implemented will lead us to a promised land. In a fragmented world, having the sophisticated dialogue which politics demands, is increasingly difficult.

My first thought is that we should not despair. Politicians as a breed have rarely enjoyed a good press. Shakespeare was not impressed – King Lear said to Gloucester: "Get thee glass eyes; and, like a scurvy politician, seem to see the things thou dost not" Even as the second world war entered its final and successful phase in 1944, when gradually the political parties were united in their support for the war effort, a Gallup poll found that a third of the British public believed MPs were only in it for themselves.

Nor – despite the syndrome which Maurice Saatchi so accurately describes – is there much evidence I've seen that younger people are wholly turned away from politics. MPs are deluged with requests for access to the Houses of Parliament and its work. I could spend every minute of my time in my constituency in debate with school and college students. The 2002 BBC survey "Beyond the Soundbite" found a broad consensus among 18 to 44-year-olds that the power of political opinion and beliefs can shape our fortunes. But the survey also found – and I think this is a central issue for us all – that people are disillusioned with the political process and feel that politics and the media give output but not outcomes.

The difference between output and outcomes may appear a fine one; but output is process, outcomes conclusions.

The next question is how can people be involved in the political process in a way which is more likely to give them confidence that it will secure outcomes for them?

One of the paradoxes of globalisation is that it is atomising society, breaking it down into smaller units, potentially making it more local. And globalisation makes local connections all the more important. The response to the remote, distant homogenising forces of globalisation must be local and personal. We in politics have to learn that too.

If I am convinced of one thing about the future for party politics it is that it must move from being a spectator "experience" to a contact sport, where the public engage with politicians on the field, on equal terms, not from the touchlines.

At the beginning of this lecture I drew attention to two trends. The halving of parties' memberships, and the near-trebling of funds spent in the past 25 years. The two are not unconnected. As membership levels have declined so parties have resorted to market research and direct selling techniques, from focus groups and "personalised" mail shots to "customised" and sometimes semi-automated phone calls.

I do not eschew the use of some science in politics. I've benefited myself from all these techniques and without them new Labour would never have dug itself from its old Labour trough. But these techniques must not take over our politics, because none of them can replicate the most essential element of democratic politics – argument. For example, you can hear what someone has to say on the telephone, but less often see what they mean. It's not accidental that in diplomacy or business, the most difficult negotiations have to be conducted face-to-face; there is no substitute for direct engagement. Douglas Alexander MP, now Transport Secretary, in an important Fabian essay in 2004 said "it is from the clash of moral and ethical perspectives that politics draws most of its energy". Achieving that "clash", that "energy" on the phone or within the focus group is, shall we say, very difficult. It's why the doorstep, the public meeting, serious argument in the local Council and the Commons Chamber have to be elevated as the places where our politics, our values and prescriptions, get thrashed out. Joe Klein, the US political commentator, observes in his new book "Politics Lost" that "Polling has replaced thinking and feeling, not just for politicians" and goes on to say that "pundits like pollsters get most of their information by looking in the rear view mirror" ….. "whilst real leadership has involved the defiance of conventional wisdom the breaking of rules."

Right now we – Britain's political parties – have a once-in-a-generation opportunity to get a proper balance restored between "thinking and feeling", and "marketing and polling". We can take steps to end the "arms race" of spending between the parties, which has become such a flawed alternative to the more difficult, but essential activity of persuading people to join our parties because they believe in and share our values and our approach. For 120 years we in this country have been alive to the way in which too much money spent on politics could pollute the whole process, output and outcomes together.

Significantly, the original reforms of 1883 were aimed in practice at controlling all election spending because all campaigns were primarily conducted at a local level. The 2000 Act which I introduced aimed to fill the gap which meanwhile had been created, on national election spending. But it inadvertently foreshortened the period when controls operate on local campaign spending to a few weeks, and so there's a gaping hold in the system. In my view we now have to have limits in all spending, national and local, by all parties, at all times. If and when we do that, as a result of the current review, parties will be forced if they want to survive to flourish to recruit, retain and involve more members and supporters.

The more local and personal our politics becomes, the more that trust and confidence in the system will be built up. All of us involved in politics know this, and the polling data confirms that trust levels significantly improve in respect of politicians who are known. And we need to extend the use of political public meetings where there are no controls (apart from for public order purposes) on who can turn up and debate is offered? Strange to report, it works, as my own experience in Blackburn of regular open air meetings and local residents' meetings shows.

Next, Parliament has to move fast to catch up with changes in communications. The really big difference in the last thirty years is not so much attendance in the Chamber but the attention paid to it.

The Commons itself used to have a near monopoly as the originator of political news and debate, something reinforced by the privileged access and control which that exclusive club of political journalists – the lobby – would exercise. That monopoly has now gone. Political programmes like "Today", Channel 4 News and Newsnight have become more significant than they were, and 24 hours news services demand a constant supply for commentary and debate. Internet news services record astonishing numbers in terms of page views: the political community increasingly keeps in touch and shares ideas through blogging.

The Hansard Society's Commission last year under Lord Puttnam contained some very important recommendations on how Parliament, especially the Commons, better communicates what it is doing and why. Some are being implemented, but I hope to play my part in pushing for faster and more thorough implementation, especially the recommendations about much more extensive education in citizenship and Parliament's role in delivering that. Important changes made by Robin Cook and Peter Hain as Leader of the House, such as the development of a Visitors' Centre, are literally opening up the Palace of Westminster, and are long overdue.

Two other closing points. First, we in politics have to ensure that the language we use is the language people understand. There is a tendency or all institutions to develop their own linguistic codes, but nothing detaches the public more from any conversation than the use of jargon.

Second, we need to ensure that those who voluntarily work in party politics are properly honoured for this. I am not talking about gongs or favours. But I am worried about an insidious subconscious message that may be abroad that whilst involvement in single interest groups is to be celebrated, involvement in party politics is a little unseemly. Those of us, like me, who've been privileged enough to earn their living through politics learn to take the brickbats as well as the bouquets. But the strength of our democracy in the end depends less on people like me, much more on those who out of believe in a party's values, commitment to public service but little about themselves, work as local activists in communities. We all suffer if their number and recognition is diminished.

I said in introducing this final section – of proposals for a way forward – that there is no magic shopping list for increasing political activity and belief in the process as a way of delivering positive outcomes. There isn't. There are no quick fixes. Raising political engagement in our society is hard, and will take time, energy and commitment. I just know that it has to be done.

Jack Straw spoke to the Fabian Society on Wednesday 28th June. We are grateful to the LSE for hosting this event.