He called for a quicker switch to electric cars, vans and buses and for more people to cycle and walk to work.
He also warned that diesel vehicles will have to be banned from some city centres to protect millions of people, particularly children, from being harmed by filthy air.
The Government has proposed that greater restrictions on cars, vans and lorries entering central London may be needed to improve air quality.
It suggested tighter rules for six cities in areas where air quality targets for 2020 are expected to be missed.
They are the capital, Birmingham, Leeds, Nottingham, Derby and Southampton.
“We are going to have to face up to the fact that removing diesel cars and diesel vans from the centre of cities at certain points is going to be necessary,” Lord Drayson told the BBC.
“But it’s not just about the policies that Government makes, what we are beginning to realise that it’s the contribution that we can all make to improving air quality.
“Making the choices around how we travel… choosing to walk, cycle to work, does make a difference.”
Environment Secretary Liz Truss was forced by the Supreme Court to publish new plans by the end of this year on how to tackle NO2 blackspots in Britain.
The consultation document, launched by the Department for Environment, Food and Rural Affairs, states that local authorities in six cities should “consider the role of access restrictions for certain types of vehicles” to reduce nitrogen dioxide emissions.
Mayor of London Boris Johnson has already announced plans for an ultra-low emission zone in the capital from 2020.
The RAC has warned it would be unreasonable to punish motorists who were encouraged by the taxation system to buy small fuel-efficient diesel vehicles with a low carbon footprint.