Some councils are refusing to change misleading signs on roads warning people not to drive into bus lanes - despite motorists successfully appealing against fines.
Authorities issue penalties of up to £130 in London and £60 elsewhere.
The councils have failed to act over signs on some roads ruled confusing by independent adjudicators.
The AA said it was "ludicrous" to carry on collecting fines "without remedying the problem".
Councils are under no obligation to amend signs as a result of the rulings on individual appeals.
The Local Government Association - which represents councils - said bus lane enforcement was "essential to keep traffic moving".
More bus lanes are being introduced to ensure public transport runs to time - and councils say the penalties stop motorists abusing the system.
But drivers claim some schemes are difficult to understand and they innocently end up going into a bus lane and subsequently receive a fine.
Last year 984,000 people in England and Scotland were issued with a ticket.
The total value of penalty notices issued for bus lane infringements in England and Scotland last year was £76m.
Most drivers take advantage of the discount that comes with paying early, but a small number appeal to independent adjudicators - solicitors and barristers who take evidence from the motorist and the council before making a ruling.
5live Investigates has learned of five cases where adjudicators upheld appeals and criticised signage at a bus lane on Shepherd's Bush Road in Hammersmith in west London.
Eddie Williams successfully appealed to the adjudicator, claiming the signs weren't clear, after he received a penalty notice for driving into the bus lane in Shepherd's Bush Road in 2012.
"They've known for years that this thing is unfair but they just keep collecting the money," he said.
"When the bus lane on the left ends, almost immediately a bus lane on the right hand of the road begins so it's a bit like an ambush."
At this site the warning sign for the lane is often blocked by parked buses and an adjudicator has criticised the lack of a directional arrow on the road, guiding the flow of traffic away from the bus lane.
"The adjudicator said, 'I don't think the signs are fair and I'm going to allow the appeal,'" Mr Williams added.
Hammersmith and Fulham council, which issues tickets amounting to £1m a year at the site, insists the bus lane is "clearly marked and fully compliant". It has been in operation since 2006.
Councils in Medway in Kent and Camden and Harrow in London have also refused to make changes after adjudicators found their signage around bus lanes to be confusing, while Transport for London continues to issue tickets bringing in £13,000 a week on a bus lane where two adjudicators have said the signs are unclear.
The adjudicators' rulings are only binding for the individual case being heard and councils do not have to act as a result of them.
In Hertfordshire, the county council took a different stance and refunded £1.3m to motorists after an adjudicator criticised a sign. The council says it wasn't under a legal obligation to refund the penalties but felt it was the right thing to do.
The AA says it is unfair on motorists when councils don't act on the broader points that come from the adjudicators' rulings.
"It's quite shocking that local authorities are not accepting adjudicators' findings," AA spokesman Paul Watters said.
"They've reviewed the location and concluded that an average driver would be confused so for an authority to continue issuing tickets is quite astounding."
'Plenty of notice'
A controversial new bus lane scheme in Birmingham has seen 80,000 people issued with £4.8m of penalty notices in just three months.
The council's own transport scrutiny committee has said signs aren't clear enough.
"If people do not have the information, they cannot make an informed choice. We have got to get this right if we are going to put cameras in more bus lanes in the city," said councillor Victoria Quinn, the chairwoman of the scrutiny committee.
A Birmingham City Council spokesperson said: "The bus lanes are compliant and drivers were given plenty of notice that we would be enforcing this."
When a new bus scheme in Colchester caused public outcry last year, the council apologised, promised to improve signage and refunded £1m to 30,000 motorists.
The Local Government Association said: "The fact that over 99% of bus lanes fines are not appealed clearly shows that the overwhelming majority of people who are ticketed accept that they have broken the rules.
"On the rare occasion it can be conclusively proven that a council has inadequate signage then they should act to correct the error."