Immigrant families will be kept off council house waiting lists in England for at least two years, under plans set to be announced by the prime minister.
Downing Street said David Cameron would say on Monday that councils would have to introduce a residency test to stop immigrants gaining immediate access.
Local authorities can already set their own criteria, but many do not.
The Local Government Association said it was "very concerned", and councils should decide how to meet housing need.
Both Labour leader Ed Miliband and Lib Dem leader Nick Clegg have recently given speeches on immigration.
And a poll conducted in Eastleigh on the day of the by-election on 28 February suggested 55% of people who voted for UKIP - which came second - said the issue was their biggest reason for supporting the party.
The government says about 9% of new social lettings go to foreign nationals, a rise from 6.5% in 2007/08.
Mr Cameron will say that the government will issue statutory guidance to local authorities.
He wants to remove "any expectation that new migrants can expect the British taxpayer to give them a home on arrival".
A social housing applicant would have to live in an area for between two and five years before going on the waiting list.
He is expected to say that "this will stop someone from turning up and immediately gaining access to social housing".
Exceptions will be in place for British residents who have to move for genuine reasons, such as work or because of a family breakdown.
Speaking on Sky News, immigration minister Mark Harper said: "That's one of the issues that really upsets people when they see somebody arrive in the country and they appear to get better treatment than people that have been there for a long time."
He said the government was also looking at immigrants' access to benefits, healthcare and education as well as housing.
Chief Secretary to the Treasury Danny Alexander said: "If you want a stronger economy, then we need a society that can welcome immigrants to make an economic contribution to our country, that's what I want to see.
"But we also need to have rules in our immigration system that are seen to be fair, that British people can have confidence in and that's the balance that we're trying to achieve."
The Local Government Association's Mike Jones said immigration and lack of housing were separate issues and that, while there were things the government could do to help, this proposal was "a bit of a sideshow".
"If they are in the country legally then we have a responsibility to do things that are right for people and that's housing," he told BBC Radio 4's The World This Weekend.
"Now if we don't house them that means that we are going to have to deal with them under the homeless laws which cost us a great deal more money. So I think it's down to local councils to pick and choose how they can actually deliver the housing need within their areas."
Mr Cameron's speech on immigration comes days after his deputy prime minister, Mr Clegg, made his biggest intervention to date in the debate, and at a time when concerns have been raised about the potential influx of migrants from Bulgaria and Romania when movement restrictions are loosened at the end of the year.
Mr Clegg called for cash deposits of more than £1,000 for some migrants from "high risk" countries, to be taken with their visa applications and repaid when they leave the UK.
Politicians' response to immigration has been described as "wholly disproportionate" by the Bishop of Dudley, the Right Reverend David Walker, speaking to the Observer.
"Public fears around immigration are like fears around crime. They bear little relationship to the actual reality," said the bishop, who spent 20 years on the boards of housing associations.
"The true threats to our national wellbeing lie not with those who come to visit or make their lives here but with the increasing gap between the rich and poor among us."
Speaking later on Radio 4's Sunday programme, he added: "The reason why people are not getting council housing or housing association properties is not to do with people coming from overseas, it's because governments have failed to build enough social homes and private builders, frankly, at the moment are sitting on land - they're not prepared to build them because it looks better on their balance books undeveloped."
London Mayor Boris Johnson said the capital and the UK "benefit hugely" from attracting talented people from around the world, but it was important to "strike a balance".
"If it is going to make the difference and help us to stop people coming here who want to leech off the system, to put it in crude terms, without damaging our openness to talent then I completely support it," he told the BBC's Andrew Marr Show.
Labour leader Ed Miliband recently admitted the last Labour government did not do enough for ordinary people, becoming distant on issues such as immigration.
"But high levels of migration were having huge effects on the lives of people in Britain - and too often those in power seemed not to accept this," he said in January.
"The fact that they didn't explains partly why people turned against us in the last general election."