Muslim cleric Abu Qatada will be released on bail on Tuesday, having won his appeal at the Special Immigration Appeals Commission (Siac).
His appeal against deportation was upheld after lawyers claimed he would not get a fair trial in Jordan, where he is accused of plotting bomb attacks.
He will be subject to strict conditions including a curfew when he leaves Long Lartin prison in Worcestershire.
The home secretary has branded the ruling "deeply unsatisfactory".
Theresa May told the Commons that the government "has been doing everything it can to get rid of Abu Qatada and we will continue to do so".
She added: "Qatada is a dangerous man, a suspected terrorist, who is accused of serious crimes in his home country of Jordan.
"The British government has obtained from the Jordanian government assurances, not just in relation to the treatment of Qatada himself, but about the quality of the legal processes that would be followed throughout his trial.
"We will therefore seek leave to appeal today's decision".
Mrs May went on to say that the government would continue "to pursue all avenues" with the Jordanian government. They have assured British authorities that no evidence gained through torture would be used against the preacher.
Shadow home secretary, Labour MP Yvette Cooper, criticised Mrs May's "overblown" strategy in dealing with Abu Qatada, and said "serious questions" needed to be answered.
In response to her statement, Ms Cooper told MPs: "This is an extremely serious and worrying judgement that means from tomorrow [Tuesday] Abu Qatada will be back on Britain's streets.
"I think people will be horrified across this country to learn that that is the case."
Ms Cooper also asked whether preparations were in place for a Terrorism Prevention and Investigation Measures (TPim) - but warned it would be weaker than the control order created by Labour but scrapped by the coalition.
Abu Qatada, whose real name is Omar Othman, is subject to bail conditions including being allowed out of his house only between 08:00 and 16:00, having to wear an electronic tag, and being restricted in who he meets.
However, the judge did not force him to tell the Home Office in advance where he was going if he stepped outside.
Before his bail was decided, Abu Qatada's lawyer Edward Fitzgerald QC had told the court: "There is no reasonable prospect of lawful removal [deportation] within any reasonable time.
"There is no justification for continuing to deprive Mr Othman of his liberty.
"Enough is enough, it has gone on for many many years now," the QC added.
An earlier statement from the Home Office said: "We have obtained assurances not just in relation to the treatment of Qatada himself but about the quality of the legal processes that would be followed throughout his trial.
"Indeed, today's ruling found that 'the Jordanian judiciary, like their executive counterparts, are determined to ensure that the appellant will receive, and be seen to receive, a fair retrial'."
Lawyers can apply for permission at the Court of Appeal to challenge the ruling by Mr Justice Mitting.
Jordan's acting information minister Nayef al-Fayez said his government shared the British government's "disappointment and concern."
"We will definitely study this ruling carefully with them to see what steps we can take. We understand there will be an appeal and accordingly we will work with them to be able to bring him back to justice here in Jordan.
"Concerning the fear of a fair trial for him - there were guarantees for the British government on that, but also our constitution and our judicial system guarantees him that."
BBC home affairs correspondent Dominic Casciani said the decision left Home Secretary Theresa May in "completely uncharted legal waters".
"This is an absolutely huge blow to the home secretary - a very, very significant judgement."
Judges at the European Court in Strasbourg ruled early in 2012 that the cleric would not face ill-treatment if returned to Jordan. They said a special UK-Jordan agreement over Mr Qatada's treatment, called a Memorandum of Understanding, was sound and met European standards of humane treatment.
Critically however, the judge did not believe he would get a fair trial because a Jordanian court could use evidence against Abu Qatada that had been obtained from the torture of others.
Despite obtaining additional assurances from the Jordanian authorities, Mr Justice Mitting ruled that he was not satisfied Mr Qatada would be tried fairly.
Abu Qatada faces a re-trial for allegedly conspiring to cause explosions on Western and Israeli targets in 1998 and 1999. He was found guilty of terrorism offences in his absence in Jordan in 1999.
The Palestinian-born Jordanian has been described as the spiritual leader of the mujahideen. Security chiefs believe he played a key ideological role in spreading support for suicide bombings.
He has been subject to lengthy periods of detention since 2001 although he has never been charged with a crime in the UK.