David Miliband is planning to leave Parliament to move to the US to work for a charity, a close friend has confirmed to the BBC.
He is going to become head of the International Rescue Committee in New York, the BBC understands.
Mr Miliband, 47, a former foreign secretary, was beaten to the Labour leadership in 2010 by his brother Ed.
The South Shields MP has taken a back seat since, opting not to join his brother's shadow cabinet.
There had been widespread rumours that he was set for a return to the opposition front bench, with Ed Miliband insisting recently that "the door is open".
However, the Daily Mirror reported earlier that Mr Miliband was planning to announce his decision on Wednesday morning.
The BBC understands Mr Miliband, nicknamed "Brains" by former Number 10 spin chief Alastair Campbell, will take up the job of CEO of the International Rescue Committee.
It is a charity which, according to its website, works in more than 40 countries and responds to "the world's worst humanitarian crises and helps people to survive and rebuild their lives." He is thought to be replacing Dr George Rupp.
David Miliband previously said it had been right for him "to step back" from front-line politics to avoid "the daily soap opera" that may have resulted from him working alongside his brother.
But he said this did not mean he could not make a contribution from the backbenches or that he had taken a vow of silence.
His decision will spark a by-election in South Shields, where he has been MP since 2001.
BBC political editor Nick Robinson said his close friends and political allies had not been consulted, merely told, as he "must have known they would try to dissuade him".
He said: "The truth is for many, many months now he has searched for a life outside politics, appears to have failed to find such a satisfying life, and colleagues tried to urge him to return to frontline politics and started to discuss the kind of roles he could take under his brother.
"He has clearly concluded he does not want to return to the fray, he doesn't want to serve under his brother in opposition or in government which is a serious blow to his brother, and disappointment to members of the party."
Labour MPs expressed sadness at the loss of a significant figure in the party.
"David will be brilliant over the water as he always is. He's an inspiration; know he'll be back in fight one day but for tonight am just sad," John Woodcock wrote on Twitter.
Labour MP Keith Vaz said on Twitter: "Stunned by the news about David Miliband. A huge loss to British politics. The best and the brightest leaving the country."
Also on Twitter, shadow energy secretary Caroline Flint said she was happy for her former cabinet colleague and his family and made a joke about his nickname saying "Thunderbirds are go".
David Miliband quit the shadow cabinet after losing the close vote for the party leadership, in which he was clear favourite and lost out because Ed secured trade union votes.
He has remained on the backbenches but fuelled talk of a front-line comeback earlier this year with a vocal Commons attack on the government's benefit changes.
The MP, who is vice-chairman and non-executive director of Sunderland Football Club, is married to American violinist Louise Shackelton and the couple have two children.
He secured a majority of 11,109 at the 2010 general election with 52% of the vote.