The justice minister has been urged to provide extra space at HMP Birmingham to cope with the rising number of sex offenders being sent there.
The Independent Monitoring Board said the jail was struggling to cope with more historical sex offence cases being solved, a rise in internet pornography convictions and more rape prosecutions.
It has now written to Chris Grayling asking for him to investigate urgently.
The Prison Service said the report would be "fully considered".
In its report, the board said the prison was "experiencing difficulty" in working with the Prison Service to transfer sex offenders to other jails for help in dealing with their specific problems.
'Lowest of the low'
The board said the prison's G wing now exclusively housed sex offenders and that bullying had reduced because of it.
However, it said there were so many sex offenders at the prison they were now having to be kept elsewhere in the jail until space on G wing became available.
Barbara Bradbury, chairwoman of the Independent Monitoring Board, said this "creates problems on the wings" because staff are having to operate two regimes and keep sex offenders separate from other prisoners.
"Prison culture [means] sex offenders are the lowest of the low and if [other prisoners] can get at them either verbally or physically, then they will," she said.
Mr Grayling said the coalition was increasing the availability of treatment programmes for sex offenders.
"It's obviously an important and sensitive area and one we have to look at very carefully," he said.
The prison, which can cater for up to 1,450 adult male prisoners at a time, became the first UK jail to be taken over by a private company in October 2011 when G4S secured a 15-year contract.
In the first annual report since the prison was privatised, the board said the transition into the control of G4S had "not been easy".
However, it said that although it had been "challenging" for management and staff, most had "we are pleased to say, accepted the challenge".
It said that taking over a newly-privatised Victorian prison could not be compared to managing a "purpose-built new establishment", adding that it was too early to evaluate the impact privatisation had had on the jail.
Also in the report, the board said it was concerned at the number of category D prisoners being kept at the category B jail.
It said more spaces had to be made available in open prisons to help prepare prisoners for their release.
The board has also called on Mr Grayling to find replacement beds for the prison hospital.
The report said: "They were never fit for purpose in the first place, and this has been an ongoing problem raised in our reports for several years, with ministers promising in 2008 that replacement beds would be purchased, funding having been secured."
The Prison Service said the board's report would be "fully considered" by ministers and said it would respond in due course.
A spokesman said: "We are increasing the availability of treatment programmes available across the estate for sex offenders."