The painting uses several references to depict an accurate version of its use as a prison.  In particular several contemporary watercolour paintings by Captain Durrant a royal Navy officer stationed at the castle during this time.  Also, maps and technical drawings show how the buildings looked along with their locations.
All of the historical information is shown in the English Heritage website.
 
Here are just a few salient points,
 
The prisoners’ uniforms were yellow
The prison held about 7000 prisoners at the period shown
Prisoners’ included many craftsmen who manufactured goods and sold them to the locals through the fence through the centre of the castle.  The painting shows this.
In the far right can be seen the clothing wash rooms and drying area outside the castle walls.
Prisoners’ from the war in the Carribean were held in hulk ships in the harbour and sometimes brought ashore by rowing boats to work on the mainland.  This is shown at the lower right.
The soldiers’ quarters and parade ground are shown outside the castle walls.
The castle was used as a prison (one of 12 in the UK) due to Forton prison in Gosport being at full capacity.  Portchester castle, although a ruin, was the nearest contained building capable of accommodating large numbers.