Yarmouth Castle, an artillery castle, was built after the French raid on the Isle of Wight in 1545. It was one of the last of Henry VIII's chain of coastal defences built to defend England from the threat of French and Spanish invasion. In 1597-8 an earth bulwark with bastions was added and in 1609 additions were made including corner buttresses. Further alterations were made in 1632, and in 1669 the castle was reduced in size with the demolition of the earthworks and the filling in of the moat. The fort was garrisoned until 1885.
Yarmouth Castle has one of the earliest examples of an Italianate style "arrowhead" or "diamond" bastion which replaced the Henrician style of concentric or curvilinear fortifications. The castle is built of ashlar with part of the south front made of red brick. It is square in plan with sides 30 metres long and there is a sharply pointed two-storey angular bastion at the south east corner. The north and west walls face the sea whilst the south and east sides were originally protected by a moat. There were recessed flankers that jutted out from the south east side of the bastion providing covering fire to the moat. The original entrance was to the east and had a carved royal coat of arms above it. It was built by George Mills and later modified by Richard Worsley, Captain of the Island. The earthworks were demolished and the moat filled in by Sir Robert Holmes, Governor of the Island in 1669. The castle is now an English Heritage property and is open to the public.