This decorative castle was designed by architect Richard Jupp, with the first stone laid on 2 April 1784. It was built to commemorate Commodore Sir William James (1721-1783) who, in April 1755, attacked and destroyed the island fortress of Suvarnadurg (Severndroog) of the Maratha Empire on the western coast of India, between Mumbai and Goa. After the death of the Commodore, the castle was built as a memorial to him by his widow Lady James of Eltham.