Build in the 11 century to protect the Bishop of Durham from attack, it was occupied by University College of Durham since 1840.