The building has served as a dormitory, bookstore, and tea room since its completion in June 1935. Plans for the building were first presented to the Board of Visitors in 1931, who approved of the plans and accepted a bid from J.W. Davis of Newport News for $62,810. The architect hired to design the building was J. Binford Walford. In 1934, the contract for the project was awarded to Peters Construction Co. of Newport News. The construction of the building was made possible by a Public Works Administration (PWA) grant and loan of $650,000. In keeping with tradition, the building was designed to mimic the Georgian architecture of its neighboring structures. It replaced the Taliaferro Building and served as a men's dormitory, but in 1978, it became the first co-ed freshman dormitory. The building can house up to 60 students. In 1986-1988, Taliaferro Hall housed freshmen designated as Presidential Scholars.
- University Archives Buildings File (2007), Taliaferro Hall, Special Collections Research Center, Earl Gregg Swem Library, The College of William and Mary.