The Colonel Spencer Inn was originally built as a private residence in 1764 on the land owned by Colonel Joseph Spencer of Connecticut who served in the French and Indian Wars, and was one of the first proprietors in this area and one of the charter grantees of the town of Campton. Joseph Spencer, promoted to a brigadier general, had served under General George Washington and the Continental Army at the Siege of Boston in 1776.
Before the end of the war, the property was sold to Benjamin Baker, who also served with distinction in the Revolutionary War and played prominent roles in the early years of the republic. Joseph Spencer went on to be the first Delegate for Connecticut in the Continental Congress. One of the Baker brothers also served under George Washington at the Siege of Boston in 1776, and the second brother participated in the Battle of Bunker Hill.
The guest room names are from the families who owned the home over the years and had active rolls in the formation of the Plymouth and Campton communities.