Charles Butler, a Geneva attorney, built the Prouty-Chew House as a Federal style home in 1829. Phineas Prouty, a local merchant, purchased the home in 1842. The property remained in the Prouty family for 60 years. Between 1858 and 1883, extensive alterations were made to the house, giving it the eclectic look seen today. Beverly Chew, the great-grandson of Phineas Prouty, purchased the home in 1921.