Legend has it the stone structure on the site now known as the Fortress was originally constructed in the 1860's as a fort with gun-ports, and a defense against Indian incursions into the valley. During this time period, many able bodied men had left their farms to fight in the United States Civil War. Jospeh Hassler, a French stonemason, is thought to have been responsible for the construction, as he had other contracts in the valley including a fine stone barn on the Thompson property.
The first wedding on the property took place in 1861 when John C. Ware, the son of the valleys first settler Captain Ware, married Elizabeth Ann Finley, daughter of Waresvilles first postmaster. R.H. Kincheloe, one of the original settlers in the valley purchased the property in 1870 from the European and American Colonization Society. Sources place L.D. Bownds in the valley as early as 1882, when he spent the first night in a rock house at Kincheloe Flat later buying it. In 1885, Bownds moved from Arkansas with his wife Sue, two children, his mother, and uncle, and three orphan children.
The property remained in the Bownds family for 137 years until it was purchased by the current owners in 2019.