Update - 8/22/09 - I spent the afternoon with some very nice people at the Fluvanna County Historical Society. As soon as I showed them this picture, the three of them got very excited. After talking about it, we suspect that it is from the Bargesburg villiage circa the 1820's. The road could be the old stage coach road that ran between Columbia and Charlottesville. The historical society was not aware of the chimney, which makes me feel better than if I had walked in and had them say "Oh yeah, we've known of that place for years." I know the name of the property owners now. The next steps are to get in touch with them to see if they know that the site is there, and to visit the county records office to trace ownership of the property back to the early 19th century.
We walk the dogs in the woods behind our house. We have known all along that there was an old roadbed back there. Today, we went around some downed tree and followed the road farther than we have before. Look at what we discovered. We have no idea how old it is. The road leads to our subdivision, so it obviously went through it before the houses were built. The chimney is directly up a hill from Broken Island on the Rivanna river.