By October 1867, Joseph Goodrich had completed the third floor addition to the curious hexagon inn he had constructed twenty-two years earlier at the confluence of a militia trail and pathway worn by Native Americans in the Wisconsin Territory. From the time he had opened the inn in 1845, as many as twenty to thirty stagecoaches per day were stopping at the Milton House, prompting Goodrich to double its number of guest rooms by adding six with a third floor.

Months earlier his only son, Ezra, completed construction of an impressive brick home across the roadway from the inn. It was one of the early large brick homes constructed in the fledging community Joseph Goodrich named Milton. That same year, the academy Goodrich founded in 1844 received its charter as Milton College.

On a blustery October day, Joseph Goodrich made the twenty-mile round trip to Janesville in an open-air wagon for a load of lumber. Goodrich needed the lumber to repair a crumbling outside wall of the five-section business/residence block he had attached to the inn. On the return trip through a driving, cold rain, Goodrich caught a chill and fever he never shook. On October 9, 1867, the young town lost its founder. His death shocked and saddened the pioneer community and was felt throughout the region, sending reverberations back to his New England roots.

The weekly ministry publication of the Seventh Day Baptist Church, The Sabbath Recorder, established in 1844, reported the death of Joseph Goodrich with words that were appropriately reflective and uncannily prophetic in assessing his lasting and endearing influence in life and after death.

“It is not common that the decision of one walking in the ordinary paths of life should affect immediately and in after life more individuals and more interests of the highest value, than did his selection of this spot for his residence and his labors. The friends who accompanied him, settled in the vicinity. . . He attracted from several societies, in the east and southeast, prominent men and women, industrious, intelligent, religious and enterprising.”

When considering the history of the Milton House, one need not stray too far from the words of the Sabbath Recorder 153 years ago. Indeed, Goodrich’s “selection of this spot for his residence and labors affected more individuals and more interests of the highest value” than did any individual before or since.

It’s rare for one of the first buildings constructed in any community to still stand 176 years later. The Milton House was not the first building constructed in the village. In truth, so very few buildings and and none nearly as substantial came prior to the Milton House. The hexagon inn still stands as solid as the rock of which it was built in 1844 and towers as a symbol of all the valued interests that Goodrich brought to the community.

Goodrich and the Milton House are intertwined with the establishment of the community’s Seventh Day Baptist Church, Milton Academy and the first cemetery. They are one in the same with the large public square, the coming of the railroad and the reason for the separate village of Milton Junction.

Goodrich was followed west to the Wisconsin Territory by people of all walks of life. They became merchants, ministers, farmers, academics and Civil War veterans. They strove to a high standard of industriousness, turning a crossroads community into a village; churches into vast congregations; an academy into a college.

Above all, many who followed Goodrich west were like-minded abolitionists, compliant on many levels with his moral devotion to assist southern slaves fleeing bondage en route to freedom.

Interests of the highest value.

Perhaps the most enduring interests of the Goodrich family is that of the Underground Railroad activity that took place in Milton and at the Milton House during the two decades prior to the Civil War. It is that history that led to the Milton House Museum to be designated a National Historic Landmark in 1998. The museum is the only tourable site in Wisconsin that has been documented by the National Network to Freedom as having had Underground Railroad activity. Here is what is known today about that important history:

Much of the history of the Underground Railroad in Wisconsin was lost to time and scholarly indifference during the years following the Civil War through the end of the 19th Century. People involved in the illegal practice of assisting freedom seekers in the years prior to the Civil War largely kept their activities secret even long after the war was over. Many of those involved with the movement of freedom seekers through the area chose to remain silent about their activities, whether because of fear of legal retribution or simply because they did not see themselves as doing anything beyond their own personal moral obligations to assist other humans.

Nor did the workings and logistics of the Underground Railroad capture the attention of scholars and historians until late in the 19th Century. This fact confines an understanding of the regional workings of the Underground Railroad to lore and tales gleaned at individual sites and communities without the advantages of seeing a broad view of the informal workings of the Underground Railroad network. It wasn’t until the 1890s, some thirty years after Joseph Goodrich passed in 1867, that William Siebert began interviewing those who assisted freedom seekers in the years prior to the Civil War. Siebert’s life work — volumes of information collected from the 1890s into the early decades of the 20th Century — was focused on the activities of abolitionists in Ohio, Pennsylvania, New York and other areas in the eastern United States where people had been assisting fugitive slaves since the 1700s and before.

The Wisconsin Territory did not see an influx of white settlers until the late 1830s and early 1840s. Many were not unlike Goodrich, coming from western New York with an abolitionist zeal. The only person from Wisconsin interviewed by Siebert was A.P. Dutton, an abolitionist from Racine who owned a warehouse on the Racine Harbor along the banks of Lake Michigan. Dutton told Siebert that between the early 1840s and the time of the Civil War he assisted more than 100 freedom seekers onto steam ships, captained by abolitionists, who transported the freedom seekers north on Lake Michigan and into Canada.

One of the key details provided to Siebert by Dutton is that all of the freedom seekers who came his way were from Missouri. One possible route followed by freedom seekers involves the Rock River through central Illinois. Several documented Underground Railroad sites follow the river north. Sites include locations at Knox College in Galesburg, south of the Rock River, north to Princeton, Prophetstown, Rock Falls and Byron.

Two of these locations have direct connections to Milton and the Goodrich family and each is – as is the Milton House – a part of the National Network to Freedom and documented as having had Underground Railroad activity. The Owen Lovejoy House in Princeton had direct links to Milton Academy. Lovejoy was a friend and associate of Ambrose Coats Spicer, the president of Milton Academy from 1851 to 1858. The Lucious Read home in Byron, was constructed in the 1840s by Pardon Kimball, whose sister, Ida, married Jerimiah Milton Davis, son of Jane Goodrich Davis, the daughter of Joseph Goodrich.

Freedom seekers from Missouri or Arkansas could make their way north along the Rock River from station to station, crossing into Wisconsin and as far north as Goodrich Crossing at the southern mouth of Lake Koshkonong, some thirty miles north of Byron. The area known now as Newville was called Goodrich Crossing in the 1840s and 1850s due to the fact William Anson Goodrich, younger brother of Joseph, operated a ferry boat at that location. It’s likely William Anson was a key contact with freedom seekers on the river and brought them either directly to the Milton House or first to the Seventh Day Baptist Church and academy in Albion, located about five miles north of Goodrich Crossing. From the Milton House, oral histories tell us, freedom seekers continued an eastern movement toward the Racine Harbor or other points of debarkation along the shores of Lake Michigan.

Due to the lack of documentation concerning the movements of fugitive slaves and those who helped them, it falls heavily on the first-hand accounts to give us a picture—however hazy—of the Underground Railroad network. Several stories and pieces of evidence suggest the Milton House played a key role in the eastward movement of fugitive slaves through southern Wisconsin.

Local tales and rumors of the Goodrich family’s efforts to hide and assist fugitive slaves prior to the Civil War permeated the village well into the early 1900s. It was common knowledge to many locals that the Milton House “was a safe refuge for the fugitive slave,” as stated in the biography of Joseph Goodrich printed in the 1877 United States Biographical Dictionary, Wisconsin Volume.

Several local first-hand accounts of the Underground Railroad in Milton are important stories told to those who tour the Milton House Museum. When Mabel Van de Mark — daughter of Jane Goodrich Davis and the longest-surviving granddaughter of Joseph and Nancy Goodrich — began compiling the Goodrich-Davis Family History, she included two pieces of information that add to the Milton House’s connection with the Underground Railroad. In one of the stories, Van de Mark talks about how her uncle Ezra Goodrich made a point to show his nieces the tunnel connecting the basement of the Milton House with the cabin located behind the inn. During the 1890s, Ezra took Van de Mark to the tunnel and told her his family used the basement and tunnel to hide fugitive slaves prior to the Civil War.

Another relevant story was handed down through generations of the Davis family by an uncle of Van de Mark’s, William Coon Davis. Davis was born in 1843 in Milton, the younger brother of Jerimiah Davis, who married Jane Goodrich. He was a young man in the late 1850s when he was asked by Joseph Goodrich to drive a wagon filled with hay from the cabin behind the Milton House east toward Elkhorn along Territorial Road and to stop at a certain inn. The approximately twenty-mile trip began after dusk and the only instruction given to Davis was to go into the inn, have a meal, and return the wagon to the Milton House. He was given no instructions to unload anything from the wagon or to load other materials for the return trip. Davis related that while he was driving the wagon east on Territorial Road, he could hear rustling in the back of the wagon—whispers and movement. He knew there were people under the hay. Davis said that he did as instructed without investigating the wagon’s contents. On his return trip, he knew that the wagon was empty, save for the hay.

A few years after this incident, in 1861, Davis enlisted in the Union Army and fought in the Civil War. He was discharged in Nashville in 1865. Davis moved to El Monte, California in 1894 and engaged in walnut farming. He died in El Monte in 1931 at age 87.

If this story is true, the Davis tale serves to illustrate how Joseph Goodrich kept his activities secret. Davis was a young man Goodrich trusted with this important errand. By not telling Davis the nature of the wagon’s true cargo, Goodrich limited the knowledge Davis had about the illegal activity.

A similar story of freedom seekers hiding in wagons came to the Milton Historical Society through Edna Dearborn in 2016. Dearborn’s great-grandmother, Jeannie Hull Mudge, was the daughter of Varnum Hull, one of the New York Seventh Day Baptists who followed Joseph Goodrich to Wisconsin. Hull was the pastor of Milton’s Seventh Day Baptist Church from 1850 to 1856 and then the Rock River Seventh Day Baptist Church, located between Milton and the Rock River. According to the story told by an elderly Mudge to a young Dearborn during the 1930s, Mudge spoke of the times when she rode on wagons driven by her father between Albion and the Milton House. Each time they made that trip, Mudge said, the wagon carried sacks of potatoes. Under those sacks were freedom seekers being taken to the Milton House. Albion is located about nine miles northwest of Milton and just a few miles north of the Rock River and Goodrich Crossing. Albion began as a Seventh Day Baptist settlement founded a few years after Goodrich began platting Milton. Albion also had a Seventh Day Baptist Academy and church.

The Albion church was founded by Oliver Perry Hull, younger brother of Varnum. Although not documented by the National Network to Freedom, the original Albion church building is locally rumored to have been a safe haven for freedom seekers. According to a story relayed in 1950 to the Madison Capital Times, local historian Claude Stout claimed that in the 1850s, radical abolitionist students at Albion Academy dug a subterranean chamber under the Albion church for the specific purpose of hiding freedom seekers. It is possible that freedom seekers were getting to Albion through William Anson Goodrich and his Rock River ferry.

These are enthralling stories shared with guests who tour the Milton House Museum. In and of themselves, however, these tales do not justify the National Historic Landmark status of the Milton House. Nor does the National Network to Freedom authenticate Underground Railroad activity based solely upon oral histories.

The Milton House earned those unique designations through the Andrew Pratt papers. Andrew Pratt is the only freedom seeker known by name to have come to the Milton House. Pratt’s fascinating story will be shared in the next Historic Musings.