Parts I-V: the Berry Inn Ball, the Berry Inn Brawl, Caffee in Hiding, Trying Times, and Conversations with Caffee
by Nettie Potter
The Walker House is an imposing three-story structure located in Mineral Point, Wisconsin. Since its alleged establishment in 1836, the building has changed hands many times. Originally an inn and pub, the Walker House has also been an art gallery and meeting hall over the years. The old building is expensive to maintain, which could account for the frequent shifts in management. But there may be another reason why buyers leave so quickly. Some say a tavern spirit haunts the premises, eager to avenge his death.
Legend has it that after a night of drinking and partying at the Walker House, William Caffee killed his rival, Samuel Southwick, in a dramatic duel (Brown, 209; Norman and Scott, 194 Haunted Wisconsin). He was then sentenced to hang next to the tavern. A few conflicting accounts say that after murdering Southwick in a nearby town, Caffee went to the Walker House to hide. Authorities found him, put him in shackles, and condemned him to hang near his old hideout (Godfrey, 39). Others say that Caffee was a notorious horse thief. When he stopped at the Walker House for a drink, a mob carried him off and lynched him (Akamatsu, 167; Steiger). Whatever the case may be, Caffee got a taste of “frontier justice” when he was hung outside the Walker House in 1842. His was the last hanging in Wisconsin, next to the state’s oldest inn (Balousek, 44; Jacobson). Since then, people say Caffee’s ghost has roamed the Walker House seeking revenge. To this day, workers at the inn see his apparition and flee in terror.
What really happened to William Caffee? Was he a horse thief, a gentleman, or a cold-blooded killer? Most importantly, is he still making his presence known? The true story of the Walker House ghost may differ from what the locals are used to, but it is just as compelling and strange. This ten-part series will explore the life, death, and afterlife of a Mineral Point icon.
Part One: The Berry Inn Ball
It all started at Berry Tavern, now called the Lamar House, on the night of February 22nd, 1842. Adeline Berry and her husband Fortunatus Berry were hosting a “Birth-Night Ball” to celebrate the 110th anniversary of George Washington’s birthday (Breihan et al.). They invited everyone they knew to their inn (and home) at Gratiot’s Grove, in what is now as Shullsburg. As part of the festivities, the guests would participate in an advanced form of square dancing known as the cotillion (Breihan et al.).
One of those to receive an invitation to the ball was William Caffee, a 29-year-old miner from White Oak Springs, Wisconsin. Bill was an aggressive and determined man, perhaps because he became an orphan at the age of eight and was left to care for his three younger brothers (Find a Grave). Although Bill could be crude and brash at times, he had a soft spot for dancing. Each dance gave miners a rare opportunity to socialize and woo the ladies. As such, Bill tried to behave himself at Berry’s gatherings.
But on the February 22nd party, he had too much to drink, and his short-temper got the better of him. He claimed he would like to kill, or at least punch, Charles Gratiot (“Trial”). Gratiot liked organization and insisted that callers bring only a set number of people to the dance floor at a time. Caffee hated this rule (and therefore Gratiot). He also assumed the dance caller, Amos Culver, left him out of the fun by avoiding his name. Caffee asked Amos to join him for a drink, hoping to get some answers. Culver explained that he had called Caffee’s name last dance, and did not usually call names twice in a row. Caffee demanded to see the list of names. After Culver gave him the list, Caffee ran off with it yelling, “If I can’t dance, then no one should!” (“Trial”).
Part Two: The Berry Inn Brawl
Caffee hid the list in his pocket, then went outside with his friend Charles Lamar for a chat. He warned the other partygoers not to follow him for the list, threatening, “I’ll cut the God-damned heart out of any man that gets in my way” (“Trial”). Sam Southwick ignored the warning and started advancing toward Caffee. Southwick was the cousin of Adeline Berry and a “peaceable man” according to most accounts. But, unfortunately for him, Southwick chose this day to strike up a fight. He picked up a piece of firewood and looked ready to swing.
Caffee told Southwick to stand back or be killed. He had never met the man before but knew instantly that he did not like him. Sam Southwick did not move very fast as he was a 54-year-old man with a limp. Caffee pulled out a pistol almost immediately and shot him near the heart. Lamar checked in on Southwick and asked how he was doing, to which Sam responded, “I am a dead man” (“Trial”).
Lamar turned on Caffee. “What have you done?” he asked. Bill simply said, “I told him to stand back” (“Trial”). By now, a crowd of people had gathered at the scene. Lamar tried to fight Caffee but stopped when he shouted, “Charley, don’t lay your hands on me,” and someone in the crowd said Caffee had a knife (“Trial”).
According to court testimonies, Samuel Southwick died at 1 am on February 23rd, 1842, a few minutes after the shot was fired. Caffee fled from the scene.
Part Three: Caffee in Hiding
After the shooting, Caffee ran to his hometown of White Oak Springs, Wisconsin. When he arrived in the area, he met Charles Lamar, Samuel Scales, and Samuel Dunbar, who had all been at the ball when the shooting occurred (Breihan et al.). They tried in vain to get Caffee to turn himself in. Lamar has been a constable and told Caffee he had a solid argument for a self-defense case. Caffee was unconvinced and rode off on horseback with his brother Ben. He traveled past Iowa and eventually wound up in St. Louis (Breihan et al.).
During his travels, Caffee met one of a notorious band of horse thieves and counterfeiters. (This encounter started the rumor, mentioned earlier, that William Caffee also stole a horse.) When Caffee tried to turn in the gang member and collect his substantial reward, the criminal reportedly exposed him to the sheriff as the man who shot Sam Southwick (Breihan et al.). Both men ended up in jail.
On June 17th, 1842, Wisconsin territorial governor James Doty “appointed the politically connected Mississippi steamboat captain John Atchison territorial marshal” (Breihan et al.). He then sent Atchison on a mission to pick up Caffee in St. Louis. He promised Atchison a $113.75 payment for returning the prisoner. The next month, Atchison arrived in Mineral Point with Caffee but never received any money (Breihan et al.).
Part Four: Trying Times
Caffee’s trial began on September 13th, 1842, in a small log courthouse in Mineral Point – the Iowa county seat at the time (Carted; “Iowa County Courthouses”). Judge Charles Dunn presided over the court. Attorney General Mortimer Melville Jackson served as the prosecutor, claiming that “All the circumstances of this case conspire to establish, clear as the sun at noon day, and as strong as holy writ, the proof of the prisoner’s guilt” (“Caffee’s Trial”).
Moses Strong served as the defense attorney, assisted by Platteville mine owner Lorenzo Bevans. John Blackstone, Fortunatus Berry, R.C. Buzan, Laban Cassidy, Amos Culver, William Dering, Beon Gratiot, Charles Gratiot, Edward Gratiot, J.R. Gratiot, Samuel Huddlestone, Charles Lamar, David Lufkin, and Joseph Scales were among the many who took the witness stand (Breihan et al.; Crawford, 113; “Caffee’s Trial;” “Trial“). Each of their testimonies matched up, with the exception of Joseph Scales. While the rest of the witnesses stated that Southwick did not hit Caffee, Scales thought he saw Caffee dodging Southwick’s blows (“Caffee’s Trial”).
On September 16th, the jury pronounced Caffee guilty. He showed no emotion during the verdict, remaining “cool and unconcerned” (“Caffee’s Trial”). Caffee’s counsel requested a second trial, so the court decided to reconvene on September 21st. On that day, the motion for a new trial began with affidavits claiming there had been some drinking going on in the jury room. This motion was overruled when the deponent, Thomas Gloster, admitted that he had bought the liquor in the first place (“Caffee’s Trial”). On September 24th, Caffee came to court in irons. He objected to the law clerk, “I don’t think I have had a fair trial, as the evidence against me was not true” (Breihan et al.; “Caffee’s Trial”).
The clerk lamented that someone as young and handsome as Caffee should be sentenced to death. He expressed his sincere hope that Caffee would repent, and that the execution would prevent similar incidents in the future. After insisting that the trial was indeed fair, he announced the sentence:
It is the Judgment of the Court, and the Sentence of the Law, that you, William Caffee, for the crime of Murder, of which you stand convicted, be taken hence to the jail of this county, and there safely and securely kept, until Tuesday, the First Day of November next, on which day, between the hours of 10 o’clock, A. M. and 4 o’clock, P. M. you shall be taken from said jail, and then and there publicly hanged by the neck, until you are dead. And may the Lord have mercy on your soul.
“Caffee’s Trial”
Soon after the verdict, Caffee’s friends allegedly made plans to break him out of the Mineral Point jail but did not follow through. Instead, they petitioned the governor to stop the hanging and issue a less severe punishment (Crawford, 112). Governor Doty thought that long term imprisonment was not enough for a murderer and did not commute the sentence. In a letter to the Iowa County sheriff, George Messersmith, he wrote:
There is little doubt that if Caffee had not lived in a society in which the practice of carrying about the person concealed weapon was not only tolerated, but considered rather as distinguishing the gentleman, than the ruffian or bully, he would not have been guilty of shedding Southwick’s blood. It is painful to see the effect of this barbarous custom visited upon one so young; but I do not learn that since his confinement he has exhibited any evidence of contrition, or that he is fully sensible that his situation which has awakened so strong a sympathy in the bosoms of his friends is occasioned by his violation of the laws of his country, or that he is yet aware that those laws are supreme and that no man has the right to take the life another for any private injury.
James Doty
Caffee did not have any desire to apologize for his actions. Instead, he was hell-bent on making life difficult for his captors. Each week, a blacksmith named James James arrived to re-rivet Caffee’s iron shackles. Whenever he removed the constraints, Caffee threatened, “You had better be careful how you put the irons [back] on, for if I should get loose, you might be sorry” (Carted; Crawford, 112). Caffee’s claims that he would get free were not taken lightly. Mineral Point authorities stationed four armed men to watch Caffee’s cell, just in case (Crawford, 112)
Part Five: Conversations with Caffee
Despite Caffee’s rough facade, he was quite light-hearted around visitors. He frequently joked about his upcoming doom in a way that he upset his friends and family. When his brothers came to visit him for the last time, he apologized for not taking his death more seriously (“Execution of Caffee”). On Halloween night, the day before he went off to the gallows, he wrote letters to his loved ones asking forgiveness. He shared his account of what went down at the Berry house and his sudden spiritual revelations. Two of the letters appeared in a local paper and are copied below.
The first letter is to his uncle, A.S. Daugherty:
Dear Uncle,
It was your request that I should leave a statement of those who swear false against me, which I declined at that time; but to satisfy your mind that false evidence convicted me, I will give you a full detail of the transaction. I went to the ball at Capt. Berry’s on Tuesday night, as I usually went to balls, with only the arms that Bryant stated. About 9 o’clock, I had some words with Gillet. I finally told him if he did not let me alone that I would cut him open; to which he asked me not to jump on him; I replied I would not, and so left him. Everything was peaceable until about 12 o’clock when they refused to call my name. I went to Culver and asked him to call my name, which he refused to do. I then asked him to let me have the list, and he gave it to me. I went down stairs, and said I was going home; I was followed into the bar room and collared, and the list demanded of me; I told them they should have it as soon as I looked at it. I do not know who it was collared me, but I pulled loose of them, and went out of the house. When I got to the door, I met Lamar, and stopped to talk with him. I pulled out my knife and said if they undertook to take it from me, I will cut some of them open; and put it up again; and while we were standing there, three or four came out with clubs, and Southwick struck me on the shoulders, and Lamar pushed me off the steps. I stepped back and told them not to strike me any more. Then Southwick turned a club at me, and said to “beat him to death” or something of that kind; but Southwick kept on striking at me, and me backing out, and asking him to desist, till I got to the ditch, and fell to my knees, and when I raised I shot. When I fired I said, “Now God Damn you, you have got it.” I then started home, and Scales came to me and told me to get into the State as quick as I could; and I went to the Springs, and there saw Lamar again; and he told me not to go, for I was defending myself, but I told him it would cost me a good deal, and so I made off. I made no other threats than I have mentioned; and all others that was swore to was false. You heard the evidence, and who gave it. I will not call any names; only Capt. Berry, as he swore to the most of any; though I freely forgive, and do sincerely hope, when they are called to stand before the tribunal of Almighty God, that all their sins may be forgiven, and that they may sit on his right in peace for ever. As my time in this world is running short, I will close by giving you my dying thanks for your support on my behalf. Give my love to Aunt Polly, Margaret, and all the family. These handcuffs are so in the way or I would write more. Tell brother Ben farewell. Farewell to all! (“Caffee’s Letters”).
The second letter is to his cousin, Miss Margaret M. Daugherty:
Dear Cousin,
When you was here you asked me to join you in prayer, which I thought trouble at that time. But thanks be to God for his merciful kindness to me in giving me grace to believe myself forgiven. He has shown me his only Son crucified for my sins as plain as I see to write to you. I can now ask you to hold out faithful, and send me in that world where all tears are wiped away. Tell brother Ben to address himself to our heavenly Father who alone can save in hour of trouble. I want you to write to all my brothers, and tell them I die happy, and that I hate to leave them but I do not hate to die, for I am going home, and hope to meet all there. Tell Levi [that] when he was here, [I] was in a passion, and did not talk rational, but the Lord has taken away that stubborn heart, and gave me a heart to know and feel his love. I could write forever, but my time has come. I must go home; so I must bid you farewell, but not forever, I hope. Farewell! (“Caffee’s Letters”).
Towards the end of his life, Caffee believed he was going to a better place. Convinced of his salvation, he looked forward to what awaited him on “the other side.” According to one account, Caffee “was not without hope of happiness” after his death (“Execution of Caffee”). Caffee knew he was “going home” after the hanging, but was “home” actually the Walker House?
Come back later this week for parts six through ten of “Tavern Spirits”!
[Editor: The above “Works Cited” page includes sources for both these five sections and the next installment of parts six through ten. Because of that, it comes with an unusual disclaimer. As Potter will show in the next five sections, a few of these sources–particularly those claiming Caffee stopped in at the Walker House–have not checked primary sources of the trial! That’s one reason for putting this story in print. In contrast, those sources closer in time to the actual event are true. The next five sections will look in more detail at the many ghost stories surrounding William Caffee and others at the Walker House.]
