A prominent local merchant, Robert Calef, originally built the two-story house in 1707. Later, Benjamin Church bought the house from his heirs to use as a Boston residence. With his practice just a few blocks away, the house was an ideal location for Church.
In 1754, Haytham Kenway and Charles Lee visited the house to recruit Benjamin Church to their cause. When no one opened the door, Charles kicked it down, only to find the house ransacked and Church missing. After that, they continued their search throughout the city.