David and his younger brother, Richard, were both born in the Dutch Republic. However, their parents moved with them to London, England. Their father was a dockworker, so the two were raised near and in the dockyard, and they would frequently do errands for sailors and merchants to pass the time and earn some extra money. They were also known to get into many fights.
In 1750, the brothers joined the crew of the Stalwart under the command of Robert Faulkner, whom they served loyally under until his disappearance in 1753. Though he was presumed dead, Faulkner had actually gone into exile at the Davenport Homestead.
The Clutterbucks spent years travelling on various merchant and privateer vessels—almost always hiring on as a pair. They eventually settled semi-permanently in Martha's Vineyard, taking it as a place to rest between contracts before they would get tired of all the quiet and hire on to another crew.