The house, now in the possession of The Society of the Colonial Dames of New Hampshire, was built in 1763 by Captain John Moffat, who had come to America as commander of one of the King's mast ships which took, for use in the Royal Navy, their cargoes of masts in Pepperell's Cove at Kittery Point. Captain Moffat married Catherine, daughter of Robert Cutt, 2d, and settled in Portsmouth, where he prospered and lived to the age of ninety-four years. The year after the completion of the house, Captain Moffat's son Samuel married Sarah, daughter of Colonel John Tufton Mason, descendant of the Provincial Proprietor, and lived here five years. At that time his business of ship owner was so unprofitable that he failed, and to avoid the severe debtor laws fled to the West Indies.

At the sale of the house, Captain John Moffat bid it in, and moved hither from what is now State Street, bringing his daughter, Catherine, who later married General William Whipple.

General William Whipple was born in Kittery in 1730, and early followed the career of a sailor. Before he was twenty-one he had command of a vessel and made many voyages to Europe and the Indies, as well engaging in the slave trade and importing negroes from Africa. Leaving the sea in 1759, he was engaged in trade with his brother, when the growing discontent of the Colonies enlisted his sympathies. Elected a member of the Congress which met in Philadelphia in 1775, he was chosen member of the Provincial Congress, which assumed the government after the commencement of hostilities. Re-elected to Congress in 1776, he signed the Declaration of Independence. The following year, he was given by the State command of the First Brigade of its militia, and marched against Burgoyne, but soon after his arrival in camp the British general surrendered. Whipple was selected as one of the officers to guard the captive troops to Winter Hill near Boston. Serving after the Revolution as judge of the Supreme Court, he died very suddenly November 10, 1785, leaving no children. Tradition affirms that he personally planted the horse chestnut trees now before the house.

The following year, Captain John Moffat died, and the estate was plunged into a long litigation. The court finally decided in favor of the executors and against Madame Whipple, who removed to the Plains, leaving Robert Cutt, a son of Samuel Moffat, in possession.

Doctor Nathaniel A. Haven, Member of Congress, later purchased the estate from Moffat, and gave it to his eldest daughter, Mary Tufton, the wife of Alexander Ladd, whose descendants held the property until the recent (note: written in 1913) conveyance to the Society of the Colonial Dames.

The house was the first square three-story house in New Hampshire, and is of unusual architectural interest. The hall is a reproduction of the hall in Captain Moffat's father's house in England, and the carved mantelpiece in the living-room was attributed to Grindling Gibbons, a celebrated architect of about 1666.

Almost as well known as General Whipple in Portsmouth were two of his slaves, Prince and Cuffee. Brought from Africa before 1766, when they were about ten years old, the General took Prince with him on the expedition against the British, and gave him his freedom at that time. After the General's death, Prince and Cuffee lived in a small house, on land given them at the foot of the garden on High Street.

UPDATE: The Moffatt-Ladd House is still owned by the Colonial Dames and still open to the public on Market Street.