The house was bu ilt in 1838 by Reverend Samuel W. Cozzens (1801-1875), minister of the First Evangelical Congregational Society in Milton from 1837 to 1847. Rev. Cozzens had been pastor of the Marblehead Congregational Church before he came with his wife Abigail Bass Cozzens and his children to Milton, and where he was once said to be "a faithful and ernest laborer and great blessings descended upon the people during his ministry." Following his departure from the society, he preached at the Old Stone Church in East Milton from 1848 to 1851. 

The house was a four-square late Greek Revival house, built on land originally owned by Nathaniel Tucker, and he and his family lived here until 1848 when the property was sold to Edward Baldwin (1811-1870). In the mid-19th century, Mr. Baldwin owned an extensive estate that extended along the length of Reedsdale Road (then White Street) crossing Central Avenue. The house was set close to the street with terraces that extended to the stonewall. Edward and Ellen Watson Baldwin's daughter, Ellen, married George S. Cushing (1830-1898) in 1862 and their descendants would live in the house until the mid-20th century.

About 1880, the house was greatly remodelled "a la Suisse" as the Cushing Family called it, with broad plazzas, balconies, overhanging gables with decorative barge boards and drops as well as fanciful pinnacles at the center of each gable's pediment. The outward appearance of the house was completely changed and it became one of the best examples of Stick Style architecture in Milton, which was popular in the period of the 1870 to 1885 period. This style was greatly used by John A. Fox (1833-1920), noted for remodelling older houses, a well-known architect from Dorchester and considered the "Father of Stick Style Architecture" in the United States. The side of the house, seen in the photograph, faced Canton Avenue and had a projecting second story porch complete with Stick Style details; turned wood balustres and fanciful cut work, all of which was now available through special mill orders. The house, a large rambling structure with a servant's wing, had been remodelled in the latest architectural style, but a century later it was demolished by the trustees of the church.

Today the former carriage house, according to Elva Proctor, is the parsonage of the First Congregational Church and is the residence of the Reverend Jeffrey Johnson and his family. After the house had been sold by the Cushing Family, the church decided to sell their parsonage across the street (the Tuell-Smith House next to the library which had been purchased in 1959 and occupied by the Tatro Family until 1974) and remodel the carriage house as a parsonage. In 1975, the carriage house was stripped of much of its onetime fashionable Stick Style detail, and "Colonial Revivalized" as it appears today, with the doors becoming keystoned arches in which windows were placed. Surprisingly, the lancet arched granite piers, the stonewall and the terraces still remain but the Cozzens-Cushing House was regrettably demolished in the early 1970's.