St. Peter's Church was founded on January 1, 1827, as the Episcopal church for the growing community in Morristown. Its first services were held in the home of George Macculloch – a prominent town member and builder of the Morris Canal whose mansion stands near the church. The parent Anglican and Episcopal congregation had existed in the area and called itself St. Peter's since the 1760s, but with the anti-Church of England sentiments during and following the Revolutionary war, St. Peter's, like other Episcopal congregations, did not recover and become mainstream until well into the 1820s.

In 1840, the Reverend William Staunton introduced the parish to a movement that emphasized the Episcopal Church's catholic origins and apostolic succession as the ties to the Apostolic community and its Eucharistic worship. The current building was designed to include the styles and art of early and medieval Christian liturgy to engage modern worshipers in the Eucharist in this same way.

By 1887, the large congregation, and growing men and boys’ choirs, justified the erection of the current, massive edifice. The architect of this structure was Charles McKim of the firm McKim, Mead and White that built the old Penn Station, New York, as well as Columbia University, and the Rhode Island State House.

Over 24 years of construction, McKim and the congregation built one of the finest examples of neo-gothic architecture in the United States. St. Peter's is modeled on classic English-medieval parish churches. It also incorporates elements from other Christian periods including chancel mosaics, a baptistry, and the Siena-marble altar each in the Byzantine style, and a Spanish-baroque rood screen. The Norman-style bell tower has 119 steps, and a carillon with 49 bells–one of the largest in the country. The entire parish complex is made up of a cemetery, rectory, great hall and parish house, in addition to the Church, and is known to have at least two secret passageways still in use, concealed behind bookcases and in cloisters connecting the various parts of the property. It was consecrated on November 2, 1911, by the Rt. Rev. Edwin S. Lines, Bishop of Newark, after a parish breakfast that hosted over 500 people.