In 1771, Richard Smith made a deal with the town of Salisbury: he would bring 200 books from London if a sufficient number of citizens would subsidize the purchase. This was an offer they couldn't resist; thirty-nine people contributed funds, and thus the Smith Library was born. In the preamble to the regulations of their new library, the original subscribers set forth their reasons for accepting Smith's offer:

Whereas, we the subscribers looking upon it consistent with our duty to promote and encourage every rational Plan that may be proposed for the Encouragement of true religion; for the Promoting of Virtue, Education, and Learning; for the Discouragement of Vice and Immorality...a Library of Books on Divinity, Philosophy, and History, &c. may be conducive to bring to pass the above laudable design...

Over the next century, the Salisbury free library evolved. In 1805, a gift of books from Caleb Bingham, a Salisbury native, created the Bingham Library for Youth, followed by the Church Library, the gift of Miss Harriet Church. In time, these various collections, along with additional purchases, were gathered together and housed at the Town Hall.

In the early 1890s, Jonathan Scoville, another Salisbury native, left $12,000 in his will for a library building. This bequest, together with contributions from other Scoville family members, financed the construction of a gray granite building, built from native stone quarried near Lion's Head Road. Miss Grace Scoville donated a tower clock, which continues to chime the quarter hours with four, eight, twelve, and sixteen notes from Parsifal.

The original building was designed to serve the town as its library and community center. In addition to a reading room, with children's corner, reference department, and stacks, there was an auditorium with piano, stage, and balcony. Refreshments for various literary and musical events were prepared downstairs in the kitchen and pantries. A 15th Century stone carving, sent by Salisbury Cathedral in England, was placed over the fireplace at the far end of the reading room, where it remains today.

By 1981, the library had outgrown the space available, and a new rear section was built along the outside walls of the original building, providing room for books, offices, and community events.

The library's holdings have now grown to over 30,000 items, including a well-rounded core collection of fiction and nonfiction, books-on-tape and videos, large-print books, reference material and maps, periodicals and newspapers, The Lakeville Journal on microfilm, genealogy and local history, as well as books from the original collections. The library also houses impressive works of art, including portraits of members of the Scoville family by Ellen Emmet Rand.

The library is fully automated, with an online card catalog, Internet workstations, electronic databases, and computers available for word-processing.

The Scoville Memorial Library is a not-for-profit association and is governed by an independent board of trustees. While it serves as the town's public library, it is funded largely by private donations.