The Chester Library Association, dating back to 1875, was the forerunner of the present Chester Public Library. A $3.00 fee made one a member of the association with the privilege of borrowing one book a week. The library was housed in a room on the second floor of the Stone Store, which was then a general store conducted by J. Kirtland Denison. The volumes in this library consisted of books formerly owned by the old library association and others given by various interested people in town.

From the Stone Store, the library was moved to a room in the building next to Robbie's and then to the top floor of the old Selectmen's building. In 1895, the association was dissolved and the library became a town institution, receiving its support form the town and state. For many years it was open three times a week (Wednesday afternoon, Saturday afternoon and Sunday evening).

By 1900 there were 1096 volumes in the library and during the preceding year, 318 people had withdrawn books. Mrs. Emma Leet Perkins, the first librarian with a salary of $50 a year, held that office for about five years. At the end of the fourth year, 399 names were listed on the library records. Fiction was the most popular, with the demand for books exceeding the supply. When Mrs. Denison became librarian, the library was closed for a month while the books were examined and classified, after which a new check-out system was inaugurated. About that time a fine set of Shakespeare's works was presented by the Library Club, and a little later, thirty volumes of public documents were received from the State Library. By 1905 the Library boasted 3191 volumes, and had outgrown the space in the Selectmen's building.

At this time, S. Mills Ely, who was born in Chester, proposed to give the town a library building as a memorial to his parents who had lived at 14 Liberty Street. At a special town meeting on February 17, 1906, this offer was accepted. The Ecclesiastical Society of the Congregational Church deeded some land to the town as a site for the building, on condition that should the library remain closed for a year the lot would revert to the donor. On that land, near the school and churches, a stone building was erected. This is the current location of the library. It was formally dedicated and opened with public ceremonies on Tuesday, Aug. 6, 1907. The old library was closed for two months prior to moving the books into the new building. By this time there were 3500 volumes to be moved.

By 1961 much more space was needed in the building. Originally the juvenile department was in the fireplace room, the adult section to the left as one entered the door, and the magazines and some non-fiction on the right. The old heating system in the basement was replaced, that room painted and fitted with shelves and furniture, and an outside entrance made at the rear of the building. The children's department was moved into this new room, which also had a small area for storage.

Throughout the years, the amount appropriated by the town for the library has increased. In 1910 it was $500; in 1925, $750, with the salary of the librarian $212. In 1936 the town allotted $1050, and in 1949 the total appropriated was $1500, with $100 being received from the State Board of Education for the purchase of certain recommended books.