The first library in Sunnyvale was a Reading Room organized by the Women's Christian Temperance Union and staffed by volunteers. Opening in 1908, the Reading Room's collection consisted of fifty books which had been discarded by the San Jose Library.

Six years later, and just two years after the town of Sunnyvale was incorporated in 1912, the first public library in Sunnyvale was established. On May 7, 1914, a petition calling for the creation of a public library, and bearing the signatures of 177 residents, was presented to the town's Board of Trustees, who authorized the creation of the Sunnyvale Public Library. The Librarian reported that in 1915, the library had been used by 426 borrowers and had added 366 books.

In November 1917, the Sunnyvale Board of Trustees voted to participate with the Santa Clara Free County Library, making the Sunnyvale Public Library part of the county library system. In 1917, the county library provided 350 books to the Sunnyvale Library, for which the library paid $159.75.

The Library was relocated to the ground floor of the Wright Building on Murphy Avenue in 1923. The Library continued to grow. In fiscal year 1932-1933, the Library contained 4,664 volumes, had 1,533 registered borrowers, and had circulated books 33,345 times.