One of the first settlements in the Town of Bayview was known as the Village of Houghton. Complete with its own train depot and post office, the village sprang up above the brownstone it began to quarry. The Hartley Quarry, Smith Babcock Quarry, and Prentice Quarries all opened in the 1880s. Men with stone cutting skills were necessary laborers in the quarries and as they took up residence the community grew. The "Great Monolith" stone exhibited at the Chicago World Fair was cut from the Prentice Quarry. Many local buildings in the City of Washburn are made from these brownstone quarries. The quarries closed in the mid 1890s. Many of the quarries had already filled with water as springs were sometimes tapped while cutting;  additionally, the abandoned quarries filled with water. The swimmable waters of the Bodin Brownstone Bowl led to the creation of the World's Log Rolling Championships during the early 1920s. For several years, this event brought thousands of people to the quarry's edge. People also settled along the lakeshore, and many families fished for their livelihoods. Some of their old homesteads, boathouses, docks, and cribbing remain visible on the water's edge.

In the 1890s, immigrants from Finland began to settle among the stumpage of the forests west of the Sioux River. Farming was the primary occupation in the "Finn Settlement." The men worked the fields sharing horses to make a team, while the women tended the herds and children. The Finns were great for building halls for meetings and gathering places, and one was erected where the Finn Memorial stands today. Initially a wooden bridge was built to cross the Little Sioux River as it was the primary passage out to the railroad (which basically followed the lakeshore north and south). The present bridge, built in 1928, was built with labor from the people of the Finn Settlement.

Early settlers of the Town of Bayview had to work long, hard days to make ends meet. When the week's work was done, there was always room for fun, and the need for communion of neighbors was always evident. The town schoolhouses and Finnish Workers Hall were often the sites of community dances, pie socials, and school programs. Women's groups formed to provide fellowship and to address community needs. The shoreline women between the Sioux River and Houghton Point formed the Sunshine Club. They gathered not only to visit but also to support their community and those of greater need. Similarly, the women of the Finn Settlement also met as the Homemakers Club. Many of the children were also organized and continue to be a part of the Friendly Valley 4-H Club.

The educational needs of the children were met through the creation of four rural schools, years before the formal school district was created in 1913. The Gibson School (also called the Finn School) started in 1903-1904. The Houghton School was also used as a Town Hall and is still standing on its original site. The Sioux River School was built on the corner of Kjarvick and McCulloch Roads. After the closing of the Roosevelt School, the building was used as a town hall until it was replaced by the current town hall in 1974. Note is made of a Pike Quarry School operating 1916-1917, but no further references exist except Esther Olson as teacher. Records show that these schools closed about 1938-39 when the City of Washburn took over the educational system.

The Town of Bayview was formally created by Chapter 89, Laws of 1913, from sections of the Towns of Washburn and Bayfield along with a section of the City of Washburn. The first town meeting was held in 1913 in the Houghton School and officers were elected. The Town's assessed value was set at $270,000. For the next few years, petition requests from freeholders within the Town were heard, and the laying out of roads began.