The Madison Farmers Market is the largest producer-grown farmers market in the country, and is definitely worth a visit on Saturday morning (starts at 6:30 am, ends at 1:30 pm). Grab baked goods, flowers, fruit/veggies and Stella's famous cheese bread.
The Terrace is the main gathering spot on UW-Madison's campus - a great place to grab a beer, some cheese curds, and ice cream from the Daily Scoop (made on campus!) and enjoy the view of Lake Mendota. (You may have to plan around the Homecoming/football crowd, however - will update once the game schedule is announced)
Located just east of downtown, Olbrich Gardens is a beautiful, peaceful spot to explore for an hour or two. Don't miss the spectacular Thai pavilion! Open 10am - 6pm every day. Free entry to the outdoor gardens!
If you stay in one of the Stoughton hotels, venture downtown to the classic midwest mainstreet, featuring lots of cute shops and restaurants. Read a feature on the town here: https://isthmus.com/news/community/a-trip-to-stoughton-is-many-things/ A few other Stoughton recommendations: Wildwood Cafe: https://www.wildwood-cafe.com/ Overt Space (fun gallery/shop): https://overtspace.com/ Dune Gifts: https://www.dunegiftandhome.com/