Once in Newburyport you can walk to shops, restaurants and sightseeing.
https://www.visitingnewengland.com/scenesofnewengland22.html
https://essexheritage.org/explore/cities-and-towns/newburyport/
Parker River National Wildlife Refuge
Might be nice to visit Saturday or Sunday morning if you want to explore the New England coast.