Plenty of boutique stores, restaurants and coffee shops to visit. The best part of the town is between Church Street down to Water Street, where you can visit Water Street Market or hop on the Hudson Valley Rail Trail for amazing views of the Wallkill River and Shawangunk Mountain ridgeline.
The premiere location for hiking and rock climbing in the New Paltz area. There are hikes for all skill levels here. The Lake Minnewaska Carriage Road loop trail is less than a mile and a half long, is rated easy and will give you great views of the rock cliffs, lake and fall foliage! I would also recommend visiting Awosting Falls, its easy and a half mile in from the trailhead.
This is the original site of the town of New Paltz, founded in the 1700's by French Huguenots. Here the stones houses built by the settlers and their families still stand today. The Van Aken family can actually trace their lineage to the original settlers of New Paltz, Pierre Deyo and Hugo Freer. You can either drop by and walk down the historic street yourself or take a guided tour. It is only a slight detour from Main Street in New Paltz
Founded by the Smiley Family in 1869, Mohonk Mountain House is a Victorian castle nestled in 40,000 acres of protected forest, overlooking a pristine glacial lake. Here you can either stay as a guest at the hotel (it's not a far drive but I wouldn't recommend driving up to the mountain late at night) or visit for the day. Day visitors can utilize the spa, explore the beautiful grounds and enjoy their fine dining.
If you have young kids, this newly built theme park might be a perfect thing to do while you are in town. It's only about 30 minutes south from the venue too.