Just the ferry ride there and back from downtown Seattle is worth it for the views, even if you turn around and get right back on. Bainbridge is also a great place to spend some time exploring for a few hours with tons of galleries, shops, and cafes. Cars are welcome on the ferry, but consider renting a bike from the Ferry terminal!
Not only will you get great views, but in mid-summer you'll also be treated to watching the hang gliders (i.e. crazy people) jump from the summit. Trailhead can be tricky to locate, as evidenced by that one time Kalie and Jeff accidentally added many miles each way to the route from starting from the wrong spot... whoops.