Train
Really pretty and quick ride along the water - about 90 minutes total to Poughkeepsie and leaves out of Penn. There is a bar cart too ;)
Train
Good ol' MTA
Travel Note
James Baird is about a 15-20 minute drive outside of Poughkeepsie. If you prefer to drive, there is free parking inside.
Hotel
There are a few hotels available, but this is one of the more conveniently located ones right in the Poughkeepsie downtown area.
Hotel
Another hotel option :)