Museum of pre-Columbian art. Our favorite exhibit is the Galeria Erotica :P
If you are into books and want to learn about Peruvian literature, this can be a great stop while you are in the downtown area.
Catholic church and convent where you can see the catacombs with bones from priests buried there.
Great place to dine and experience Peruvian folklore dances. My favorite dish was the "Cuy Dorado" if you are up for trying guinea pig.
Delicious Caldo de Gallina (hen soup)