Just walking around, you will see a great deal!
In the midst of the maze-like medina are the colorful leather-dying pits. Any number of young boys will offer to guide you to them: just listen for "you want to see the tanneries?" The tour is free (though it is appropriate to offer 1-5 dirhams to your "guide").
Bou Inania madersa: a breathtaking 14th-century religious college. The best example of Islamic architecture a non-Muslim can see in Fez, with wooden walls elaborately carved with geometric patterns and Arabic calligraphy, and a beautiful minaret. In the courtyard there is a portico with a still-functioning mosque, separated by the rest of the courtyard by a small moat.
The view from the hills surrounding the old city is spectacular- there are two fortresses overlooking the old city, the Borj Nord which contains an armaments museum, and the Borj Sud, which is being developed for tourism.
The Merenid Tombs next to the Merenid Hotel, provide excellent panoramic views over the medina and the wider city, as well as the olive tree lined hills surrounding the city, and sanctuary from the bustle of the rest of the city. Beware of the odd opportunistic tout.
The Sofitel Palais Jamai terrace has an incredible view over the medina if you are willing to pay 30 dirhams for a glass of tea in order to access it. This is particularly worthwhile if you can time your visit to coincide with the call to prayer, as you can hear multiple minarets from the terrace.
Entrance to the Moulay Idriss II shrine, the tomb of Fez's founder, is limited to Muslims, but the view from just outside its doors is still well-worth hunting down. The mosque is just off the Talaa Kbira near the Souk Attarine.
Similarly, the Qaraouyine library and mosque and the al-Tijani mosque have beautifully decorated exteriors and worth a visit even by those who cannot enter them.