What's the difference between a Windows Game Library and a Class Library? Is it just that one starts off by including the XNA Framework?
Also, where are the differences between the Windows / Xbox360 / Zune Game Libraries? They all seem to start off with the same References (Microsoft.Xna.Framework, Microsoft.Xna.Framework.Game).
There is no real difference. It is just a project template, it helps you get all the project settings right. Yes, the assembly references are one of the biggies. I don't have it installed to check, but you can use, say, notepad to look at the .vcproj file to see what settings are overridden from their default.