The Attendance By Event Report shows all past events, with the attendance at each event.
To produce this report, go to Events → Event Reports → Attendance By Event.