I never minded gym class, really enjoyed some of it, but I don't believe it benefits physical fitness or has any other value in education. The kids who enjoy it and really get useful exercise are the active ones who are getting lots of excercise outside the school day, so the extra PE isn't needed. The kids who hate it and slack off through it are the drones, and the expense of supporting PE programs is wasted on them.
There's tremendous value in the voluntary sports programs supported by school systems. The kids who participate get invaluable experience in fitness, competition, patience, discipline, and other qualities that will benefit them in later life. I believe mandatory PE is a waste of time and money, useful only to the teachers' unions for the extra jobs it generates. At best, it's just a recess period for the older kids.