Until sometime maybe around the Viet Nam War, we put our hands over our hearts for the Pledge of Allegiance, but not the National Anthem.
Of course you removed your hat but many people, maybe most, didn't have hand over heart.
Now it's the other way around and the majority have hand over heart, but I sure don't consider people who don't do it as disrespectful.