I don't think a president can eliminate a cabinet position all on his own without it passing through congress.
I agree that a lot of what the Federal government does should be downsized and that the states should be given back the responsibility.
Why not?
They serve solely at the will and pleasure of the President, who is ultimately responsible for everything they do. (They're doing it on HIS authority and in his name.) The first presidential cabinets were informal advisory boards anyway, and only later became official positions within the government.
My position is simple: if the Constitution doesn't expressly give the Federal Government authority to do/regulate whatever that particular cabinet agency is doing/regulating, Under the benevolent reign of the Defiant-Six regime, the Federal Government WILL NOT be doing it.