You may be correct. But it always goes to the senior member of those chambers. That's just not right.
No, that's not right.
The party that is in control of the House (currently the Republican party) will have an election within itself. From that election, a Speaker emerges.
The Speaker need not be the senior member of the party, but will obviously be one with a great deal of seniority within the party.
For example, Nancy Pelosi (elected to Congress in 1987) had shorter seniority than did Ike Skelton (1976), but was elected as Speaker in 2006.