The practice is perfectly legal. A 1968 Supreme Court decision established the benchmark of "reasonable suspicion" - a standard that is lower than the "probable cause" needed to justify an arrest.
That correctly states the law.
Police can make investigatory stops without probable cause to question people about situations or circumstances that only potentially indicate criminal activity, they don't have to suddenly have to go from zero to 'Evidence is that more probaly than not, there was X crime here and you were involved in it.' Nobody would ever get arrested if that was the standard, except possibly for people who got caught in warrant checks during traffic stops (Of course there would be a lot fewer warrants, because only about a tenth of the crimes solved now would ever make it that far).
The frisk is an adjunct to a lawful investigative stop based on simple reasonable suspicion that something is up and the guy being stopped may know something about it. The primary purpose of the frisk is for officer safety, not so much for gathering evidence.
Cops being human cops, some of them, even some whole precincts or departments, get carried away with it and pervert the actual rule.