I did this once but not by accident. My dad and I were driving along the north side of Edwards Air Force Base. I saw a sign for an entrance and decided to go ask about visitor tours. Turns out is was the wrong entrance for that and the guards there were not happy about it. They took copies of our drivers licenses and then watched us closely as we turned around.
I looked it up later online and scheduled a tour where we entered at a south gate. You park outside and then a bus takes you on the tour.