There has been a debate for decades, if not centuries, as to whether the slaughter of the innocents actually happened, or was a "pious fiction" inserted to make Jesus parallel with Moses: there is a tradition from ancient times where the hero must escape death while in childhood.
To be sure, the Jewish-Roman historian Josephus was not real happy about Herod IV's reign, which included a great deal of intra-family murder, and catalogs a good number of the king's crimes. And yet Josephus mentions nothing about any "slaughter of baby boys." To be sure, the Romans would not be happy about such a disturbance to their Pax Romana, but possibly such a crime - if it happened - might have escaped their radar. On the other hand, Herod IV seems to have walked the line properly enough that the Romans left him alone most of the time, even when he was murdering members of his family. Would the Romans under the first emperor Augustus countenance widespread violence by a client king? I would say no.
And yet the Romans had difficulty in finding competent and non-corrupt procurators to handle the area. One historian says that the situation around the time of Jesus clearly shows Roman mismanagement, with e.g. Pilate showing ambivalence (his handling of the Jesus crisis by sending it to the Romans' client king Herod Antipas was the correct move, since the Romans did not want to deal with religious disputes) in governing the area (his handling of other incidents, however, according to Philo of Alexandria and Josephus, was dreadful).
Anyway, was there a "Flight to Egypt" that could be parallel with the "caravan" today? No. And there is another problem: how did John the Baptist, also a newborn baby only a few months older than Jesus, escape this massacre of baby boys? The gospels are mute on that subject. No mention of his escape.
Conclusion:
If Joseph and Mary did go to Egypt, they probably went to Alexandria, which had a large population of Jews...and they went LEGALLY!!! The Roman Empire had open INTERNAL borders, like crossing from Ohio to Indiana, but definitely protected its outer borders from the barbarian darkness surrounding them.