Have you named that poor baby yet?
When you bring him home, put him in the bathroom or wherever you are going to have the litter box for several hours, so that he will use it and know where to find it, before he goes "exploring" about his new home.
When we moved, I did that with the girls and put their food and water in there too. It kept them relatively calm with the movers and all the strangers about, and also helped them adjust. I put the carrier in there too, and they slept in there the first night.