He remembered the promise his ancestors gave to Joseph. Joseph made his sons promise to take his bones out of Egypt when it came time to go to the Promised Land. In the last chapter of Genesis, Joseph was embalmed and placed in a coffin. So presumably, Moses packed up a mummy—and maybe a sarcophagus—and took it with him.