There aren't enough dead Egyptians, or slaves, to account for the workforce needed, not by a long shot. Under such conditions, the avg slave who started working at 13-14 (puberty) would have only lived 5-7 years, accounting for accidents and physical wear and tear. They simply didn't reproduce fast enough. Frankly, there weren't enough people in the world at that time to build those things, given projected mortality rates and lifespans (based off the bones we've unearthed).
If you remove the impossible, whatever is left, no matter how improbable, is the answer. Unfortunately we don't have enough data. We're missing a very large piece of the puzzle, probably a construction technique that we haven't conceived of, a technology that's been lost to history. The details of the "process" are said to have been in the Library of Alexandria as late as the 3rd century CE, but the Roman Church made sure we'd never get our hands on any of that information. Thanks guys!