I believe the problem is trying to cast the experiment into the particle or wave camp. On the one hand, you have waves (the double slit interference experiment). On the other, you have photons going through holes along with their "shadow" partners. But in reality, both of these simple pictures don't tell the whole story. Quantum mechanics tells us that, in fact, particles have an infinite extent as long as its particle nature is not defined by an observation. Therefore, even though we want to think that one particle is going through one hole, it really is one particle going through many holes at the same time. Weird, but it's a better Occam's razor solution than introducing parallel universes and their shadows.