If it's just reflecting off a white wall, you should be safe. The dot will be painful to look at but for pratical purposes safe. This is because you're recieving diffuse reflections, the light is being scattered in all directions, so actually a small amount of the coherent, collimated light is reaching your eyes.
Now, this applies only to some extent. For instance, don't try that with a class IV laser. Class IV lasers even require skin protection to prevent burning.
Another factor to consider is wavelength. Green lasers are a lot "brighter" than red ones. So you can have a 500mw red laser that "looks" safe, but is not, while you may have a 100mw green that looks dangerously bright. And worst of all, you can have a class IV laser in the UV or IR region that is invisible to us, and you'd be ablivious to the dangers.
The real danger from all this comes when trying to align the laser. You run the risk of the beam hitting a mirrored surface like a metal object or mirror, and the light reflected off that object can enter your eyes. VERY dangerous.