Alter 3d environment by running algorithm on 3d captured imagery.

Will it be possible to dynamically alter the visual 3d environment that the user is actually in? Capture the image of the wall in front of the user in real time, add video effects or animations, and project it back to the user? I'm imagining animated painterly effects of existing environment.

Alter 3d environment by running algorithm on 3d captured imagery.
 
 
Q