A method for animation using found objects

Kottke has a neat post about dancing twigs.

Artist Chris Kenny uses bits of twig from tree branches to make these interesting found art pieces that exploit the human tendency for pareidolia, including the one above of twigs in motion.

It strikes me you could generalize this into a method for generating animation using any type of found object.

1. Generate an animation using standard methodology. For example, you might use a stick figure animator to create a sequence of dancing image.
2. Search within a body of found images to find the one most similar to the generated image.
3. Replace the generated image with the found image.
4. Assemble the found images into a movie.

I’ll (lazily) skip steps 1 and 2, because they are hard and you can easily visualize the output, but step 3 might lead to this sequence of twigs:

And step 4 would lead to this movie:

As an extension which allows for filling in gaps where there was no suitable found image, you could train a GAN on the pairs. The GAN would be rewarded for generating the found image from the source image.