- Create archive of film.
- Run a pose detection algorithm on all of it, outputing joint data of the detected poses. (Dataset A)
- Film some fresh choreography.
- Run the pose detection algorithm on this. Again, outputting joint data. (Dataset B)
- Run an algorithm that, for each frame, of B finds the pose from the archive (A), that is most similar.
- Place the most similar frames in order and export as a film.
Things we need
Pose detection algorithm. Can use openpose - tick.
Fresh choreography. I can put on some trousers and then I’ll be right with you - tick.
Algorithm that can find the most similar pose . Need some time to develop an optimal one of these, but won’t be too hard.
Mario Klingemann is doing cool things GANs.
Here's a comparison of the two models. Observation: very few people have their mouth open in classical portraits. pic.twitter.com/SPuUFUnW88— Mario Klingemann (@quasimondo) January 31, 2017
Something like this, but with frames from dance videos instead of drawings at each point. And 24 frames per second instead of this slow.
I suspect it may look a bit like ‘Loving Vincent’ in which every frame is an oil painting. There is something disjointed about the animation but there is enough for a coherent and moving pattern to emerge.