Former Facebook researchers; Meta have developed an AI system that can identify and automatically animate the human-like figures in children's drawings with a high success rate, and without any human guidance.
Meta researchers said in a statement that "Humans" in children's drawings come in many different forms, colors, sizes, and scales, with little similarity when it comes to body symmetry, morphology, and point of view. They have also commented that "we're excited to announce a first-of-its-kind method for automatically animating children's hand-drawn figures of people and humanlike characters (a character with two arms, two legs, ahead, etc.) that bring these drawings to life in a matter of minutes using AI.”
By syncing and refreshing the artwork to its prototype system, parents and children can experience the excitement of watching their drawings become moving characters that dance, skip and jump. They can even download their animated drawings to share with friends and family. Besides, the system allows them to download a video of the animated drawings to share with friends and family. The animation process involves, AI identifying the humanlike figure in a child’s drawing, and then the system separates the humanlike figure without including anything else on the page.
Next, it pinpoints to the joints of the figure so that the animation can move about appealingly, and after that, the system animates the drawing. Parents can also opt in to have their child’s drawing used to continue to teach the AI model. “If parents choose, they can also submit those drawings to help improve the AI model," Meta said.
Originally, animating children's drawings of people is distinguishing the human figures from the background and other types of characters in the picture. Object detection using existing techniques works quite well on children's drawings, but the segmentation masks aren't accurate enough to be used for animation. "To address this, we instead use the bounding boxes obtained from the object detector and apply a series of morphological operations and image processing steps to obtain masks," the researchers explained.
Using Meta AI's convolutional neural network-based object detection model, 'Mask R-CNN', to extract the human-like characters within a child's drawing for processing. 'Mask R-CNN' is pre-trained on one of the largest publicly available segmentation data sets, but it's made up of photos of real-world objects, not drawings. Animating the 2D figures using 3D motion capture, the researchers took advantage of the fact that many children draw using what is referred to as a twisted perspective.
The Meta team remarks that "We take advantage of this perspective in our motion retargeting step. Independently for the lower and upper body, we automatically determine whether the motion is more recognizable from a front view or a side view." In the future, we can testify that an AI system could take a complex drawing and then instantly create a detailed animated cartoon using multiple fantastical characters interacting with one another and elements from the background. "With AR glasses, those stories could even seem to come to life in the real world, dancing or talking with the child who drew it just moments earlier. The possibilities are as limitless as the human imagination," said the researchers.
