Meta, formerly known as Facebook, has announced the release of a novel dataset of nearly 180,000 annotated amateur drawings to help other AI researchers and creators to innovate further.
Drawing as a Universal Way to Capture Ideas
Drawing is a near-universal way for people to capture a character, scene, or idea quickly. However, an abstract or non-realistic appearance can make a drawing incomprehensible to AI models trained on images of real-life objects.
Meta’s Animated Drawings Demo
Meta created an AI system research demo to bring artwork to life through animation. The browser-based demo allowed people to upload images, verify or fix a few annotation predictions, and receive a short animation of their humanlike character within their drawing.
More than 3.2 million people from around the world visited the site, and 6.7 million images were uploaded to the demo.
New Dataset to Analyze and Augment Amateur Drawings
To teach AI to recognize all the different ways someone might draw a humanlike figure, a large dataset of sketches from budding artists would be required.
With the new dataset from Meta, researchers, and practitioners can build tools to unlock new digital-physical hybrid experiences, such as new forms of storytelling and greater accessibility in art.
The dataset includes annotations of bounding boxes, segmentation masks, and joint locations, which could provide more ways for models to identify or animate drawn figures.
Fine-Tuned Computer Vision Models
Meta structured the task of generating an animation from a single drawing of a figure as a series of subtasks: human figure detection, segmentation, pose estimation, and animation. The company’s system incorporates repurposed computer vision models trained on photographs of real-world objects.
Because the domain of drawings, including that of children, is significantly different in appearance, Meta fine-tuned the models using the Amateur Drawings Dataset.
Dataset with Real-World Conditions
For those in the AI community targeting any tool or algorithm that uses pen-and-paper drawings, Meta’s dataset is distinctive for its size and in-the-wild nature. It reflects real-world conditions that aren’t present in digital drawings and high-resolution scans.
Open Science Approach
In keeping with its approach to open science, Meta is sharing the animation pipeline code and this dataset in hope that it will be of interest to other practitioners – both AI researchers and members of the broader research community.
Drawing as a Natural and Accessible Modality
Drawing is a natural and expressive modality that is accessible to most of the world’s population. Meta’s work will inspire a new generation of creators with its expressive and accessible possibilities.
Conclusion
Meta’s new dataset and animation code will inspire a new generation of creators with its expressive and accessible possibilities. The company hopes they will be an asset to other researchers interested in exploring potential applications for their work.
Meta’s approach to open science is evident in its sharing of the animation pipeline code and dataset. Drawing is a natural and expressive modality that is accessible to most of the world’s population. Meta’s work will make it easier for other researchers to explore tools and techniques specifically tailored to using AI to complement human creativity.
