How Microsoft's caricature AI turns photos into portrait cartoons
To become an outstanding cartoonist can take years of practice, but that skill -- or at least the facial caricature part -- has now been automated by a group of Microsoft researchers. The group's work employs a type of AI algorithm called generative adversarial networks (GAN), the favored deep-learning technique for creating'deep fakes' or manipulating real video and images to create very convincing fakes. Instead of deep fakes, Kaidi Cao of Tsinghua University, Jing Liao of Microsoft Research, and Lu Yuan from Microsoft AI Perception and Mixed Reality, used GANs to create a cartoon generator capable of outputting caricatures like a human artist. The deep neural network consists of two caricature GANs, or CariGANs: CariGeoGAN for modeling the geometric exaggeration of a person's face and transferring it from photo to cartoon, while CariStyGAN transfers the style of caricatures to a given face picture. The researchers focus on shape exaggeration and appearance stylization on the basis that these are the two key aspects to drawing a caricature that is warped but still recognizable.