JALI: An Animator-centric Viseme Model for Expressive Lip Synchronization

  • The JALI suite of tools: jAnalyze, jSync and jRig:
    • Automatically generate fast, accurate, adaptable, animation curves
    • Allows the animator to apply their skill and impress their style on sparse curves, with fewer edits
  • jAnalyze takes in any audio .wav and its .txt transcript and creates and accurate alignment. The Alignment tells jSync where each phoneme begins and ends in the audio file, creating the basis for animation. An accurate alignment is critical: bad alignment will create bad lip-sync.
  • jSync, our patent-pending method, originally presented at SIGGRAPH 2016 creates the best lip-sync on the market: Faster, with the lowest processing footprint, and the fewest noticeable errors. We have developed JALI to create eliminate errors from the most perceptible to the least perceptible.
  • jRig turns phonetic speech animation curve output from jSync into FACS action units, and provides interactive control over speech styles like mumbling or shouting. jRig provides an ethnicity and gender morphable FACS-based face rig template that may be used as is, or connected to any custom face rig.

MARKET OPPORTUNITY

  • Game developers, animated TV and film character animators and production studios achieve a jump start in an industry that increasingly demands efficiency and innovation from production pipelines without sacrificing the quality and nuance of hand-crafted animation.
  • Advantages:
    • Visual effectiveness - Coarticulation Engine: Traditional one-phoneme=one-viseme phonetic sheets, that map a phonetic sound to a single viseme or mouth shape, common in cartoons for over 50 years, do not capture the expressive variation of human speech, making realistic characters look robotic or creepy
    • Fast, low processor overhead: Rule-based methodology where a short sequence of conditional statements produces the animation required. No need to set-up or teardown a neural-network, or calculate matrices
    • Easy connection to existing rigs

DEVELOPMENT STATUS

  • JALI’s patent-pending technology – created by an Oscar-winning team with decades of research experience (Pif Edwards, Chris Landreth, Karan Singh & Eugene Fiume) is now available for licensing

Company: JALI Inc.

ID:

P1950

Keywords:

3D imaging , Animation , Artificial Intelligence (AI) , Companies , Computer Graphics , Software

VPRI Contact

Donna Shukaris

Commercialization Manager
Innovations & Partnerships Office (IPO)
(416) 946-7247

Related Resources