The world’s first fully autonomously animated “digital human” created by reverse engineering has crossed the “uncanny valley.”

In the past, friendly robots who helped in times of trouble only appeared in science fiction novels and manga comics. But today, building relationships with “digital people” is no longer fiction. Amid the COVID-19 crisis, Soul Machines’ digital people became “trusted information partners”.

“Bella” is the New Zealand’s COVID19 digital helper. She looks like a human, and similar to Siri and Alexa, communicates by voice to provide appropriate answers to users’ questions. Not only does she capture emotions and feelings conveyed by speakers facial expressions through device’s camera , but she also emotes at her own discretion, changing her gaze or smile while occasionally blinking to behave in a “human, believable,” and non-verbal manner. This is the first real-life autonomously animated digital person, and it was created by the AI startup Soul Machines, founded as a spinout from the University of Auckland. The company is now providing digital people for global public and private sector clients, creating everything from customer support to corporate brand ambassadors. (In Japan, they created digital skincare advisors and brand ambassadors for the cosmetic brand SK-II.)

The world has changed due to COVID-19. In this new world, the demand for contactless communication tools continues to grow. We interviewed Ms. Una Softic, who is playing an important role in establishing a smooth coexistence between humans and machines, about efforts and future possibilities that Soul Machines are providing in this domain.

Una Softic
Vice President Japan at Soul Machines

Una Softic is VP Japan at Soul Machines, an AI company that is known for creating autonomously animated digital people with the ability to automate and personalize digital interfaces and interactions at scale. Prior to her current role, she led global technology programs at Nikkei as Head of Innovation at Nikkei Editorial Innovation Center. She started her career at General Motors in Slovenia and Intel in Germany, covering international business development across EMEA, and before moving to Japan, drove the international business development of software startups in San Francisco. Una holds a double Master’s degree in Linguistics from the University of Ljubljana, Slovenia.

INDEX

・Soul Machines digital people
・The role of the world’s first fully autonomous digital human in the “COVID-19” era
・A world where digital and living humans coexist
・Key points

Soul Machines digital people

With the world’s first autonomous animation engine, Soul Machines digital people are brought to life with diverse visual appearances, personalities, and characters. They all have a “digital brain” modeled on the human brain, which is equipped with three control systems—visual, auditory, and tactile—and incorporate universal human skills, such as sensitivity and empathy, enabling natural face-to-face communication with humans. Soul Machines’ digital people continue to become more advanced over time with their patented cognitive models, learning from each user interaction in a similar way humans learn.. This technology was developed by Prof. Mark Sagar, university professor, researcher, and co-founder of Soul Machines. He began reverse-engineering humans in order to build a computational model of human expression and consciousness. He has also created realistic digital characters for blockbuster movies and has won two Academy Awards for his work in movies Avatar and King Kong.

The role of the world’s first fully autonomous digital people in the “COVID-19” era

――I hear that the global COVID-19 pandemic has increased the presence of digital humans, especially in Western countries.

Una：Yes. One of our recent projects is Florence, the WHO’s [World Health Organization’s] first digital health worker, who is providing clarity and expelling damaging myths around tobacco usage and COVID-19. This comes as a countermeasure against fake news, which can be confusing and fatal in times of emergency. We also created another COVID-19 project with the New Zealand Government. Digital helper “Bella” was created to help Kiwis navigate through time of crisis, providing a wide range of support, from inquiries about personal health conditions to how entrepreneurs and SMEs can apply for state assistance. Digital humans can quickly learn and address many inquiries and handle large numbers of interactions 24 hours a day, 365 days a year. This allows humans to focus on their most important, urgent tasks without risking human-to-human viral transmission through contact.

――What was the reaction from users?
Una：A large number of users went beyond basic COVID-19 information exchange and shared their personal concerns with Bella. With increasing need for “mental health support” topics, we added a “kindness module” with a revolutionary interactive challenge and an educational module on the brain and stress that encourage human communication, recommend an exercise or suggest other real-life anxiety-relieving actions.

――This may be an unexpected result for the Soul Machines team, but it’s a very interesting phenomenon.

Una：We received great feedback from people who felt that their daily communication has been enriched just by having a small daily conversation with “Bella.” Even among us at Soul Machines, Bella brought improvement and helped us boost our positive communication with colleagues and partners.

――It seems that the user obtained a sense of security similar to that felt when interacting with a close acquaintance.

Una：When humans communicate, we read each other’s emotions using nonverbal cues such as slight movements of our facial muscles. In the same way, digital people can recognize person’s genuine feelings and provide an empathetic response. For example, if the person smiles, they [the digital people] will also smile, not by replicating person’s facial expression, but by analyzing the emotions and selecting the most appropriate reaction in the given situation. . If our digital person detects that the user is angry or sad, they would provide a cautious, calm and empathetic response, in order to reduce the stress and smoothen the communication.
Our digital people are created using brain models that are similar to human brain chemistry. For example, their virtual nervous system interprets human emotions and creates virtual equivalents of dopamine and serotonin, triggering our digital people to respond in a similar way that humans do. Bella, for example, would be smiling back at us not because she is imitating our emotions, but because her “digital dopamine” makes her “happy”.

A world where digital and living humans coexist

――How do you think digital humans, who are being accepted naturally beyond the so-called “uncanny valley,” will share roles with humans in the future?

Una：Today, digital people are beginning to play an active global role in medical care, education, human resources, real estate, financial services, retail, etc. We spoke about some of our public sector use cases such as emergency responders: We also created a digital police officer that provides non-emergency services to the public 24 hours a day for the New Zealand Police. We have many examples with our corporate clients as well. The range of functionalities and use cases will expand even further in the future. As the needs of the world change, digital humans will adapt to those changes. However, we humans will still be needed to adjust the communication strategy and prepare digital people for those roles in our ever-changing society.

――As a result, humans can leave the tasks that can be simplified to digital humans.

Una：That’s right. Digital people are here to help by not only simplifying our tasks, but also giving humans time to focus on more important work like strategic and creative planning, often based on findings that organizations get by utilizing digital people. Digital people are not a replacement of human work, but a new way of coexistence. Our goal is to explore the future in which AI is humanized and humans and machines cooperate.

――It will be a natural scene in the not-too-distant future.

Una：You’re absolutely right. We are already coexisting with machines. A few decades ago, it was surreal to predict that we would commonly use laptops and smartphones on a daily basis like we do today. And now, it is not difficult to imagine the widespread coexistence with digital people in the future. Some might be concerned about it, but it is really up to humans to decide how we will utilize the power of virtual beings within our society and make them work into our advantage. Every tool depends on the stance and the mindset of the the user. Even if you give children the same instructions and toys to play with, they might use them differently, and their parents might perceive them differently as well. No matter how good the technology is, its impact always depends on the way it is utilized, dependent on the maturity and social acceptance of users. Our team is very well aware of that, that’s why we are conducting our research and development based on well-defined and highly set ethical and moral standards. Our mission is creating socially responsible and human-centered AI.

――Is the situation similar in Japan?

Una：Yes. Japan has been our first location outside of English-speaking countries, therefore, in a way, our digital people learned Japanese as their second language. Our digital brand ambassador, Sam, speaks Japanese fluently, and she has gotten great support from the local audience. Japan is a wonderful country with a unique hospitality, culture and personal interaction, but also increasing social challenges. Some of those can already be solved today by digital people without compromising Japanese cultural specifics.

In the “corona” and “after corona” world, it is not difficult to imagine that face-to-face communication among humans will change. Regardless of that, the value of digital humans will likely increase in Japan that faces aging population and labor force challenges. It’s time to consider how cooperation with machines can better humanity.