Overview

Our current extended reality worlds lack consistent synchronous interaction from users and therefore tend to feel empty. From open-ended conversations to decision-making, XRAgents are dynamic parts of your world that can serve as digital friends, clones, fictional characters, and more.

"Am I real?" the virtual human asks you- how do you respond? Do you say that it's not, and risk giving it an existential crisis, or do you feed into its delusion? Of course, none of these "humans" are real, but, when we see something with a face, something that seems like a person, we tend to anthropomorphize. We'll assign human-like characteristics to things that mimic the appearance of us, we can't help it.

The virtual worlds of tomorrow won't be complete without characters that have the dynamic interaction capacity that human beings naturally excel in- characters that can engage in sophisticated conversation, adding greater depth and believability to the extended realities that we'll be more frequently immersing ourselves inside of.

We wish to democratize the ability to create believable characters in virtual worlds. Characters that have emotional capacity, real-time high quality facial animations, we set a plan to develop a virtual humans platform that allows users to create, interact with, and share customizable digital agents. Agents of all kinds can be made with just a simple description. This can be used to give people access to high quality virtual teachers, philosophers, poets, scientists, the types of people from history and fiction, and even imagined unrealities of Lovecraft, Fifthworldproblems, or the SCP Foundation.

Goals

1. Create dynamic virtual agents with open-ended conversational ability

2. Compose a large library of pre-loaded characters with certain personalities and contexts

3. Develop a platform for easily creating, interacting with, and sharing your own characters with others online

4. Experiment with creative works involving XRAgents.

5. Collaborate with groups across industries to bring these XRAgents into real world applications.

Outcomes

1. We developed a proof-of-concept in which users can engage in an open-ended conversation with an AI-driven XRAgent.

2. We developed a platform for creating custom XRAgents in which users can describe the characteristics of the XRAgents and we integrate this knowledge into the prompting of the GPT3 model that drives the agent.

How We Built It

The creator platform for XRAgents was built with the Streamlit web framework, in which user-inputted XRAgent names and descriptions are stored in a database, loaded, and queried as needed. The conversational interaction is driven by GPT3. The rendering and animation of the hyperrealistic 3D avatar is accomplished using an automation of NVIDIA’s Metahuman platform, in combination with Azure’s TTS service for producing audio from GPT3-generated text.

Future Plans

We plan to pilot the technology we developed in various industries, including to serve as AI tutors for universities and to serve as service assistants for businesses.

Philosophical Considerations

The word "avatar" is derived from the Sanskrit word avatāra meaning “descent”. Within Hinduism, it signifies the material appearance or incarnation of a powerful deity, goddess or spirit on Earth. What are the implications of a synthetic intelligence that starts to have the ability to weave seamlessly into our own linguistic patterns? If we can create these interactive personalities, would we make characters that can urge us to become better people, or will we mostly use it to entertain ourselves at the odd uncanny things that these entities we create will say? Will people start assigning true agency and humanity to these entities as they get to know them more deeply, or will they not be considered human? Can these entities increase empathy in people for those they don't understand, by having a chance to have a risk-free conversation?

What if you could make a version of yourself? Perhaps it might allow you to self-reflect better, and to see yourself how other people see you. This might allow people to engage in practices of "metacognition", or "thinking about thinking" in a more intuitive way. It's hard to lie to yourself if you're looking right at yourself. Could we make it your voice too? It might be a bit strange, uncanny, bizarre, but the future is never constrained to our own beliefs.

Can you collaborate with these entities to help in your work, or for idea generation? Perhaps it could increase the rate of technological innovation. The endless capacities of general language agents and the recent progress in that will only make these agents increasingly more sophisticated.

And perhaps one day, they will too have physical bodies, and then what shall these digital agents want to dream up and construct? Even if we give something a seed "personality", that doesn't predict everything about what it will do or say.

Of course, we're nowhere near general intelligence, but this gives us a glimpse of what it might be like one day in the future.

There are many unknowns, and it's exciting to be able to face it head-on.