MirrorVerse

MCU

role

UI/UX Designer, Developer

user research
prototyping
wireframing
interface design
testing

Team

Jiahui Liu

Date

Jan - August , 2023

MirrorVerse combines ChatGPT's conversational abilities with MetaHuman Creator's realistic avatars to create a unique virtual platform for self-interaction and reflection. This platform allows for genuine dialogue and emotional feedback with a lifelike virtual version of yourself. Engage in authentic conversations where the virtual "you" offers comfort, advice, and diverse responses, enhancing self-engagement beyond real-life limitations.This is my graduation design project, and I wanted to challenge myself with something entirely new. Inspired by the rise of ChatGPT, I wondered if I could use it for self-healing purposes.

Tools v& Method

Figma, UE5, Photoshop, ChatGPT, Metahuman, FaceGenModeller, Azure Speech

I used Lean UX methods to quickly create prototypes, and repeatedly tested prototypes to improve my design

Challenge

The challenges I encountered were primarily about improving ChatGPT's ability to simulate the user's tone more accurately, making the interaction feel more authentic. Additionally, I had to consider whether the uncanny valley effect would cause users to feel uneasy about their 3D digital image.

Background

The Proteus Effect

Research shows that digital avatars can influence offline behavior through the Proteus effect, where changes in one's online persona impact self-disclosure and negotiating skills.

Have you ever talked to yourself in a mirror?

Positive self-talk enhances attention, mental state, and self-perception. While traditional mental health support is well-studied, new technologies offer fresh opportunities. Tools like the Portable Interactive and Meditative Mirror (PIMM) bring these practices into the digital realm, collecting emotional data to manage emotions more effectively.

Technical Study

ChatGPT EA ability & Role-playing ability

Conversational agents like ChatGPT hold significant potential for mental health applications. Research is assessing its emotional awareness (EA) capabilities, making it a potential tool for cognitive training and psychiatric diagnostics. Using natural language processing and large text data, ChatGPT generates human-like responses and simulates role-based interactions, creating engaging and immersive experiences.

MetaHuman is a complete framework that gives anyone the power to create, animate, and use highly realistic digital human characters in any way imaginable.

MetaHuman

What’s the Goal?

To combine MetaHuman and ChatGPT to train it to represent ourselves and create a space that allows us to engage in self-talk and interaction in the virtual world. This innovative approach has the potential to help us achieve a deeper level of self-reflection, emotional well-being, and reduce negativity.

Why I choose lean UX method?

THINK

MAKE

TEST

With limited time and budget, Lean UX offers a collaborative, iterative approach to quickly improve projects through a cycle of thinking, making, and testing.

01 THINK

what are the problems?

I wrote down some Assumptions from both a business perspective and user’s perspective to help me clearly define the problem I was trying to solve. I list part of the assumptions in the table below

part 1: users

01-Who is the user?
Individuals seeking mindfulness and self-awareness.
People undergoing therapy or self-help routines.
Students preparing for presentations
02-What problems does the Product solve for the user?
Get some encouragement, inspiration, or help them to realise a side of themselves that they don’t realise
03-In what context does the user use the Product?
When they want to have a pep talk before a competition
When they feel down and want to talk to them self to cheer them up
When they try to engage in self-reflection by talking to themselves

part 2: Business

01- The customer needs to engage in introspective dialogue and self-reflection in a safe and private environment.
02- The customer wants a realistic and interactive experience with their digital avatar, intuitive user interface, and authentic and interactive experiences from our service.
03- I will get my customer base by targeting online personal development communities, marketing on professional training platforms, and through social media campaigns.
04- My primary competition is AI-driven therapy bots, and virtual communication training platforms.
05- We will beat the competition by offering a unique experience of talking to one's own self, integrating high-level AI for emotional intelligence, and ensuring utmost data privacy.

04- What features are essential for the user? And why?
Virtual avatar-Generate an avatar that resembles the user
Online healing space-Users can have private conversations with their avatars
Record/Delete conversations - Users can review or delete their conversations at any time
05-How should the product look and behave?
Make the user feel relaxed, or they want to feel futuristic

How to resolve problems?

According to those assumptions I did some Brain Writing to help me to think of some solutions. I did two rounds. The first round is a direct idea in my head, and the second round is an improvement based on the ideas of the first round

Round 1: Initial Ideas

Create a digital avatar from user photos or scanned image.
Voice tests to enable avatars to mimic user voices.
AI role-play for behavioral training of the avatar.
Advanced customization for avatar appearance and voice.
Provide different contexts for conversations.
AI facial expression recognition to adjust avatar responses.
Use as a supplementary tool in therapy.
Offline access to saved conversations.
User-generated scenarios, letting the community create and share their templates.
Invite friends to interact with each other’s digital avatar.

Round 2: Idea Development

Multi-angle photo guidance for realistic avatars.
Customisation options for the voice of digital avatars.
Provide guidance, prompts and templates for training the AI for role-playing.
Enhanced appearance customization: skin tone, hairstyles, clothing.
Provide different tips for starting a conversation e.g.: job interviews, or pep talks.
Avatars can do more than just mimic the user’s facial expression.
Therapeutic subscription packages.
Can share the saved conversations with others.
Rating and rewards for top community scenarios.
A network where you can interact with friend’s avatars when they provide consent.

I then compiled all these ideas into the following Assumptions Mapping table to help me better understand which problems are most important and need to be solved first

unlikely

significant

minimal

likely

Advanced customization for avatar appearance and voice.

Voice tests to enable avatars to mimic user voices.

AI facial expression recognition to adjust avatar responses.

Customisation options for the voice of digital avatars.

immerse yourself in virtual games

invite friends to interact with each other’s digital avatar.

offline access to saved conversations.

set interactive scene/ background

highlight the useful information the avatar provided

Create a digital avatar from user photos or scanned image.

AI role-play for behavioral training of the avatar.

Provide guidance, prompts and templates on training the AI for role-playing.

Multi-angle photo guidance for realistic avatars.

Provide different contexts for conversations.

Provide different tips for starting a conversation eg: job interview, pep talks.

User-generated scenarios, letting the community create and share their templates.

Use as a supplementary tool in therapy.

A network where you can interact with friend’s avatar when they provide consent.

Can share the saved conversations to others.

Rating and rewards for top community scenarios.

Therapeutic subscription packages.

Enhanced appearance customization: skin tone, hairstyles, clothing.

Avatars can do more than just mimic the user’s facial expression.

persona

AMANDA

28, London
Marketing Professional

FRUSTRATIONS

1.Skeptical about the efficacy and realism of AI-based tools.
2.Concerned about the privacy of her conversations.
3.Unsure about how this could integrate with her existing self-improvement routines.

QUOTES

"I need a tool that allows me to practice conversations safely but realistically. It should be customizable and versatile enough to simulate different scenarios I might encounter."

MOTIVATIONS

1.Wants to improve her communication skills for professional growth.
2.Needs a platform to practice conversations for various situations, like job interviews and negotiations.
3.Interested in self-reflection and personal growth.

GOALS

1.An easy-to-use platform with a focus on privacy.
2.Advanced customization options for creating a realistic digital avatar.
3.4.Avatar customization based on her own photographs and voice.
5.AI role-play features to train the digital avatar's behavior.
6.Offline saving and secure access to her past conversations.
7.Different scenarios for practice, like job interviews, and family conflicts.

User journey map

Stages

Storyboard

Before using the product

Engaging with the product

Reflection and Advocacy

Quotes

"I wish there was a way to practice conversations and self-reflect."

"Let’s see what this platform can offer"

"Creating a realistic avatar could be really cool, but also a bit creepy"

“Let’s get started!”

"It’s really useful to see a different perspective of myself"

"I should tell more people about this."

Emotions

Pain points

When talking to yourself in the mirror, your actions are only reflected back at you.

Lengthy sign-up, privacy concerns.

The complicated process could confuse users.

Don't know how to start a conversation

Want to review the conversations at some point

Limited community engagement.

Opportunities

The AI technology can not only mimc user behaviour, but also react and provide insight/emotional support.

Quick sign-up process, robust privacy policies.

A clear step-by-step process guides users to create a digital image of themselves.

Provide users tips to start a conversation

Save conversations
Highlight the useful information

Interact with user’s friends' digital avatar
create a community

02 MAKE

After I identify assumptions that need to be tested and validated. I decided to build an MVP prototype that would allow testers to interact. This prototype can interact with their avatars in a text-based or voice-based conversational interface.

Interactive prototype

Step 1: Generate your own digital image

I started by generating my own 3D model from the photos in FaceGenModeller, then imported my assets into Unreal Engine to set up the skeleton and rigging for the model. And customised the skin textures as well as tweaked the look.

Step 2: Accessing ChatGPT and AzSpeech in UE5

Accessing ChatGPT and Azure Speech in UE5 allows me to have real-time interaction with my models; then I let the voice drive my model to speak and redirected my character animation

wireframes

03 TEST

For the initial test, I recruited two participants and asked them to provide photos that were used to create their digital avatars. In addition, I asked each participant to write a paragraph about themselves. Using this information, I trained ChatGPT to mimic their speaking style and thought patterns. After the test, I conducted interviews to gather their insights.

Insights -Test round 1

1.  Participants felt the digital image resembled them but lacked detail.   Allowing adjustments to their avatar's appearance could increase satisfaction.
2.  They found the avatar's movements stiff and its speech expressions unnatural.   Improving contextual expressions, like frowning instead of smiling to show sympathy, would enhance the experience.
3.  Sound quality needs improvement, such as training the avatar to replicate the user's timbre and accent accurately.
4. Participants noted vague answers due to insufficient data and misinterpretations.   Providing templates and guidelines could help users give more detailed information, improving ChatGPT's replication of their thoughts and tone.
5.  Participants sometimes struggled to maintain the conversation.   Offering chat prompts and having ChatGPT ask questions could make interactions more natural.

Due to time constraints, I focused on training ChatGPT to better represent participants' thoughts rather than perfecting the avatar's look and sound. I provided participants with a template to input richer information and modified prompts to have ChatGPT ask questions instead of just answering or repeating information.

Insights -Test round 2

1.   Alter the instructions being given to ChatGPT so that it does not recycle or repeat the information provided in such a unrealistic way.
2.   Provide more specific instructions to ChatGPT to not answer questions in such a robotic way.   It should ideally not offer answers in a list.   A friend in conversation would not respond to questions in such a logical way.   Instead, it can be given directions to answer more freely and conversationally.
3.   Provide further instruction to ChatGPT intially to have it overall respond in a more emotional manner.   For example, if the user is acting aggressive towards the digital avatar, it can react in turn aggressively, or become offended.   Similarly, if the user is sad, the digital avatar can react sympathetically or with a counteracting emotion that could try to improve the mood of the user.

Due to time limitation, I mainly focus on the test of prototype instead of the test of the entire project process. The test includes whether the digital image meets the user's expectation or whether it arouses the uncanny feeling; And whether ChatGPT can perfectly reproduce a user's tone of voice or thoughts.

04 final design

final lean uX canvas

Business Problem

Users need a safe space to externalise and reflect on their thoughts without judgment.
People often struggle to process their feelings and thoughts internally
They want the technology to help them to have realistic conversations with themselves.

Solution Ideas

Business Outcomes

A clear step-by-step process guides users to create a digital image of themselves.
Advanced customisation options like hairstyle, eye color
Provide guidance, prompts and templates on training the AI for role-playing.
Text-based or voice-based conversation interface.
Questions templates or tips for users who are unsure how to start.
a community that allow user’s to Interact with their friend’s avatar
User-generated scenarios, letting the community create and share their templates.
Secure storage for past conversations with encryption to ensure privacy.

Make the process of creating a digital avatar easier for users
Relieve uncanny feelings with their digital avatar
Reduce user’s mental interaction cost
Increase user engagement
Retain users

Users & Customers

Individuals seeking mindfulness and self-awareness.
People undergoing therapy or self-help routines.
Anyone who needs a space for introspection or decision-making.

Users Benefits

An easy-to-use platform with a focus on privacy.
Creating a realistic digital avatar.
AI role-play features to train the digital avatar's behavior.
Offline saving and secure access to her past conversations.
Different scenarios for practice, like job interviews, and family conflicts.

Hypotheses

1.We believe that convenience will be achieved if user creates a realistic avatar with a clear step-by-step process that guides users to create a digital image of themselves.
2.We believe that the lessening of uncanny feelings with the digital avatar will be achieved if user can create a realistic digital avatar with advanced customisation options.
3.We believe that reducing mental interaction cost will be achieved if the user can get a realistic avatar with guidance, prompts and templates on training the AI for role-playing.
4.We believe that increasing engagement will be achieved if user can interact with their avatar with text-based or voice-based conversation interface.
5.We believe that increasing engagement will be achieved if the user can create or use the scenarios templates with the user-generated community.
6.We believe that saving user’s time will be achieved if user starts a conversation with tips or some questions templates.
7.We believe that retaining users will be achieved if user can check their conversations with secure storage for past conversations.

What’s the most important thing we need to learn first?

What’s the least amount of work we need to do to learn the next most important thing?

The Uncanny Valley effect may upset some users.
The platform can be misinterpreted as a substitute for professional therapy or counseling.
Given the sensitive nature of the content, there is a risk of data breach.
Ai cannot fully simulate human emotions to produce a deeper level of empathy

Understand the deep needs of users through user interviews
Create an MVP and run an experiment to test it
Get feedback from participants and keep optimising your mvp for multiple tests
Complete your prototype design