Human-Computer Interactoin for Mixed Reality

HomeWork1:

Task:

Watch the video called Hyper Reality (https://www.youtube.com/watch?v=YJg02ivYzSs ). Hyper Reality presents a dystopian future using Mixed Reality technology. It consists of multiple scenarios in which potential dangers are presented. Come up and present TWO additional scenarios from our daily life in which Mixed Reality technology can create problems for the user or can be abused by content creators, authorities, users or other entities. Elaborate what makes this scenario abusive and reflect upon techniques how we could avoid this trajectory. Present your scenarios in text AND visuals. This means that you have to explain the scenario and either create sketches, take screenshots from the internet or even present an own short video.

Answer:

First scenario: The evolution of scammers

In order to enhance the user experience, the mixed reality technology could need access to loads of confidential information about its user. The great side about this feature is a perfectly tailored experience for the user but unfortunately it has a dangerous downside. In fact Hackers are evolving with each new piece of technology coming to the market. Meaning that the mixed reality technology won’t be entirely protected. And even if we pour a lot of resources into making it safe, someone can make use of a fault in the system and turn the whole technology to his advantage. “Phishing” is a well known cyber attack that consists of sending fraudulent messages planned to trick a human into giving up his sensitive information or deploying malwares in his system. With the immersion brought by the mixed reality technology phishing techniques can evolve into something more dangerous than emails, text or websites. Scammers can design a mixed reality game in a public space, where they would play a game of guessing like guessing in which cup the ball is for money. The victim would have in front of him a really persuasive scene where he can see other players that are part of the scam, winning easy money with flashy visuals and convincing sounds. The victim in this case will think that the game is easy and will pay the entry fee using his electronic wallet. Little did he know that before that game was over, the scammers would have his credentials and steal his money.

How to avoid this situation?

In order to avoid these kinds of situations, you should always pay attention to these types of games even in real life. Besides, with the evolution of mixed reality technology scientists should be also enhancing the user’s protection parallely. For the scenario given before, I would say that the user should have an e-wallet linked only to that single device. And it would be separated from his personal bank account. Unfortunately in this case, the mixed reality’s experience can be disturbed because the user should always repeatedly transfer money to it but it is a necessary action to avoid thefts.

Second scenario: Cyber attacks

The idea is inspired by the announcement of metaverse by Mark Zurkerberg a couple of months ago. Mark has shown some cool features of the virtual universe that meta aims to create and one of them was the AR grafiti. In case this cool piece of art becomes popular in the future, hackers can use the abundance of this art to dissimulate their tricks to hack people’s systems. And what makes it more dangerous is the fact this digital art is shareable between users.

How to avoid this situation?

In order to avoid this kind of situation, I would say that Meta should put a lot of effort into making this feature secure for the user. On the other hand, the user should be also vigilant to this kind of art, maybe by not starting to scan everything and be aware of any suspicious signs?

HomeWork2:

Task:

Read the “Ultimate Display” and discuss how Ivan Sutherland’s vision of the Ultimate Display fits into the Reality-Virtuality Continuum of Milgram. Is it a discrete point ? Is it an area ? Is it inside the continuum or is it even surrounding the continuum ?

Answer:

In The virtual reality continuum, Milgram separated the real and the virtual environment. In this context, the mixed reality would be a mix between these two entities. In this continuum we find augmented reality which is closer to the real environment: which consists in blending the technology with real world objects. Whereas on the other side of the continuum we find augmented virtuality which is the representation of real objects and their states in media. Personally, I find that the ultimate display described by Ivan Sutherland can't fit entirely in Milgram’s continuum. The idea behind Milgram’s continuum of virtual reality is based on a tradeoff between real environment and virtual environment. But Ivan’s vision describes the ultimate display as a fusion between these two.Meaning that the technology would control the matter of the element displayed in the virtual reality and its corresponding object in the real world. Finally, I would say that this display should be surrounding this continuum.

HomeWork3:

Task:

Think about THREE alternative ways that we could implement locomotion. You should focus on the following questions: 1. What is the goal of your locomotion (e.g., be the fastest, be the most enjoyable, ...) ? 2. How does your locomotion system work ? (e.g., what is the metaphor for the user and how do you want to implement it) ? 3. How do you want to evaluate if you were able to achieve your "goal" ? (e.g., what do you want to compare against, what metrics do you want to collect and how ...)

Answer:

a) The idea

The origin of this idea is simply the roller skating sport. This locomotion technique aims to be the most enjoyable with delivering a good sense of presence.

b) How does the system work:

With this locomotion technique the user needs only to mimic the arm gestures shown in the video above. I will explain in the coming part:

Sliding movement: The relative difference(w.r.t an initial position) of elevation in each of the controllers will contribute to the speed of the locomotion and will also determine the direction of the movement. The main movement will be a smooth translation as if the user is really roller skating. For this purpose, a physics system integrated in unity might be very useful.

Speed boost: The user can choose to give the locomotion a speed boost by crouching while moving his controllers as described in the main movement part.

Stopping: Stopping with this locomotion technique will need the user to be crouching while keeping his controllers still.

b) How to evaluate this technique:

This locomotion technique is designed to enhance the speed metric compared to other locomotion techniques. In order to evaluate this aspect, I thought about comparing it to any teleportation technique. Being widely used in the market and being a fast locomotion, teleportation can act like a ground truth for our surveys to measure the speed of the roller skating technique. The only thing that can make the study incorrect is the fact that the roller skater, as opposed to the teleportation, needs a good amount of physical effort from the user side. To remedy this, we can maybe make the experiences a bit short with a large number of users.

a) The idea

The user will move simply by levitating his body and moving towards his goal. This locomotion technique is designed to be precise and enjoyable at the same time.

b) How does the system work:

The system will be implemented using unity and built for the oculus quest 1 or 2.

This locomotion technique will be based on the hand tracking feature provided by the oculus.

Here a the suggested hand gestures for controlling the locomotion tecnhniques I used only pictures for clarity reasons:

b) How to evaluate this technique:

This technique is designed for accuracy and speed. I find that evaluating a technique that tries to enhance both of these metrics is a little tricky. Because there is also a trade between the two terms. The only experiment that I can think of to evaluate them simultaneously is to create an infinite parkour with collectibles. In the experiment, each user will start with a fixed period of time and with each collectible earned will increase the time remaining.

a) The idea

As you can see in the picture above the idea came from the tomb raider game, where the player can overcome obstacles by hitting a certain spot with his arrow and swing or be pulled to the goal position.

b) How does the system work:

Bow mechanism: The user will handle the oculus’ controllers as a real bow, the left controller will be used to aim at a target and the right one will be used to determine the power of the pull of the arrow

Two types of cord: Standard cord: This cord will be used whenever the user has a certain elevation of terrain in front of him and will use it for climbing or swing between ledges. Bungee jump cord: This cord will be used for horizontal movements, the elasticity of this rope will be used to throw the player’s body into a certain position. → After the player throws the bungee jump cord, he will have the choice to either be pulled to that point or attempt another throw.

b) How to evaluate this technique:

This technique is designed to boost the fun metric of a locomotion. To evaluate such a technique, I would suggest making a survey with a unique experiment. The user will be put in an environment with obstacles to overcome. The experiment will be mainly an exploration game. And the measure will be based on how much time the user will spend playing it. I suggest also comparing it to any popular exploration game like “ the climb”.