Perceptual Foundations of Augmented Reality

Definition Perception: The top-down way our brains organize and interpret information and put it into context.

Definition Sensation: The bottom-up process by which our sense, like vision, hearing and touch, receive and relay outside stimuli.

Visual Perception

What is Perception?

Perception consists from three main processes called recognition, organizing, and interpreting. Recognition happens basically every day, we are aware of everything, such as sounds, lights, and colours. Then you are organizing everything that you perceive. Interpretation refers to the process by which we represent and understand stimuli that affect us.

Recognizing + Organizing+ Interpreting = Sensory Information

What is Visual Perception?

Visual perception is the end of product of vision.

“Perception is not something that happens to us, or in us. It is something we do… Vision is a mode of exploration of the environment drawing on implicit understanding of sensorimotor regularities”

Optical Illusion Highlighting Visual Perception

How Vision Works?

The eye passes through the cornea, and go down to the iris. The light reflects and sort of vision is also a chemical reaction.

Preattentive Features

Vision helps us to understand the distinctive features of the environment.

Gestalt Theory

Figure and Ground explains how we put different elements together to make one scene or a whole image.

Similarity states that things which share visual characteristics such as shape, size, colour, texture, value or orientation will be seen as belonging together.

States that things which share visual characteristics such as shape, size, colour, texture, value or orientation will be seen as belonging together.

Proximity Elements tend to be perceived as aggregated into group if they are near each other.

Common Fate: Objects which are facing the same direction or appear to be travelling in the same direction are usually grouped together.

Image of No.10

Continuity: In order to fill in missing data we often see things as continuous or whole.

Image of No.11

Closure: If we have a large pattern with missing components we tend to fill in the missing parts to create the image we actually see.

Image of No.12

Area: When areas are overlapping, the smallest area is seen as the figure and the larger is the ground. When we look at this object we see this as one object on top of another instead of a hole in the larger area.

Image of No.13

Symmetry: We tend to organize complex objects into a whole, we are more likely to group symmetrical objects.

Image of No.14

Vergence-accommodation conflict: There is a disparity between the physical surface of the screen – accommodation - and the focal point of the simulated world you’re staring at - vergence.

Image of No.15

Spatial Audio

Detecting a weak sensory signal like a random clap in daily life isn’t only about the strength of the stimulus. It is also about the psychological state, your alertness and expectations in the moment.

SIGNAL DETECTION THEORY

Signal detection theory is a model for predicting how and when a person will detect weak stimuli, partly based on a context. Our hearing is adjusted by our preferences by our personal perceptions that is valuable for us.

The paranoid parent’s brains are so trained on their baby, it gives their senses a sort of boosted ability, but only in relation to the subject of their attention.

Image of No.16

Touch

PROPRIOCEPTION

Proprioception is the perception of the world with our body, so how perceive and understand things around us. our perception embodied and active when we see, we are ready touch, and we tend to visualize our path to the object. Our body experienced all directions of the environment, we understand the space in 3D. For example, when we see something that we need to grab, we will understand it is a shape.

Image of No.17

UI

UX (User experience) and UI (User interface) is different. UI can deal with traditional concepts like visual design elements such as colours and typography. UX is all below user interface. User experience design is a human-first way of designing products. Don Norman, a cognitive scientist and co-founder of the Nielsen Norman Group Design Consultancy, is credited with coining the term “user experience.

Image of No.18(1)

Today UI design is mainly focused on the 2D screens. We are living in a 2D world, everyone has a phone, and we use to all references to understand what the camera means, what is this icon.

However, AR (Augmented Reality) points to use all the sense to move away from the screen. In the 60s, the person called Douglas Engelbart, he invented the modern mouse, but the aspect of that not invent something that will constantly use, it was about inventing augment human intelligence that could help us to experience our world in more senses. He wrote articles about augmenting human intelligence in using all senses that we use our bodies, speech to work with technology.

In 1968, Ivan Sutherland who is very famous founder of VR and MR, he said “if the task of the display is to serve as a looking glass into the mathematical wonderland constructed in computer memory, it should serve as many senses as possible”

Image of No.18(4)

In terms of the spatial holograms and spatial interactions, it shows that objects are really realistic that we can work with.

Image of No.18(5)

Affordances

Affordances are the possible actions. Donald Norman (1988) said “possibilities for action that are readily perceivable by individuals”. He invented the storm and created the notion of affordances that means projects represent their functions at 3D world.

UI in AR

The lateral field of human view is about 140 degrees, but with the peripheral visual is 200-220 degrees compared to peripheral vision of the body. We need to follow the natural ways of perception, and don’t put objects on the back bunny.

Image of No.19(1)

The Hierarchy of Needs in AR

Image of No.19(2)

Immersion and exploration

user is the centre of the environment
users will want to try their own ideas
scale makes a huge impact on presence
small details matter

Attention

users have freedom to look anywhere, so capturing and guiding attention is important
audio and visual cues help nudge users in the right way
forced attention – not always a good idea

UI TOOLBOX

We use our gaze (vision) as the way we see things around; we use gaze cursor as the metaphor of the mouse cursor on the computer; Voice can activate different voice commands; We can use spatial audio as 3D emergent audio (an immersion sound) to guide people to show how sounds moved where an object is located by positioning 3D sound in different places; Gestures is how we can use gaze to activate and trigger with these holograms.

Note: when you create UI, you need to know:

-Ergonomics -Safety guidelines for the content

Clicker, laser pointer -Other people and their voice commands

Text

— avoid large amounts of text to instruct of inform users with information within an augmented environment

— consider how you will draw attention to text within your environment to capture the user’s attention

Colours

— Shadows and lighting may change the way colour appears on objects and make things.

— By modifying the saturation and brightness of a single hue, you can generate multiple colours— darks, lights, backgrounds, accents, eye-catchers— but it’s not overwhelming on the eye.

— darker colours tend to carry more visual weight, and these kinds of elements need to be balanced out with lighter colours.

The shade of the centre dot is the same in all the squares

The shade of the background influences how we perceive it. All squares are uniformly shaded, but each square lighter on its left edge than on its right edge.

Storytelling

Storytelling comes from writers, it can be histories, stories of legends.

Storyboarding

Storyboard is the collection of the shots of the images for movies that will be created.

But in 3D, the simplest way is to go to the 3D modelling application, such as Unity, Maya, and Cinema 4D, to create a story using prefabs, the 3D models of objects, and explain what is going to happen with the launch examination, and how the user is going to interact with objects.

Map your story

This will be an inherently interdisciplinary endeavour, and it will need contributions from designers, data engineers and visualizers, brain scientists, 3D programmers, mechanical engineers, materials scientists, artists, storytellers, and others.

Uncanny Valley

You need to find that level of the character that will work in AR, and also try to find the balance between how real your character is going to be, and how empathic you want to person feels of that character. Don’t fall down about your character.

Image of No.22(3)

Summary

AR UI is quite different from the traditional UI and requires new ways of approaching the challenge of informing and empowering users.