Assignment 2

Immersive Technology, Computer Vision and
Intelligence for Social Justice

In our first lecture we’ve seen how VR can be used for education, medicine, where it can be used for training in surgeries, for example and for social justice, especially in the forensic field. We talked about Forensic Architecture, which is an agency based at Goldsmiths, University of London, investigating human rights violations, including violence committed by states, police forces, militaries, and corporations. It refers to the production and presentation of architectural evidence relating to buildings, urban environments—within legal and political processes.

Their work involves open-source investigation, the construction of digital and physical models, 3D animation, virtual reality environments and cartographic platforms.

Architectural analysis and digital modelling techniques enable to unravel that complexity, and to present information in a convincing, precise, and accessible manner–qualities which are crucial to the pursuit of accountability.

source: https://forensic-architecture.org/

Introduction to gender, development and ICTs

This reading made me understand more and more how the tech industry is still highly male-dominated and because of this, most of the contents, apps and tools are designed by men for men, creating a big gap in unequality among women and men. Following a few statistics on the paper, in the UK only 12% of the jobs in tech are held by women, while in the US is 25%.
This situation is the result of many factors, like geographic location which has a dominant influence on gender difference for internet/mobile access.
As it always has been internet can be an amazing tool if used with a positive purpose, but on the other hand can be a hate device, able to spread anger and negativity quickly. An example can be how both ICTs (information and communication technologies) helps give voice to feminists and minorities, and on the other side, misogynist groups can use online resources to spread hate.
That’s been the reason why there are still a lot of people that are against new technologies because the is still a huge slice of people that take advantage of ICTs for bad purposes, instead of building something useful that can help merge minorities and people in difficulty with technologies and improve.

source: https://www.tandfonline.com/doi/full/10.1080/13552074.2018.1489952?scroll=top&needAccess=true

Coded Bias

Coded Bias on Netflix takes exactly about this issue, how AI is based on some Date and algorithms that identify more easily men and light-skinned individuals (this when talking about facial recognition). Cathy O’Neill defines algorithms as ” using historical information to make predictions about the future”, therefore if the notions and information we give to technologies and computers are from the past, consequentially is unavoidable that we encounter this kind of sexist and racist episodes in software. Further examples of discrimination by AI software can be the Amazon one, where an AI software was used to scan and analyze resumes and it ended up rejecting all women applicants. Another example that involves in this case racism by AI is the one that regards United Health Group, where their main algorithm was prioritizing healthier white patients, over sicker black ones.
AI has so much potential, but if the information that built the data behind it is not updated these days it can just be destructive. UK police started using facial recognition without a legal basis, by putting cameras in public spaces that could recognize peoples’ faces and associate them with potential wanted criminals, the problem is that this software is inaccurate, leading to unhappy mistakes where the machines associate criminal profiles of innocent people walking by.

source: https://www.netflix.com/gb/title/81328723

Visual Storytellig

Shapes, Propts, Lines and Colour

Visual storytelling is how the story/narrative is told through the use of shapes, colour and prompts and what they communicate to the audience. In VR, these elements are fundamental since it is an immersive technology, where the visual aesthetic plays the principal role, environment and its colour and shapes are the first things we experience when wearing the headset for the first time. Our brain is almost overwhelmed by visual elements, therefore these must be well thought and have a specific aim meaning in the scene, otherwise, the user might just get distracted by them and not keep the attention on the actual narrative.

When taliking about shapes, these can be sintetized in the three basic ones:

humans have been giving meanings to shapes since the antiquity, associating them to things they experiences throughout the day

triangle: fear, distrust, suspition

square: manmaid, stability, strenght, oldfashioned

circle: completeness, balance, maternity, endlessness

________________________________________________

Horizontal lines have more or less the same meaning as squares, they remind of the horizon, so stability, it is also considerable as a leading line, which are those lines that our eyes follow in the landscape that take to the main elements in the scene

| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

Vertical lines rimind of power, but if repeated they make you think about prison, being imprisoned

Colours as well assume a huge meaning in the environment and how the audience perceives them. I think that visual elements in VR are fundamental and they are really important for narrative comprehension. When immersed our brain is completely focused on our sight, in discovering the “new environment” around us, therefore it doesn’t pay a lot of attention to a voice narrating for example. That’s why visual elements are so important because, if effective they can almost replace voices or any kind of language communication. Visual elements are such a great tool because are discreet, but at the same time powerful, and our brain detects them almost automatically.

Virtual Reality lets developers and artists play with hue, values, saturation and brightness, creating both unrealistic and realistic coloured environments, making people able to me part of them and interact with the surroundings. It is not only an aesthetical choice, but it let players be engaged both on their emotional side.

Color is a powerful communication medium, that has been used through the years between film, television, 2D gaming and virtual and augmented reality to create interesting and engaging scenarios to keep the audience attention high and curious. If we look into films, it can be noticed that colour is not only used passively for the environment or object, but it can be used as a signal, for example, when a character changes suddenly his/her attitude towards the narrative, directors typically make their characters change clothes of take of a particular piece of clothing, revealing a new colour. Also, in VR color can be used actively, for example, could be used to change scene smoothly, by changing its value, or if staying in the same scene, it can be used to focus the attention to another point of view.

Prompts are part of the visual storytelling, they help at the scene, by saying something more about the characters or the time in which the experience is. There are different types of it like personal, which belong to the character and say something about the personality, or practical prompts that do not necessarily belong to the narrative but are useful for particular moments in the scene. More or less almost everything in the scene can be used as a prompt, and colour is one of them.

Light

Light is highly related to color because it can change its perception, by varying the value, making colours darker or brighter, this can also arouse a series of emotions. But light also has its meanings in storytelling, for example, by nature, light tells which part of the day is, like morning or evening, and people usually associate to different periods of the day, different feelings like a bright morning can be felt like positive, full of energy, while evenings have that romantic/nostalgic atmosphere. Light in cinematography has a major role, because it tells emotions, focus the attention on particular characters and reveals their intention or doubts in the scene. I think that in VR light is important, but not as much as in films, because since BR is 360° experience, we can’t have specific shorts because the main camera is our eyes, it is more an environment light, therefore I suppose that the colors are much more significant to express emotions and moods. Maybe a good use of light in VR is when used as a prompt, like a practical prompt, for example, illuminates a specific object or character in the scene.

Games as a cheating lesson

In this lecture we focused on the first part about Google Maps, and how much trust we give to it. Firstly we thought about when you go on holiday or in a new place, what is the first thing you do? Is checking google maps to know where to go if we have an address for example. But the question is, how truly reliable is, is it 100% mirroring the reality?

That’s when our certainties started becoming doubts, we used as an example Palestine, which is a country, but on Google Maps its borders are not the same ones as the other countries’ borders, as is not a “full country”. This lecture can be compared to Coded Bias, because they talk almost about the same thing, at least we encounter the same doubts. All this internet web site and softwares are based on Data, and as I’ve already said, data is “using historical information”, and this data is storaged by people. Therefore is like those people in charge of the data are making the software and are reasonsabe on “how the software thinks and act” because it is based on the data they had. And if we keep analyzing the few people that insert data have a specific background, gender, ethnicity and so on, in consequence there might be sexist and racist episodes regarding AI softwares.

Politics of Immersive and the Mediated Reality

If by real we mean how we perceive the world through our 5 senses. If we define VR as a computer generated 360 experience, Virtual Reality and the Physical reality are the two extremes, and in the middle we can find mixed reality, cinema, augmented reality, theatre which is closer to physical reality than cinema because is a life performance, where is some cases it can be interactive, and news, which are not part of the physical reality, because are always mediated, to make them more interesting, therefore by mediated reality we mean when the information before coming to you go through a media. However, even Physical reality can be or is mediated, an example can be constructivism, where people usually try to create their own vision of the world, which is built on what they perceive from news, media, social media, school, our senses and I think that geographic location has a big impact on human perception of the reality. Just think about memories, for example, when talking about a shared memory with someone and they don’t recall few facts or you have remember different things for the same memory, that is how we all build without even noticing own personal model of the reality. Therefore, if even physical reality is mediated, we can say that everything is a representation.

Jean Baudrillard Hyperreality

According to Jean Baudrillard, we live in an age where everything is a representation, a virtual representation, virtual representation of ourselves, for example, he says everything is a simulacrum, not real and he calls this Hyperreality. In the physical reality we can encounter loads of simulacrum, during the lecture we talked about a Sherlock Holmes house in London, or Harry Potter studios, this is simulacrum examples because this character or places are fictional, they were not real people but a physical representation of the house/streets were done. Even with social media, we people have a representation of their selves, for example on Instagram there is a fashion influencer called @lilmiquela, which is a computer generated avatar, that looks aesthetically really close to a real person, that posts fashion tips/outfits and so on, and pictures with real people. Of course behind her profile there is a group of people managing the account making it “look” as real as possible, creating a simulacrum.

Digital Wonderland

Digital Wonderland is a platform created in 2002 by Marie Laure Ryan, describing it as a “computer generated three-dimensional landscape in which we would experience an expansion of our physical and sensory powers, leaving our body and see ourselves from the outside”.

This concept is really close to VR Chat, where people interact, built relationships, share emotions and feeling, true emotion and feelings on a VR platform. This can make us think that at the end VR is not that far from physical reality, because it can not be considered the opposite of if we can experience such emotions.

I personally wouldn’t consider VR the complete opposite of the “real world”, because despite is is computer generated graphic world, we still experience true sensations, feeling, we still have our own perception of it through our 5 senses, as we do in the physical world. A surreal environment, a landscape that doesn’t reflect exactly the reality won’t make the experience less real, if we still can feel and experience emotions through it. If we keep analyzing this, we can end talking about how companies try and try to built softwares that have a great definitions, looking almost real, but if our can’t feel any emotions, if you don’t feel immersed it all that effort to make it look real doesn’t lead to anything, and maybe a low poly VR game can be much more interesting compared to it.

360 Gaze

We talked about 360 panoramic paintings at the start of the lecture, seen as a second order reality, where the first order is the physical environment. The second order in this case is trying to convince the audience the painting reality, creating an immersive experience, in a live experience.

Romanticism and VR

Romanticism, artistic movement developed around the end of the 18th century is characterized by subjectivity and an emphasis on individualism, spontaneity, freedom from rules, most of all the environment reflects one’s emotion, and not the physical reality for what it exactly is.

This is a pretty well known paining from the Romanticism movement of Caspar D. Friedrich. The man stands upon a rocky precipice with his back to the viewer. He is wrapped in a dark green overcoat, and grips a walking stick in his right hand. His hair caught in a wind, the wanderer gazes out on a landscape covered in a thick sea of fog.

During the lecture we were asked to think about a Never Ending VR story based on this painting, where we could take inspiration from a book, film, story dream or whatever it was helpful.

I made a padlet because I felt it was visually easier for me to organize the ideas: https://artslondon.padlet.org/aghaffarian12201911/cwoqsrrsmt1hr1k2

How you can see from padlet,

• I took my inspiration from the film “Groundhog day, where the protagonist relives the same day every day, as he is stuck, where every day starts and end in the same way, and he has to find out how to get out of this neverending loop.
• Looking at the image I feel a nostalgic sensation, him looking in the horizon makes me feel like he is thinking about the past and what happened in his life and why he is now there by himself, to make it VR I thought that it should be interactive maybe, so the protagonist can exchange information /talk or interact with object and people in the scene, 360 environments, visual elements colours, prompts, light change.
• Every time the journey restarts there is a new little element that the player has to find through clues in the environment to find out step by step details about the story, in order to get free from the infinite loop the player is stuck in. It can be objects that unlock new scenarios or people to talk to.

Immersion and Theatre

The Theatre has existed since the ancient times, from the Greeks until nowadays and always tried to create immersion, for example through light, stage scenography, to create a sense of reality. But with this kind of theatre the audience is sitting and watching the play, creating a 4th wall, that limits the immersion. Lately from the 2000 theatre started to change, trying to create a fuller experience for the audience, trying to tear down the fourth wall, with performers trying to interact directly with the audience, or with the absence of a stage but with a more dynamic environment, where the audience can move and explore around. We were shown Ghosts, by Jonghee Woo,

I liked this play, I am not a big theatre fan, I struggle to keep the attention throughout the whole play, but with immersive theatre it gets much more interesting. I found this play engaging and I liked how the actors perform, but I don’t think that they broke the 4th wall, because, yes the audience was moving around and was able to be really close to the actors without any stage boundaries, but the performers didn’t interact directly with them, the audience was still there listening and watching remotely.

sources: https://uh.edu/engines/romanticism/introduction.html https://en.wikipedia.org/wiki/Wanderer_above_the_Sea_of_Fog

Fourth wall in VR

Talking about the fourth wall in VR I think is a bit of a broad concept, because it depends on which VR experience we are considering, it can go from a 360 VR movie to an interactive game or training program.

Most of the time when experiences or games are interactive, the audience is much more engaged, so I think that you don’t feel the presence of the fourth wall as much as a 360 VR movie which can be compared to the traditional theatre setup. So overall interactions help a lot to not fell the 4th wall presence, but they are not the only thing that keeps the agency in a VR experience. The environment, plot, elements and whatever has to do with keeping up the player’s attention and interest.