AR / VR / Extended Reality

From New Media Business Blog

Jump to: navigation, search

Contents

Extended Reality (ER)

Extended reality (XR) is the overall umbrella term that covers virtual reality (VR), augmented reality (AR), and mixed reality (MR). Its purpose is to mirror the real physical world with the digital world to allow interaction. [1]

Extended Reality AR, VR, and MR [2]

Virtual Reality (VR)

Virtual Reality (VR) is a computer-generated simulation of a three-dimensional image or environment that can be interacted with in a real or physical way by a person with equipment. It requires the use of computer technologies (headsets) to allow the virtual/simulated environment explored in 360 degrees. A special feature of VR is that it stimulates our senses of touch, sound, sight, and smell to make the individual immersed in the virtual world. [3]

Types of Virtual Reality

Nintendo Switch: Animal Crossing [4]

Non-Immersive

Non-Immersive reality is a computer-generated virtual environment where the user is aware and controlled by their physical environment while interacting with a virtual world. Out of all the three types, it is the least interactive. An example would be playing Animal Crossing on the Nintendo Switch using the joystick controllers. [5]

Arcade Game [6]





Semi-Immersive

Semi-immersive is a mix of both non-immersive and fully immersive virtual reality. It is more interactive compared to non-immersive and provides users with a partially virtual environment as they are still in their own physical environment. An example would be individuals playing car racing games at an arcade. They would feel the chair moment and wind making the experience feel realistic. [5]

Fly Over Canada [7]




Fully-Immersive

Fully-immersive virtual reality convinces the user that they are completely disengaged from the real environment and are now in the virtual environment that was created. It is the most realistic simulation experience, it provides sound and very high-definition images and videos. Fly Over Canada is a virtual reality experience in Vancouver where it allows individuals to fully immerse themselves into flying over Canada. Users would physically feel like they are moving and gives an amazing bird's eye view of the beautiful city. [5]



History of Virtual Reality

The Wheatstone mirror stereoscope. [8]

The 1800s

Sir Charles - Innovation of Stereoscope

In 1838, scientist Sir Charles Wheatstone outlined the concept of "stereopsis" or "binocular vision". Stereopsis is the visual ability to perceive the world in 3D. This led him to innovate the stereoscope where individuals can view different objects from different depths and angles. [9]

Pygamalion's Spectacles Short Story.[10]



The Early 1900s

Pygmalion's Spectacles

In 1935, Stanley Weinbaum, an American Science Fiction writer presented a fictional model for Virtual Reality in his short story Pygamalion's Spectacles. In the story, the main character meets a professor who invented a pair of goggles which allowed a movie to appeal to a human's five senses: taste, touch, smell, sound, and sight. It was the very first prediction of VR. [9]





The 1950s to the 1970s

The Sensorama VR Machine.[11]


Sensorama

In 1956, the very first VR machine Sensorama was created by cinematographer Morton Heilig. This movie booth combined 3D, colour video using stereoscopic technology with audio, scent producers to release different smells, and vibrating chairs to enhance the user's experience.[9]










Telesphere

The Telesphere Mask.[12]

In 1960, Heilig also invented the Telesphere Mask. This was the very first head-mounted display (HMD) that provided 3D images with wide vision and sound. However, at this time, there was no motion tracking. At every angle, individuals were only able to view the same image.[9]








Sword of Damocles

The Sword of Damocles.[13]

In 1968, Sutherland created the first virtual reality HMD, the Sword of Damocles. To use this head-mounted display, it required the connection with a computer rather than a camera. With tracking technology in the headset, users are able to change their perspective on the image when they moved their head side to side, up and down. However, this project did not succeed as the equipment was too heavy for users. [9]





VIDEOPLACE

VIDEOPLACE.[14]

In 1975, Krueger's VIDEOPLACE was the very first interactive in-person VR platform that was displayed at Milwaukee Art Centre. It utilized computer graphics, projectors, video cameras, and position-sensing technology that did not require goggles or gloves. The user would use their fingers to draw in the air and their recorded hand movements would be directly transferred onto the screen. This platform allowed users to interact with each other in different rooms while engaging on the same platform. It encouraged the idea that people could communicate in a virtual world even when they were not physically close to each other. [9]



Aspen Movie Map

Aspen Movie Map.[15]

In 1977, the Aspen movie map was created by MIT (Massachusetts Institute of Technology) led by Andrew Lipman. This map enabled users to virtually explored the city of Aspen in Colorado like with our present-day Google Street View. It consists of three modes: Summer, Winter, and Polygons. Photographs from a car driving through a city were utilized to create the visuals of the map. The purpose of this creation is to suggest that VR can transport people to different places without physically being there. [16]






The 1980s to the 1990s

VPL Research

VPL Research.[17]

In 1985, Jaron Lanier and Thomas Zimmerman founded VPL Research. They were the very first company to sell VR goggles and gloves. The gloves that they sold were able to monitor hand movements by using light emitters and photo cells from the gloves' fingers.[16]





NASA VR Training

NASA VR Training.[18]

In 1989, Scott Foster, founder of Crystal River Engineering Inc, received a contract from NASA to develop the audio element of Virtual Environment Workstation Project (VIEW) - a VR training simulator for astronauts. This was the first time the public saw that VR was introduced into space.[16]





VIRTUALITY LAUNCH

VIRTUALITY Arcade Machines.[19]

In 1991, the Virtuality Group launched Virtuality. These were VR arcade machines where gamers could play in a 3D gaming world. It was the very first mass-produced VR entertainment system. It allowed multiplayer games and featured popular arcade games, like Pac-Man. [16]





Nintendo - Virtual Boy Console

Nintendo: Virtual Boy Console.[20]

In 1995, Nintendo launched the Virtual Boy console which played 3D monochrome video games. It was the very first portable console to play with 3D graphics. However, despite the hype, it was discontinued after one year as it had a lack of colour graphics, poor software support, and was not comfortable to use. The physical build of the headset was bulky and required too many extra devices to support the games. [16]






The 2000s

Google Street View

In 2007, Google introduced street view which most people are familiar with. This street view collaborated with immersive media which was identified as the contractor who captured imagery for four out of five cities mapped by Street view using moving cars. The idea of being able to transport virtually and tour around different places in the world was supported through this application. [16]


VR Product Lines

In 2014-2016, many VR products were created, these include the Oculus, PlayStation, Google Cardboard, and Samsung Gear VR.


VR Gets Real

2019, this is the year where Forbes describes it as VR gets real. This is because Facebook's headset, the Oculus Quest has created lots of interest, selling out in many locations generating revenues over $5 million for the company. Also, many virtual reality experience hubs were opened, increasing demand and curiosity for VR.

Technology behind VR

VR Headset [21]


The main component of virtual reality is a VR headset, which is a head-mounted device that creates an immersive 3D experience. Most VR headsets include screens, cameras, motion sensors, and infrared LEDs. The lenses are positioned between the LED screen and the user’s eyes, which is what makes the image that you’re looking at 3D and realistic. [22]


Each VR headset has visual features that help improve the quality of the user’s 3D environment. These features include the field of view (FOV), latency, and frame rate.


VR Screen [23]


Field of View (FOV)

The field of view (FOV) is the extent to which a person can see through their eyes or a device, while the latency is the delay time between the user’s movement and the change of the VR display. A VR headset replaces your natural FOV with a computer-generated FOV. Usually, we can see a FOV of around 200° to 220° of surrounding content without any movement, while a VR headset allows us to see in the binocular FOV. VR hardware designers are creating devices with a 180° FOV and latency of 30 to 40 milliseconds. [22]


Monocular vs. Binocular FOV [24]

Frame Rate

The frame rate is the rate at which the graphics processing unit (GPU) can process the visual images per second. The human eye can see up to 1000 frames per second (FPS), and most developers aim for a sweet spot of about 90 FPS. Anything less than 60 FPS may result in motion sickness. Going forward, VR hardware developers are aiming towards 120 FPS to further improve the user’s 3D experience. [22]

3D Spatial Audio

Besides the visual components of VR, 3D spatial audio is also very important for creating more immersive experiences. VR sets rely on spatial audio technology to match the in-real-life experience for the user. When navigating around the VR environment, 3D spatial audio helps mimic sounds similar to the real world. [25] Companies like Spotify, Youtube, and Apple also use 3D audio. [26]

Head and Position Tracking

Head and position tracking also increases the user’s engagement in VR by allowing users to move around in the virtual space and adjust to their position. VR companies use two different types of head and position tracking simulators: three degrees of freedom, where users can look left right up and down, and six degrees of freedom, where users can look everywhere in 360 degrees. Motion controllers can also be used to help make the environment feel more immersive. With motion controllers, users can walk around the virtual space, wave their hands, and participate in other interactive activities. [22]



The Three Categories of VR

VR devices can be broken down into three categories: Mobile, Desktop, and Standalone. [27]


Google Cardboard [28]

Mobile

Mobile VR set allows you to connect your VR set to your mobile smartphone or tablet.


Google Cardboard

Google Cardboard is an example of a mobile VR experience. It is a simple and affordable option to experience virtual reality by using Cardboard-enabled apps on almost any smartphone. Before Google Cardboard was discontinued on March 3, 2021 [29], you had the option to buy a Cardboard-certified viewer on the website, or build your own viewer that fits your phone’s screen size. Afterwards, you can download Google Cardboard’s VR apps, such as Youtube VR and VR theatre. Most Cardboard apps can be used with Android 4.1+ and the latest iOS smartphones. [30]




Oculus Rift S [31]

Desktop

Desktop VR set requires a computer to link to your VR set.


Oculus Rift S

An example of a desktop headset is the Oculus Rift S, which was mainly used for PC-powered VR gaming. The Oculus Rift S was co-developed by Lenovo Technologies and Facebook Technologies—a division of Meta Platforms. Before it was discontinued in June 2022, the Oculus Rift S was accessible with a wide range of PCs. The Rift S was known for its comfort because of its halo headband which has better weight distribution and light blocking. [32]




Standalone

Standalone VR set does not require connecting to any device.


Meta Quest 2

After the Oculus Rift S discontinued, Facebook released the standalone Meta Quest 2 on October 13, 2020. This all-in-one set retails for C$609.99. The Meta Quest 2 provides new opportunities to meet and connect with others in VR, from attending a live show in VR to participating in an instructor-led workout. You can still connect this set to a PC through Oculus’ Link feature. It uses 6 degrees of freedom, which tracks the movement of both the user’s head and body with realistic precision. In addition, this VR set uses 3D positional audio and a 3.5 mm audio port that lets you play with or without headphones. [33]


Meta Quest 2 [34]



Real-World Applications of VR

Real world applications of VR, include real estate, the automotive industry, and the retail industry.

VR in Real Estate [35]


In Real Estate

The real estate world started implementing VR more during the pandemic, including virtual staging, 3-D tours, and virtual property showings. In real estate, we found that listings with virtual tours get 87% more views than those without. VR is now a major focus in the real estate world, as it improves the experience for the buyer and helps them get a better picture of how to make a house a home. For instance, VR allows the buyer to decorate the house how they want, from placing furnishings in the right spot, to applying different color choices. Additionally, VR allows the buyer to view their housing in different lighting conditions so they can get a sense of what their future homes could feel like before moving in. [36]

VR in the Automotive Industry [37]




In the Automotive Industry

The automotive industry has also been using VR to visualize every part of the vehicle and simulate the vehicle's performance. For example, Land Rover has been using VR since 2008 to visualize vehicles in their creation process. Andy Richardson, Manager of the Jaguar Land Rover Simulation Group in the UK writes, “virtual reality has completely transformed how we design and engineer new vehicles today”. [38] The designers and engineers develop new vehicles in three-dimensional virtual reality so they can look at how every component of the car fits together. This ensures that each of their vehicles are designed perfectly and are built just as intended.

VR in the Retail Industry: Walmart [39]




In the Retail Industry

Lastly, the retail industry uses VR to train employees. For example, Walmart has been using VR since 2017 for training, improving the employee experience, and assessing the employee’s skills. Andy Trainor, vice president of learning at Walmart, believes that VR is beneficial to employee training as it “allows you to artificially create scenarios that you can't recreate on the sales floor in a way that associates can learn in a safe environment”. [40] For example, VR can help prepare employees during stressful situations such as Black Friday. VR can also be used to assess how an employee would respond to an angry customer. This way, employees can improve their customer-service skills before handling customers in real life.

Augmented Reality (AR)

AR Defined

Augmented reality is an enhanced version of the physical world around us that is achieved with the use of digital visual elements, sound, and other sensory stimuli. Augmented reality involves overlaying visual, auditory, and other sensory information in order to enhance one’s experience. Augmented reality typically combines three basic features, such as combination of the real world and a virtual one, real-time interaction with the virtual world, and 3D registration of both virtual and real objects. Though very similar, AR differs from Virtual Reality (VR) in that VR aims to totally immerse users into an entirely different environment, typically a virtual one created and rendered by computers. AR aims to use the existing real-world environment and to place virtual information in it to enrich the user’s perception and experience of the real world. [41]

Forbes:Augmented Reality [42]

AR Process

The AR Process [43]

The AR process first begins with a form of input of the real environment. AR-enabled hardware, such as phones and smart glasses, typically hosts a system of cameras and 3D depth sensors in order to see and understand the physical world. The images provided by the AR-enabled hardware, such as an image from a smartphone camera, provide a real environment input.

From there, the AR-enabled hardware helps map the environment and builds a virtual model using sensors and specialized software, such as computer vision and machine learning. It is through computer vision and machine learning that the computer will be able to recognize the user’s surrounding environment and will be able to recognize the object’s distance and position relative to the camera.[44] This process is referred to as tracking, or localization, which helps map the environment and keeps track of where the object is. Localization is done through semantics, which tells the computer exactly what is being portrayed, and through geometry, which ensures the real environment input is positioned at the correct angle so that AR content will be displayed in the right place and angle. [45]

After the tracking and localization process, a rendering process is conducted by the AR software, in which digital content is displayed on the original image. The AR system will place digital content on the virtual map built in the previous stage, which is unseen to the user and makes it seem like the AR content exists in our actual environment. From there, the AR experience with additional visual, auditory, and sensory information is overlaid and the user is provided an interactive experience that combines the real world and computer-generated content. [44]

Technology behind AR

There are three key technologies associated with the augmented reality experience: intelligent display technology, 3D registration technology, and intelligent interactive technology. [46]

Intelligent Display Technology

Intelligent display technology is quite broad but can typically be broken down into three different display categories. The first display category includes helmet displays, which can be defined as a piece of equipment the user typically fixes over their head, which helps superimpose images onto real scenes in real-time. The second category would be handheld Device displays, which are the most popular and typically associated with smartphones. The last category is an all-encompassing "other display devices" which contains a variety of different devices such as PC desktop displays. For example, a PC desktop display can help match real-world scene information to three-dimensional virtual models and display the overlaid images on said desktop. [46]

AR Glasses [47]

3D Registration Technology

The second key technology associated with AR would be 3D registration technology, which enables virtual images to be superimposed accurately in the real environment. The 3D registration hardware typically sets a reference point to realize the determination of direction and position of the real scene, of which is the basis point for all AR processes.[46]

Intelligent Interactive Technology

The third key technology would be intelligent interaction technology, which allows AR systems to conduct a variety of intelligent interactions, device interactions, location interactions, tag-based or other information-based interactions. People give specific instructions to the virtual object in the scene, and the virtual object can make some feedback. [46]

AR Applications

AR in Commerce

Sephora and AR

Sephora's Visual Artist [48]

Makeup mega-corporation Sephora leverages augmented reality by developing ‘virtual try-on’ services for makeup. From home, customers are able to use their phone cameras to personalize their shopping experience by trying on makeup virtually using AR, such as matching their skin tone to a specific foundation shade. The company is well known for its "Sephora Virtual Artist", in which users are able to try on shades of lipstick, eyeshadow, false lashes, and many other makeup products sold in stores. [49]

Warby Parker and AR

The eye lens company Warby Parker utilizes augmented reality as part of their business model. Warby Parker, through the help of Apple's iPhone selfie camera and ARKit technology, offers a virtual try-on feature that allows customers to preview glasses using augmented reality and helps visualize how certain frames, colours, and materials will appear on users' faces. Warby Parker uses a placement algorithm that mimics the real-life process of placing a pair of frames on your face, taking into account how your unique facial features interact with the frame. [50]

Ikea and AR

In a similar vein, furniture giant Ikea utilizes Apple's ARKit and iPhone Camera to employ augmented reality as part of the business model. In 2017, the company released their "Ikea Place" app, which allows users to test Ikea's products prior to actually purchasing, providing features such as realistically-rendered, true-to-scale 3D products. The app can visualize products in its users' space, helping to automatically scale products based on the room dimensions observed by the camera. As Michael Valdsgaard, the leader of digital transformation at Inter Ikea Systems, says "Augmented reality and virtual reality will be a total game changer for retail in the same way as the internet. Only this time, much faster." [51]

IKEA Place App [52]

AR in Entertainment

Pokemon Go and AR

Pokemon Go [53]

AR can be used in gaming, a popular example being Pokemon Go. As users walk around the real world whilst holding their smartphones, Pokémon figures appear on the game map, which users can interact with using their cameras. When users come within a close enough range as they walk around the real world, various Pokémon will appear on the device screen depending on the users' location, in which users will then throw Poké Balls at them to capture them. The purpose of the game is to collect as many different characters as possible, which requires individuals to interact with both their real environment and a virtual one on their smartphone screens.

The explosion of Pokemon Go's popularity has greatly benefited the AR industry, with the New York Times reporting Pokemon Go as being an industry leader, helping to bring Augmented Reality to the masses. Pokemon Go made AR more accessible to the general public as it demonstrated to the public that augmented reality can be achieved on devices we already own, rather than just being limited to pricey headsets and hardware typically associated with the extended reality industry. Jan Dawson, a technology analyst at Jackdaw Research has stated “[Pokemon Go’s success] ​​clearly demonstrates that A.R. can cross over into the mainstream on the devices people already have [...] such as smartphones." [54]

Snapchat and AR

Snapchat is also a major proponent of AR, helping to popularize camera filters in mainstream social media. Snapchat famously utilizes AR as part of the main draws to their app, providing many beauty filters, to enhance the users' appearance in photos and videos, or humorous filters, helping to depict its users as a dog, rainbow, or tomato. Snapchat, through the use of filters, has mastered the art of viral advertisement on their apps, by allowing many companies to create interactive and fun AR filters that appeal to the younger generations. Users are able to film themselves interacting and send them to their friends, helping to increase engagement with company ads and increase brand awareness.[55]

Snapchat Filters [56]

AR in Healthcare

Hololens and AR

AR in Healthcare [57]

Microsoft is set to release a HoloLens app called HoloAnatomy that will help visualize the 3D visualization of the human body. With Microsoft’s HoloLens Headset, app users are able to see everything from muscles to the tiniest veins before their eyes on a dynamic holographic model. This will help revolutionize medical education, as medical students will be able to study the human body in 3D instead of the usual working method: black-and-white pictures and written descriptions in books. [58]

Accuvein and AR

Accuvein is a handheld device that utilizes AR to scan the vein network of a patient, helping surgeons to study their patient’s anatomy prior to surgery. Through AR, surgeons can plan procedures before making the first cut, create models made of tumours, and can also utilize AR diagnostic tools can model disease conditions. [59]

Mixed Reality (MR)

Mixed Reality Defined

Mixed reality is the blend of the physical and digital worlds, which enables fluid and natural 3D interactions between people, technologies, and the real environment. Different from AR and VR, this new reality is an emergent technology that combines those two technologies together, which blurs the line between what is real and what isn’t. It might be demonstrated by physically touching an object while also influencing a virtual world action or result.

Picture yourself sitting in your workplace. You can see all the surrounding physical objects, such as desks and chairs. At the same time, you may also engage with digital content, such as a shared document that updates in the cloud in real time or a prototype that you can test out using a digital interface. In essence, it is what mixed reality promises. [60]


Technology Behind Mixed Reality

The advancements in computer vision, graphics processing, display technologies, input systems, and cloud computing serve as the foundation for mixed reality.

Mixed reality requires five main components:

Environmental understanding

The ability to combine content from the virtual and physical worlds by mapping an area and superimposing information on it.

Human understanding

The ability to track the movements, speech, and inputs of human users through sensors or cameras.

Spatial sound

360-degree audio experiences are created to enhance the immersion and realism of digital experiences.

Locations and positioning

The ability to understand the positions of both the technology and the user at any given moment.

3D assets

Fully three-dimensional content that can be touched, accessed, and monitored in the real world. Those assets are usually referred to as holograms. [61]


MR Applications

Metaverse

24 Hours In Metaverse [62]

The term “metaverse” is a combination of “meta”, meaning “beyond” in Greek language, and “universe”. "Beyond universe" is what "metaverse" refers to. Simply put, it refers to online shared virtual worlds that individuals can access via the internet. The metaverse's vision eventually becomes more promising when combined with Mixed Reality passthrough, allowing users to perform nearly any action there, such as go shopping, attend school, or take part in business meetings.

There are already some instances of the metaverse. Virtual concerts featuring celebrities like Ariana Grande have been held on Roblox Corp. and Epic Games Inc.'s "Fortnite," in addition to other immersive experiences. Metaverse would also be possible to carry out novel, unbelievable feats like walking on the moon.

However, there are serious drawbacks and problems that metaverse is currently facing. The significant amount of money invested in a project with an unclear future is one of the major issues. Some experts worry about security, privacy, and users' mental health. [63]




HoloLens

HoloLens 2 [64]

The most widely utilised, high-performing Mixed Reality product is called Hololens. Through the lens, it projects 3D holograms that can merge with the surrounding surroundings. The lens can assess the area around it and apply virtual graphics based on the actual surroundings thanks to the environmental sensors. [65]

In 2015, Hololens was originally unveiled, and the most recent model, the HoloLens 2, has a $3500 USD price tag.

Intended functions of HoloLens: - Innovate faster: enabling organizations or individuals to quickly develop digital twins of things and test innovative concepts while wasting little money or time. You may quickly and easily test out various materials and concepts with HoloLens. - Collaborate: enabling the creation of more immersive collaborative experiences. For example, it can be demonstrated by being able to construct a holographic replica of a new car with your coworkers, even though you are all located thousands of miles apart. - Support workers: broadcasting content to associates in real-time, enabling experts to annotate what they view (such as the image of a machine in front of a different employee) and give instructions in real-time.



MR in Music

Whitney Houston Holograms [66]

Whitney Houston, a singer who passed away in 2012, utilized the MR technology for her concert by projecting 3D holograms onto the stage. The deceased diva's voice was included in the programme, along with a live four-piece band, additional support singers, and dancers.

Although the morality of using a late performer's hologram to increase ticket sales and album downloads is debatable, Whitney Houston's former manager and current President and CEO of The Estate of Whitney E. Houston, Pat Houston, claims she discussed the idea with the artist a decade ago (before the Apple iPad was invented, for reference).[67]








MR in Fashion

New York Fashion Week [68]

Digitally created psychedelic designs and stylish party attire by designer Maisie Schloss fit perfectly in with a fashion industry that is considering the potential of a virtual future. She embraced this virtual world during New York Fashion Week, showcasing her most recent line for Maisie Wilen on holographic models who stood 7 feet tall.

Visitors to the Fall 2022 show over the weekend discovered virtual models executing a series of repetitive motions like GIFs when they entered a gallery setting. Some of the dolls, which were modelled after the dolls from Mattel's 2010s "Monster High" franchise, spun in space, swayed their hips, pointed at guests, and occasionally released animated blue lightning or bubbles. Others donned green and blue body paint, creature ears, and fins.[69]






MR in Education

MR in Education [70]

HoloStudy: A mixed reality learning tool called HoloStudy was created to gain a better knowledge of people. It revolutionarily engages students while deconstructing challenging disciplines like physics, biology, chemistry, geology, and medicine.

HoloTour: Another cutting-edge application, HoloTour, uses mixed reality to whisk users away to various places and eras. Students can take advantage of educational field trips to various nations and significant periods in human history by stepping into these mixed realities, all without ever leaving their classrooms.

HoloHuman: By examining intricate, under-the-skin holographic simulations of the human body, HoloHuman gives students the opportunity to get a thorough understanding of human anatomy. The learner can examine life-sized bodies and learn how they function in a novel and engaging way thanks to this fully immersive experience.

Lifelique HoloLens: This innovative programme was created by Lilique in collaboration with Microsoft HoloLens to give classrooms, students, and teachers access to an interactive 3D model. Lilique can help make learning more engaging and connect with students at a level that was previously thought to be impossible by incorporating mixed reality models and lesson plans into the regular classroom experience.[71]


MR in Healthcare

MR in Healthcare [72]

Healthcare providers can simplify medical procedures and keep patients safe by using mixed reality. One of the important industries for the introduction of MR is healthcare, which includes training for robots and experts, phobia therapy, and surgical simulation.

The main force advancing the sector is the continued development of mixed reality technology in multiple healthcare institutions. In order to progress their health systems, nations like Qatar, Saudi Arabia, and the UAE invest in cutting-edge technology like IoT, virtual reality, and augmented reality, according to a study by Fortune Business Insights.

Future predictions indicate that the use of MR in healthcare will skyrocket, especially when it comes to virtual reality simulation and medical training.[73]







MR in Manufacturing

MR in Manufacturing [74]

Manufacturers are now employing mixed reality to drastically reduce call-out times instead of relying on engineers who have to read dense, frequently outdated instructions. ThyssenKrupp, a producer of elevators, is one such organisation. Engineers using HMDs not only receive the most recent information, but they can also view it without using their hands, thus cutting down on the time needed for knowledge checks. Additionally, when engineers call a support line, a remote expert has a "you see what I see" view, enabling them to offer assistance as if they were in person at the client site. Because of the use of HMDs, ThyssenKrupp's service calls are now up to four times shorter on average.

The difficulty is that new technologies need high data loads, the capacity to handle enormous volumes of data at lightning rates, and the capacity to scale projects in a computer environment, which is frequently not possible in typical office settings. To do this, discrete GPUs from one or more servers are used, and the produced frames are sent wirelessly or remotely to head-mounted displays (HMDs), like the Microsoft HoloLens and the Oculus Quest.[75]



MR in Military

MR in Military [76]

The US army has recently been using hololens for war simulation tests. Soldiers will be wearing hololens to simulate different situations in combat and war. However, it has been reported that soldiers felt ill after using the headset for simulations. Issues seem to be recognised and addressed by the military. Brigadier General Christopher Schneider told Insider that IVAS met "most" of the criteria, but that there were other places where it "fell short" and needed development.[77]








Future Predictions & Challenges of Extended Reality

Future Predictions

In a research article published by KPMG this year, in consultation with 15 experts, it was said that within the next 10 years we will spend more of our walking time in the 3D virtual world than we will in the physical world.[78] With XR working together with AI, individuals will be able to make more critical decisions in the virtual space, as we will have every piece of data instantly available in front of our eyes. In this context, it will change why we learn what we learn. Individuals will increasingly be assessed for jobs based on how they can use the data and information provided to them rather than what information they can remember. There will be full-time jobs in the virtual reality worlds and income-earning opportunities that are created within the environment where individuals spend most of their time. Moreover, NFTs will be more incorporated into the virtual world and will become admission tickets to exclusive offerings.

Extended Reality in the Workplace

DHL Virtual Training [79]

The COVID-19 pandemic forced many workers to work remotely, and companies needed to adjust. Now, while some companies have fully transitioned to in-person work, some companies still prefer hybrid work for many of their employees. With this Extended Reality, devices enable employee engagement and training for remote employees. For instance, Deutsche Bahn, a German railway company, has used extended reality technologies to train their employees on how to respond reliably in critical situations. DHL is also using Virtual Reality to train their employees with new and engaging scenarios. These are only a few examples of how companies in the future will be more likely to adopt Extended Reality technologies to train and collaborate with their employees.[80]



Challenges of Extended Reality

Alongside any technological advancement, there will be challenges and concerns. It is best that we plan for these events before the full integration of these devices.


User Data and Privacy

These technologies are always collecting user data to create an immersive experience for the user. However, this comes with the potential insecurity of data such as biographical and demographic details, location and movement, biometrics, and gaze tracking. With extended reality devices collecting all this sensitive data through several different information-gathering technologies, it raises many concerns about user privacy and how this data should be stored.[81]

Legal

Extended reality devices collect a wide variety of data from their consumers and having a one shape fits all for technological devices will leave gaps in the protection of its users. Instead, the government should focus on making individualized policies to make sure there isn’t overregulation, while mitigating any harm that may be caused.

Accessibility

There are great benefits of a VR headset, including being able to work remotely while being privy to the same information and education as everyone working in-person. However, that is not the full reality, as VR headsets are expensive and the costs to make them are high. This creates the risk of creating more divisions within communities especially if some are not willing to partake in adopting such technology.

Health

Extended reality devices have a near eye-display that users will be using for many hours continuously throughout the day, which can cause health concerns. Alongside the displays, the use is of concern, as headsets are often used for hours on end, making a person very stationary. Furthermore, these devices have been shown to contribute to visual fatigue, motion sickness, and cognitive overload for users.[82]

Summary

To conclude, Extended Reality works interactively between virtual work and the physical world including Virtual Reality, Augmented Reality, and Mixed Reality, which are highly involved in our daily lives. With the popularity of Virtual Reality headsets, it has developed its own area in the gaming, real estate, and automotive industries. Augmented Reality has also been developing well in the health care, commerce and phone application area by showing the virtual image in reality through technologies. Mixed Reality, as the combination of Augmented Reality and Virtual Reality, has been the most anticipated technology and it has already been applied successfully in different industries such as manufacturing, education, manufacturing, fashion, music, etc. As Extended Reality develops, it will significantly alter how we interact with technologies. However, it has also faced numerous challenges regarding privacy, security, legal, accessibility and health issues. The market is heating up for Virtual Reality, Augmented Reality, and Extended Reality in general. However, we aren't yet prepared for all of these realities. Market leaders, like Meta, need to address listed concerns, innovate in a quicker way, and prove that their visions can actually work in the real life.



Authors

Bernice Chu Aileen Dasalla Sharanjit Ghuman Lavena Kwok Amy Le Samantha Mo
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada

References

  1. https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/?sh=1711eb0f7249
  2. https://www.interaction-design.org/literature/article/beyond-ar-vs-vr-what-is-the-difference-between-ar-vs-mr-vs-vr-vs-xr
  3. https://www.alistdaily.com/glossary/virtual-reality/#:~:text=The%20computer%2Dgenerated%20simulation%20of,or%20gloves%20fitted%20with%20sensors.
  4. https://www.cnn.com/2020/04/30/tech/nintendo-switch-animal-crossing-quarantine
  5. 5.0 5.1 5.2 https://heizenrader.com/the-3-types-of-virtual-reality/#:~:text=There%20are%203%20primary%20categories,%2C%20and%20fully%2Dimmersive%20simulations.
  6. https://gfycat.com/discover/arcade-racing-gifs
  7. https://www.flyovercanada.com/
  8. https://www.https://virtualspeech.com/blog/history-of-vr/
  9. 9.0 9.1 9.2 9.3 9.4 9.5 https://www.forbes.com/sites/bernardmarr/2021/05/17/the-fascinating-history-and-evolution-of-extended-reality-xr--covering-ar-vr-and-mr/?sh=5ada7ccb4bfd
  10. https://www.https://virtualspeech.com/blog/history-of-vr/
  11. https://www.https://virtualspeech.com/blog/history-of-vr/
  12. https://www.https://virtualspeech.com/blog/history-of-vr/
  13. https://www.https://virtualspeech.com/blog/history-of-vr/
  14. https://www.https://virtualspeech.com/blog/history-of-vr/
  15. https://https://www.youtube.com/watch?v=2Ytd12d6qNw/
  16. 16.0 16.1 16.2 16.3 16.4 16.5 https://virtualspeech.com/blog/history-of-vr
  17. https://www.https://virtualspeech.com/blog/history-of-vr/
  18. https://www.https://virtualspeech.com/blog/history-of-vr/
  19. https://www.https://virtualspeech.com/blog/history-of-vr/
  20. http://www.computinghistory.org.uk/det/4595/Nintendo-Virtual-Boy/
  21. https://www.bestbuy.ca/en-ca/product/nintendo-switch-vr-set-virtual-reality-glasses-headgear-for-games-super-smash-bros-zelda-super-mario-odyssey-and-media/14348441?icmp=Recos_3across_tp_sllng_prdcts&referrer=PLP_Reco
  22. 22.0 22.1 22.2 22.3 https://www.g2.com/articles/virtual-reality
  23. https://lithub.com/virtual-reality-is-in-desperate-need-of-real-storytellers/
  24. https://www.g2.com/articles/virtual-reality
  25. https://creator.oculus.com/learn/spatial-audio/
  26. https://www.vrtonung.de/en/music-streaming-spatial-audio/
  27. https://www.forbes.com/sites/forbesbusinesscouncil/2021/04/05/what-is-virtual-reality-and-how-can-it-be-used-in-the-workplace/?sh=3a0aff197a1e
  28. https://whatvr.co/guides/google-cardboard/
  29. https://9to5google.com/2021/03/03/google-store-stops-selling-cardboard-vr-headsets/
  30. https://arvr.google.com/intl/en_ca/cardboard/
  31. https://www.amazon.ca/Oculus-Rift-PC-Powered-Gaming-Headset/dp/B07PTMKYS7
  32. https://www.oculus.com/rift-s/features/
  33. https://www.meta.com/ca/quest/products/quest-2/tech-specs/#tech-specs
  34. https://www.eurogamer.net/meta-quest-2-getting-100-price-hike
  35. https://www.passionateinmarketing.com/augmented-and-virtual-reality-is-the-future-of-real-estate/
  36. https://www.forbes.com/sites/forbestechcouncil/2022/09/08/five-ways-real-estate-agents-can-make-use-of-technology/?sh=4bc7922462f7
  37. https://techwireasia.com/2020/02/how-jaguar-land-rover-uses-vr-to-design-and-engineer-new-vehicles/
  38. https://media.landrover.com/en-us/blog/2010/future-virtual-reality
  39. https://crosswater.net/blog/vr-shopping-at-walmart/
  40. https://www.shrm.org/resourcesandtools/hr-topics/technology/pages/virtual-reality-revolutionizes-walmart-training.aspx
  41. https://www.investopedia.com/terms/a/augmented-reality.asp.
  42. https://www.forbes.com/sites/theyec/2019/02/06/augmented-reality-in-business-how-ar-may-change-the-way-we-work/?sh=c0b2f6651e50
  43. https://www.blippar.com/blog/2018/08/21/what-is-augmented-reality-and-how-does-augmented-reality-work
  44. 44.0 44.1 https://www.youtube.com/watch?v=H7ZHemE2nRs.
  45. https://www.blippar.com/blog/2018/08/21/what-is-augmented-reality-and-how-does-augmented-reality-work.
  46. 46.0 46.1 46.2 46.3 https://iopscience.iop.org/article/10.1088/1742-6596/1237/2/022082/pdf#:~:text=Key%20Technologies%20of%20Augmented%20Reality,in%20the%20development%20of%20AR.
  47. https://www.forbes.com/sites/bernardmarr/2021/05/21/transforming-retail-with-augmented-and-virtual-reality-powered-by-artificial-intelligence-the-case-of-eyewear-shopping/?sh=74b6b39f75d5
  48. https://www.techrepublic.com/article/how-sephora-is-leveraging-ar-and-ai-to-transform-retail-and-help-customers-buy-cosmetics/
  49. https://www.techrepublic.com/article/how-sephora-is-leveraging-ar-and-ai-to-transform-retail-and-help-customers-buy-cosmetics.
  50. https://technologymagazine.com/digital-transformation/warby-parker-uses-augmented-reality-allow-customers-virtually-try-glasses.
  51. https://www.architectmagazine.com/technology/ikea-launches-augmented-reality-application.
  52. https://www.architectmagazine.com/technology/ikea-launches-augmented-reality-application
  53. https://commons.wikimedia.org/wiki/File:Pokemon-go-1569794_1920.jpg
  54. https://www.nytimes.com/2016/07/12/technology/pokemon-go-brings-augmented-reality-to-a-mass-audience.html.
  55. https://www.forbes.com/sites/forbesagencycouncil/2022/11/29/what-are-augmented-reality-filters-and-how-can-you-market-your-business-with-them/?sh=5745e07f4d66.
  56. https://www.clickz.com/how-ar-works/241718/ar-snapchat-filters/
  57. https://www.hippopx.com/en/augmented-reality-medical-3d-science-technology-people-women-76450
  58. https://www.youtube.com/watch?v=gzUTT1Kygo4.
  59. https://www.forbes.com/sites/bernardmarr/2018/07/30/9-powerful-real-world-applications-of-augmented-reality-ar-today/?sh=1b9c67372fe9.
  60. https://learn.microsoft.com/en-us/windows/mixed-reality/discover/mixed-reality
  61. https://www.xrtoday.com/mixed-reality/how-does-mixed-reality-work/
  62. https://youtu.be/rtLTZUaMSDQ
  63. https://www.cnbc.com/2022/10/24/altimeter-capitals-brad-gerstner-calls-on-meta-to-slash-headcount.html
  64. https://images.app.goo.gl/6zd9DPATdb1EZAXr7
  65. https://www.microsoft.com/en-us/hololens
  66. https://www.forbes.com/sites/melissakravitz/2021/07/21/a-whitney-houston-hologram-concert-is-opening-in-las-vegas/?sh=19667fb87cb1
  67. https://www.forbes.com/sites/melissakravitz/2021/07/21/a-whitney-houston-hologram-concert-is-opening-in-las-vegas/?sh=19667fb87cb1
  68. https://www.cnn.com/style/article/maisie-wilen-fall-2022-nyfw-holograms/index.html
  69. https://www.cnn.com/style/article/maisie-wilen-fall-2022-nyfw-holograms/index.html
  70. https://er.educause.edu/articles/2018/7/mixed-reality-a-revolutionary-breakthrough-in-teaching-and-learning
  71. https://er.educause.edu/articles/2018/7/mixed-reality-a-revolutionary-breakthrough-in-teaching-and-learning
  72. https://www.microsoft.com/en-us/hololens/industry-healthcare
  73. https://www.microsoft.com/en-us/hololens/industry-healthcare
  74. https://blog.econocom.com/en/blog/hololens-brings-mixed-reality-into-industry/
  75. https://blog.econocom.com/en/blog/hololens-brings-mixed-reality-into-industry/
  76. https://www.engadget.com/microsoft-hololens-fails-us-army-tests-135010970.html
  77. https://www.engadget.com/microsoft-hololens-fails-us-army-tests-135010970.html
  78. https://assets.kpmg/content/dam/kpmg/au/pdf/2022/future-of-XR-white-paper.pdf
  79. https://www.dpdhl.com/en/media-relations/press-releases/2019/dhl-americas-innovation-center-opens.html
  80. https://www.xrtoday.com/mixed-reality/xr-predictions-2022-virtual-and-augmented-reality-in-the-future-of-work/
  81. https://itif.org/publications/2021/03/04/balancing-user-privacy-and-innovation-augmented-and-virtual-reality/
  82. https://link.springer.com/article/10.1007/s10278-022-00622-x#:~:text=MXR%20devices%20using%20fully%20immersive,or%20cognitive%20overload%20%5B30%5D.
Personal tools