Virtual Personal Assistant

From New Media Business Blog

Jump to: navigation, search

Virtual Personal Assistant is modernly a cloud-based program that require internet connected devices in order to operate. They use artificial intelligence, and more specifically voice recognition, natural language processing and machine learning. A virtual personal assistant can help users with common tasks such as sending emails or messages, calling people or even more complicated processes such as controlling other smart devices. With AI being the sixth highest technology spending priority, up from the 20th spot a year ago[1], the computing power of virtual assistants is growing immensely to include more and more functions and capabilities.










Contents

Overview

Virtual Personal Assistant History

The 1961 was the first ever isolated digital speech recognition tool, being able to recognize zero, one, two, three, four, five, six, seven , eight, nine, ten, subtotal, total, false, and off. The IBM Shoebox was developed by IBM engineer, William C. Dersch.[2] In 1970, speech recognition technology made great advancements due to interest and funding from the U.S. Department of Defense, resulting in DoD’s DARPA speech understanding research program, and was the largest of its kind then. The Harpy speech-understanding system that could understand 1,001 words was also responsible by the DoD’s program. [3] In 1990, Dragon launched their isolated speech recognition program Dragon Dictate, which was the first consumer product that supported continuous speech recognition of around 5,000 words but had a price tag of $6,000. [4] Dragon Dictate was generally used by those that had disabilities that hindered their ability to type and was not hugely popular due to the high price tag. In 2011 was the year that voice recognition was brought to the mass by Apple’s effort in Siri, in which they acquired from Sri. Siri used cloud computing, with Siri devices connected to Apple’s servers and computers, which did all the complicated algorithms. Apple’s main competitors shortly joined afterwards, with Google launching Google Now in 2012, Cortana by Microsoft in 2013, and Alexa by Amazon in 2014. [5]










Market

The VPA market is currently dominated by Siri, with 500 million Siri active devices world-wide [6], and dominating the VPA market share in the US by a landslide at 45.6%. [7] Google Assistant is the second most used VPA at 28.7%, and Alexa third at 13.2%. The others including Bixby and Cortana combines for a meagre 12.5%. [8] This gives us an approximate estimation of 1.1 billion users for VPAs globally, which means 1 in 7 people use a VPA activated device.









What is VPA?

VPA uses artificial intelligence, speech recognition, machine learning and natural language processing in order to run. In the earlier days of VPAs with IBM Shoebox and Dragon Dictate, the device that it was installed in would do all the algorithm processing, which limited the capabilities of these voice recognition software due to limited processing power. However, the newer VPAs such as Siri, Google Assistant, Alexa, Bixby, etc., all use centralized processing at the company in order to run these algorithms and hence removing the limiting power of standalone devices.


Speech Recognition

Speech recognition has four steps that allows the computer to recognize what we’re saying [9]

  1. Soundwave is received through the input (mic) to the device
  2. The analog-to-digital converter then converts all these soundwaves into digital data such as pitch, volume, silences, phonemes and removes unwanted noises.
  3. The speech recognition program then divides these data into small segments as short as a few hundredths of a second and matches these to a central database
  4. Natural Language Processing is then required for the system to understand human language

Natural Language Processing

Natural language processing is a subfield of artificial intelligence, and is essentially what’s needed for a computer to understand the human language. There are three large categories of analytics that deals with Natural Language Processing Syntax analysis - The understanding of the structure of a sentence, including the use of grammar. Semantic analysis – The understanding of relationships between words. [10] (ie. depending on the context of a sentence, the word bright could be understood as shining or intelligent) Pragmatic analysis – The understanding of the true meaning behind a sentence through context and pre-existing knowledge. (ie. For Siri to understand when you ask it if it knows the time, it means that you are inferring that you would like to know the time as well.)

Putting a sentence through these three types of analytics will result in a better understanding of the human language by the computer, and better and more accurate responses. Through machine learning, these databases are growing every time a request is being made to their servers, which allows them to constantly learn more about the human language.

Issues with VPAs

The first few years of Siri was quite frustrating for users, as there was a plethora of problems that accompanied Siri. A few more notable frustrations included: Misunderstanding of simple sentences, failure to use information in a previous inquiry in a new inquiry, accent recognition. An example of well-known misunderstanding of a simple sentence was the difference between Call me an ambulance and Call an ambulance for me, in which the result was largely different as only the latter one would result in an emergency response. The first one ended up being Siri understanding it as the user wanted “an ambulance” as his/her nickname. Any inquiries were also not linked, therefore you would have to restate your whole inquiry every time, i.e. if you asked consecutively but separately, “Check my Chicago flight for flight delay”, “what’s the weather there?” would result in a confused Siri replying “Where?” for the second inquiry. Lastly, heavy accents were unrecognized until more recently, where companies have moved in the direction of supporting more languages as well as regions, resulting in a better recognition for accents through machine learning. [11]

Future of VPA

Home appliances

Futuristic Ideas

With the emergence of so many smart home devices, several other ideas are currently being developed and may be seen in the market in the future. There are already prototypes developed that have not been officially released yet as changes are continuously being made. These additional smart home devices incorporate furniture, similar to the smart TVs or smart fridges. Two emerging ideas are smart mirrors and smart countertops.


Smart Mirrors

There is quite a high possibility of seeing smart mirrors being used near in the future. The idea is to have a touchscreen mirror built in with microphones in order for connected virtual assistants to take commands. The mirror is expected to contain features such as allowing the user to read news, emails, weather reports, schedules, and display voice search results. The purpose is to enhance people’s morning and night routines in front of the mirror so that they can look straight ahead at the mirror rather than down at their phone which may be inconvenient when their hands are busy brushing their teeth. [12] The mirror is able to behave like an ordinary mirror as well, looking as simple and minimalistic as possible and only featuring the necessary information on screen to enhance the user’s life without being too complicated.

Smart mirror technology has already been featured in several movies such as The 6th Day, featuring Arnold Schwarzenegger looking into a smart mirror similar to what was described above. [13] These film features of smart mirrors has actually inspired Max Braun, a Google engineer, to develop an actual prototype, which is still under progress and has not been released to the public yet. [14] However, he published his process of building the prototype in 2016, allowing other people to DIY their own smart mirrors at home. This was done through a two-way mirror with electronic controllers behind it. Now in 2018, the product is still not finalized and ready for distribution, but it is something we can expect for the future.





Smart Countertops

Smart countertops are expected to be used in kitchen in future years to enhance people’s kitchen experience. There has been designs of futuristic concept kitchens featuring touch screen countertops that are equipped with voice recognition. The idea is that while cooking, users will be able to summon recipes on the counter for easy access and use voice command for everything since their hands will be busy. This leads into the potential to communicate with friends while cooking, and access other information such as news reports, articles, or status updates from friends all while away from their mobile device. [15]













Automobiles

With the trend of AR technology being implemented into all sorts of industries, it will slowly become an integral part of industry processes in the near future. With the automotive industry in particular, we believe that it will become the “next big thing” to be introduced into mainstream cars. Just like ABS, airbags and seatbelts, we believe it will be the next safety feature that will be implemented into cars in the future. With complications with autonomous driving cars, especially with the incident with Uber’s autonomous car killing a cyclist in Arizona [16] and legal issues [17] when testing autonomous technology, we believe mainstream introduction of autonomous driving cars will take much longer in comparison to AR integration into cars.

AR Glasses

Regarding the future of AR glasses, we believe it will become much cheaper than it is today. With the entry price of AR glasses costing above $700 USD and with the Toshiba example in particular costing $1899.99 USD. Since there have been failures, Intel for example, with major manufacturers attempting to take a piece of the AR glasses industry, it would not be surprising if this technology takes another several years before it matures and become mainstream. Without major investment from big companies or big investments from rich individuals, it will also extend the amount of time before AR glasses will become the norm.

Hospitality

The current challenge with the implementation of VPAs in the hospitality industry is the lack of personalization and multiple IoT items that are needed for a seamless experience. The trend of VPAs in the hospitality industry is heading towards smarter rooms to create personalized experiences for their guests using IoT system controlled by VPAs [18]. According to Intel, there will be 200 billion smart objects by 2020 [19]. Meanwhile, Gartner predicts the average home could have 500 smart devices by 2022 [20]. These devices at home can sync with users’ account and the VPA will gradually understand their taste and preferences overtime. Currently, guests cannot sync their own accounts at the hotel. It cannot sync their alarm clock with the drapes, and their favourite song together, or make Amazon purchases. Marriott and Amazon are working extending the convenience of their VPAs into hotel rooms like the guest have at home [21]. In fact, Deloitte predicts hotels will have the ability in which their Wi-Fi network will recognize a guest’s smartphone, which will send instructions to the property management system to have their doors unlock upon arrival and have the lighting and temperature set to their predetermined preferences [22]. Hotels that anticipate their guest’s needs before their arrival is becoming a reality at most tech-forward hotels, and smart guest rooms are the peak of personalized customization [23].







Retail

Walmart and Google announced a new partnership that will enable voice shopping through Google Assistant. Walmart’s new Easy Reorder feature, Walmart shoppers will be able to place spoken orders for products that they buy frequently. The app understands the customer’s personal preferences based on their prior Walmart.com and in-store purchases. The future of VPA in the retail industry will place pressure on traditional supply chain management as retailers are expected to meet the multichannel consumer’s increasing expectation for convenience and speed. According to Forbes, store-based retail supply chains will have to shift to a hyper-personalized, smart consumer supply chain in order to meet consumer’s expectation. The impact of this new supply chain will punish brands accustomed to shelf-centric demand [24].

VPA will alter the retail space and the shopping experience, leading to more interactive moments that brands can tailor to the consumer's individual tastes and preferences[ii] [25]. VPAs can minimize costs and risks for users but also offer them convenience that has never been offer before. The convenience of VPAs ensure that retailers are ready wherever and whenever their customers need them. From the retailer’s warehouse to the households, there will be an uninterrupted flow of routine purchases to households and a continuous improvement on learning consumers’ criteria and optimizing whatever trade-offs people are willing to make for more complex shopping decisions.[26]VPAs will provide retailers with an unprecedented opportunity as consumers use them to find, research and buy.








Personal VPA

Apple

Siri

How does Siri work? Siri uses voice recognition on the device locally, then connects to Apple servers to run it through Apple’s algorithms and then produces results which is sent back to the device for the user.

[27]

List of iPhone Siri Capabilities:

  • Call someone/ Message someone
  • Check or play specific voicemails
  • Send an email
  • Provide weather
  • Conversions and questions that can be answered by wiki
  • Calculations
  • Take a picture
  • Change settings on phone (i.e. turn on/off Bluetooth, wifi, cellular data, increase/decrease brightness, etc)
  • Schedule meetings, events, reminders
  • Set alarms
  • Word definitions
  • Search files (Mac specific)
  • Search Twitter for keywords or find out what’s trending
  • Help with directions, ETA, find nearby businesses
  • Updates on flights schedules
  • Translation
  • Third party support:
    • Uber/Lyft
    • Whatsapp/LinkedIn/Skype/Wechat/Line, etc
    • Paypal, Square Cash
  • Smart suggestions such as “order starbucks coffee whenever you’re 3 minutes away from a certain coffee shop” “Reminder to call someone for their birthday”
  • Custom phrases to Shortcuts
    • Such as get the weather report, traffic information, ETA to work, turn on coffee maker, radio, raise blinds, etc., programmed into one voice command

Devices

WatchOS

WatchOS has become one of the major platforms for accessibility to Siri, as well as providing more and more functions that assist people. These functions include:

  • Distance and pace monitoring for walking and running (2015-now)
  • Start workouts through Siri and support for third party fitness apps (2016-now)
  • Step-by-step directions through maps (2016-now)
  • Personalized support through Siri’s machine learning algorithms and intelligence (2017)
    • Commute time in the morning
    • Nearby restaurants around noon
    • Flight time update/ boarding pass at night
  • LTE connectivity for Apple Watch, without the need for an iPhone nearby (2017)
  • Raise-to-speak Siri (2018)
  • ECG feature to detect atrial fibrillation (2018)
  • Fall detection, automatic contact for emergency services (2018)

The more important functions that has vaulted the Apple Watch from just another VPA to a VPA that could save lives are their ECG feature that can detect atrial fibrillation and their fall detection feature that automatically contacts emergency services. With falls being the second leading cause of accidental or unintentional injury deaths worldwide at 646,000 [28], the benefits of fall detection can definitely save many lives.

Amazon

Alexa

Alexa is Amazon’s personal virtual assistant featured in its Echo products. As a second generation voice assistant, Alexa does not require any buttons and can be activated through just a trigger word “Alexa” followed by the command. As soon as Alexa hears the trigger word (which is “Alexa” by default but can be changed to “Amazon”, “Echo”, or “Computer”) the microphones will pick up the user’s voice and send it to Amazon’s cloud computing system where they will analyze the request and complete the command. Through built-in phones and a speaker, Alexa is able to listen for commands and answer any questions. It is used for multiple functions in people's homes, such as the following: [29]


  • Answer questions and deliver knowledge
  • Call or message contacts
  • Play music and videos on command
  • Control smart home appliances (light switch, door lock, heater)
  • Set alarms
  • Search the web
  • Shop using Amazon Prime

Devices

Echo

Amazon Echo is a Bluetooth smart speaker launched with a voice-controlled virtual assistant called Alexa. Essentially, the Echo speaker allows users to use all the usual functions on their phone and control compatible devices using only their voice without needing to actually pull out or touch their phone. It is designed as a cylinder with microphones and internet connectivity. Echo has the ability to connect with smart home devices as long as the smart devices are installed first in its application then connected to Alexa. There is also the option to connect to an external home hub (such as SmartThings) to help directly connect all smart home devices. [30]







Echo Dot

The Echo Dot is a small version of the Echo speaker shaped like a hockey puck, which is sold at a lower cost with lower sound quality due to its smaller size. However, the Dot could be connected to other speakers for better sound quality. This product was designed to be a mini version of the Echo that carries all Alexa capabilities. This was marketed to be a cheaper alternative to the original Echo for those that want the Alexa voice assistant technology but do not wish to invest in a higher quality sound or larger speaker.[ii] [31] Also compatible with smart home devices, it gives the opportunity to take advantage of smart home capabilities without spending more money on the larger Echo speakers.





Echo Spot

The Echo Spot is a combination of the Echo Dot and the Echo Show. It is small and in the shape of a half sphere, taking after the mini size of the Dot. However, it also has a small 2.5-inch screen such as the Show and is equipped with a camera, which allows video calls. The difference between the Spot and the Show is that the Spot does not have smart home compatibilities. It is commonly used for alarms and playing music - it can also be connected to speakers for better sound quality. The size is also considerably smaller with a tiny screen and is meant to be a mini portable bedside device. It is also used as a convenient device for viewing photos and videos.










Echo Show

The Echo Show has a 10.1-inch touchscreen (rather than just being a speaker like the other Echo products) with speakers on the side. This purpose of this product is to add in video calling features through the larger screen, while still including the Alexa features of the Echo. This product is compatible with smart home devices as well. The hands free video calling option allows users to ask Alexa to call anyone from their contact list that is also using Alexa. A new upcoming update plans to allow users to video call anyone through Skype, so the other party does not have to be an Alexa user to be able to use this feature. This has mostly been marketed towards parents and grandparents who are especially interested in video calls on a larger screen without having to actually hold the screen or touch anything on there. Echo Show is also able to show videos and photos on their screen, as well as answer all Alexa inquiries in a visual way. For example, if you ask “Alexa, what movies are playing right now?”, the Show will be able to display the current movies playing. Or if you ordered food delivery, you can ask “Alexa, where is my food order?”, and it will be able to show the status of your order and where it is. [32]







“Drop-in” Feature

The “drop-in” feature is available on Echo Show and Spot devices and allows authorized users to listen and watch the room (only if using Echo Show which has a built-in camera), which people are able to use as a security camera at home as well. Since the Echo Show can also connect to smart home hubs, users can control other devices through the Show while seeing a visual image of what's going on. For example, by asking “Alexa, who’s at the door?”, the screen the show you a live feed of the front door, like a security camera. When users drop in, there will be a pulsing green light as well as a sound to alert the user to show the other end that you are watching in, and the video will be fuzzy for the first 10 seconds in order to not catch the other side completely off guard. [33] This function can be disabled, and people will not be able to drop in unless you have given permission for them to do so. This was a controversial feature due to lack of privacy, but only those with permission are able to do this so it is up to the user. It’s commonly used as a security camera, or to see parts of the house while they’re away. It’s also used as a pet or baby monitor to drop in and check how they are doing on a live feed whenever you are not in the room.

Google

Google Assistant

Google Assistant is a virtual personal assistant developed by Google, available on smartphone android devices running on Android 6.0 Marshmallow and higher and smart home hubs. It has the advantage of being owned by Google, since it can search through the Google search engine, access all Google account information, adjust settings on mobile phone or smart home devices, schedule things and send messages, and respond back in natural conversation. It is able to logically detect which direction the conversation is going so the user can speak to it as if its a human. For example, if you ask “What restaurant should I go here”, the Assistant will respond back showing a list of restaurants nearby. Then without repeating anything and just asking “Is this restaurant any good?”, the Assistant will pull up reviews. Google Assistant was surveyed to be the most accurate virtual assistant as of 2018, answering 90% of questions accurately and attempting 80%. [34]

Having access to all a user’s Google Accounts, Google has the advantage of possessing more information about its user and can access Gmail message by voice. Google Assistant can also detect up to 6 different voices and customize commands based on that (such as giving information about different accounts or schedules depending on the person asking). There are also different voices that can be assigned to the Google Assistant, and different users are allowed to pick different voices to go with their virtual assistant. Google Assistant also has the ability to translate languages when asked, so it can take what you said to it and give you a translation in a different language.

Devices

Google Home

Google Home is Google’s line of smart speakers with voice assistant technology. Similar to Amazon’s Echo speakers, the Google Home is able to respond to commands and connect to smart home devices and complete tasks as requested. Google Home only starts to pick up on what people are saying in its surroundings when it hears the trigger words, “Ok Google” or “Hey Google”, which is when it begins to process your commands and respond. Otherwise, it will be listening in the background but not processing anything. There are a few physical controls at the top of the Google Home device as well that allow the user to adjust volume, start or stop music in addition to using voice commands. As a speaker, Google Home can play music from Spotify, Youtube, Google Music, and Pandora, and can all be controlled by voice. If the user can’t remember the song title, they can even search by lyrics to find the song. It can also be connected to external devices like third-party speakers or TVs to play the music on instead if the user prefers. The device also has a child-learning system through a system known as “Pretty Please” in which the adult can set up a special word that the child must say in order for the Google Home to complete the command. For example, if the child wants to hear a certain song, they need to follow up with “Please” in order to hear the song. [35]

This helps enhance the child learning process as children learn to develop manners in order to get what they want instead of just yelling a command at the device.





Google Home Mini

The Google Home Mini provides the same functions as the Google Home, but is physically much smaller, in a pebble-shaped device that is only 4 inches in diameter. Similar to Amazon’s Echo Dot, the Mini was created as a more portable and affordable product with a lower quality speaker that can still be connected to external Bluetooth speakers. The price of $49 is much cheaper, making it much more affordable for people that want to have Google Assistant features in a speaker and don’t mind a slightly lower quality of sound. [36]






Google Home Max

Opposite of the Google Home Mini, the Max is a larger version of the Google Home with higher quality stereo speakers and multiple adaptable audios and Type-C connectors. This product has the same functions as the Google Home in terms of Google Assistant technology but is purchased for its loud sound quality that is able to fill up a room, resulting in people using this device as a sound system for their home. With the powerful sound it produces, external speakers are not required as the Home Max can cover all entertainment purposes for home use. [37]

















WearOS

WearOS is Google’s solution to a wearable operating system, the difference between WearOS is that it supports many different manufacturers and is not limited to just one brand’s products. This creates a large variety of products that consumers can choose from depending on their preference. The functions of WearOS is largely the same as any other smart wearable device, with Google Assistant as the VPA in this device, with Google pay as the paying function.















Several notable WearOS watches by different brands [38]:

Louis Vuitton Tambour Horizon – Luxury watch priced at $2,900, waterproof up to 30m, includes a 24-hour clock that shows weather and temperature for different time zones, and has a “City Guide” application geared towards global travelers.














Movado Connect – Sleek minimalist design priced at $895














TAG Heuer Connected Modular – Most expensive smart watch at $180,000, paved with white gold and covered in 589 diamonds, doesn’t have any additional capabilities over regular smart watch functions














Garmin Fenix Chronos – Extreme fitness tracking watch at $1,500, in addition to regular smart watch functions, it has heart rate sensors, can count steps, measure running speed, monitor sleep, measure calories burned accurately depending on different activities (cycling, swimming, skiing, paddle -boarding, rowing, weight-lifting), with the help of a built-in GPS.














Non-Google Devices

Lenovo Smart Display

The Lenovo Smart Display works similar to the Google Home Hub with Google Assistant and smart home compatibility features. However, it sports a larger touchscreen of either 8 or 10 inches and has a built-in camera for video calls. [39]

Along with a larger screen, the speaker placements make the sound more powerful than the Home Hub. It costs more than the Google Home Hub due to its built-in camera and higher sound quality that is needed for video calls.











Samsung

Bixby

Bixby is Samsung’s virtual assistant that was built on the original voice technology from Samsung known as S Voice. The new version, Bixby 2.0 was developed by Viv Labs, which is a virtual assistant software company that developed Siri as well before being purchased by Samsung. Created in 2017, Bixby is much newer than other virtual assistants in the market such as Alexa or Google Assistant and are still developing to provide more conversational, natural replies. S Voice technology was initially introduced with the Galaxy S3, while Bixby appears a feature for newer higher end phones starting from the Galaxy S8 , Note 8, and Tab 8. This voice assistant also became incorporated into Samsung’s smart TVs, as well as being the first to be used with a smart fridge. Bixby also appears in the Galaxy Home, Samsung’s own voice-controlled speaker. There were concerns over Bixby that it is not up to par with more popular and widely used virtual assistants already established in the market - Alexa and Google Assistant. Since both those virtual assistants are supported by Samsung mobile devices, many people prefer to disable Bixby in favour of the competitor technology. Although Bixby is quickly catching up to the AI capabilities of the other virtual assistants, it does not offer any competitive advantage as far as mobile devices to convince users to use it over the other options. Bixby also faced an issue of being accidentally activated that users found to be intrusive, as well as a notification on the Galaxy S8 that prompted the user to use Bixby every time they opened the camera app. [40] This led to people trying to disable Bixby and turn the Bixby button into a Google Assistant button, which they preferred to use instead. As a new device, Bixby still needs to earn the trust of users and they still have future developments that can catch them up to the other established virtual assistants.

Devices

Bixby on mobile devices

Currently available on Galaxy, Note and Tab 8 products and above, Bixby made its debut on these mobile devices as Samsung’s own voice-activated assistant similar to Apple’s Siri, Amazon’s Alexa, or Google’s Assistant. These devices all come with a Bixby button at the side of the phone that can be used to activate Bixby, or the user can simply say “Hi Bixby”. Similar to other virtual assistants, Bixby is able to listen for voice commands to read out the daily plans, news, answer questions, surf the web for answers, or even call an Uber ride. [41] These mobile devices are also equipped with Bixby Vision, which opens up the camera and is able to scan QR codes or scan text in Korean then translates to English, or vice versa. It is currently still limited in languages and can understand English, Korean, and Mandarin Chinese (only in phones purchased in mainland China).

Samsung Galaxy Watch

The Samsung Galaxy Watch runs on Bixby, and provides all the regular smart watch functions such as:

  • Checking notifications
  • Calls/ Messages
  • Fitness Tracking
  • Sleep Tracking
  • Samsung Pay

The function that differentiates the Samsung Galaxy Watch with its competitors is the 3-4 day battery life provided, which allows for sleep wearing and sleep tracking.

Sleep Tracking

Although sleep tracking doesn’t provide any medical usage at the moment, there are applications out there that harnesses this function to help track the data of your sleeps, and if you’re subject to insomnia, you can bring this data to the doctor to help with a diagnosis. Another function of sleep tracking is that the watch is able to tell when you’re in non-REM (rapid eye movement) sleep, and REM sleep. This allows for the application to ring your alarm during your lighter sleep periods, given that you give it a time range to wake you up at, providing a better and less grumpy morning. Other general benefits found by studies include that tracking a habit allows for and is more likely for improvement. [42]

Gatebox

The Gatebox is a product that belongs to Vinclu, a Japanese technology company acquired by LINE. Gatebox was designed after Amazon’s Echo, which similarly connects to smart home devices and interacts with the user. However, Gatebox put their own twist on this product by matching the voice to an actual character, giving it human characteristics and allowing the product to work as a virtual companion. Gatebox provides a virtual companion in the form of a female character called Azuma Hikari, who converses in only Japanese. Through infrared sensors, cameras, and projections, this character appears as a holographic female animation inside a glass case and comes with a description of characteristics (20 years old, likes donuts, does not like insects) to help her embody an actual human. [43]

She was designed to appeal to single men living alone in Japan and to provide them with company while taking on the role of a virtual girlfriend or wife. Through artificial intelligence (AI), a motion detector, camera, and microphone, she is able to intelligently engage in conversation with the user by detecting their faces, movement, and understanding their tone. Gatebox comes with Bluetooth and WIFI which allows her to connect to other devices and learn more about the user’s personality and habits in order to offer a more personalized conversation. [44] She is also known to provide advice and emotional consolation to users. The character Azuma Hikari is also able to interact with the user throughout the day by sending text messages. Many messages are designed to imitate an actual companion by sending messages such as “I miss you”, and “When will you be home?” Through these Bluetooth and WIFI connections, the Gatebox is able to connect to other electronic devices and control them.

Some of its capabilities through talking to the Gatebox include the following: [45]

  • Search for information through the internet
  • Used as a remote control for smart home devices (turning on/off lights, the TV, heater)
  • Receive weather forecasts and news reports
  • Remind users of their schedule and acts as an alarm (good morning calls)

The benefits of the Gatebox is its ability to act as a human like companion and interact with the user in a natural conversation manner. It can be used as a smart home device by controlling things as well such as turning the lights or heat on and off or setting an alarm, but does all this while acting like an actual person helping the user out. People have even connected smart vacuums to this product so that it clean the house while the user is out and can come home to a cleaned home, imitating a companion. The downside to the Gatebox is the idea of replacing human with technology to the point where people may not see a point in human interaction or relationship anymore, and may make users too connected to a holographic character. However, this product priced at $2700 has done extremely well and preorders were sold out. [46] While the Gatebox is currently in Japan, there does not seem to be plans to market it in other countries other than potentially the US, due to cultural differences as it is not as widely accepted in other countries as it is in Japan.

Smart House Application

Hub Devices

HomePod

Homepod is Apple’s response to Amazon’s Alexa and Google’s Google Home. Released in June 18, 2018, it fits amazing in to Apple’s ecosystem of products, being able to connect to the iPhone, Apple TV, Mac and any HomeKit connected products such as lightbulbs, blinds, air-conditioner, etc. Features in HomeKit include having voice activation for controlling “scenes” (different accessories together), “zones” (different rooms together), and automation settings as well including motion sensor for when you get out of bed and have a sequence of “turn on plug for coffee maker, open blinds, etc.”.




Echo Plus:

Amazon also developed Echo Plus, which works the same as Amazon Echo except it also includes a Zigbee smart home hub to connect to smart home devices. The benefits of this is that the device will detect other home devices and link to them directly without having to connect to a external home hub device. The Plus can connect and communicate directly to other smart home products without requiring wifi since some products do not support wifi and operate off Bluetooth. This reduces the need to purchase a separate third-party home hub since it is already built in. It is also a larger and more powerful speaker than the original Echo. Equipped with a Dolby speaker, the Plus puts a bigger emphasis on the speaker portion of the product by increasing its audio quality, possibly replacing regular speakers or sound systems in customer’s homes. It will still perform the same tasks as the Echo, just with more powerful sound. [47] The Echo Plus makes it easier so that they will all be controllable under the one Zigbee app rather than several. Although not as extensive as a smart home hub due to the more limited amount of compatible devices, the Echo Plus serves as a beginner smart home device that allows users to control light bulb brightness, adjust the temperature, control outlets, and control locks through voice command. [48] Even without connecting to a smart thermostat, the Echo Plus has a temperature sensor built in to read out the room temperature if the user asks. However, Amazon’s built in home hub in the Plus is still just a basic system that is not as advanced as other home hubs. The difference is that other home hubs can request things such as “play music if motion is detected” while for the Echo Plus, you would have to manually tell it to play music each time. In other hubs, you could also ask to pick the song to play, while for Echo Plus you would need to decide on a song then request Alexa to play that specific song. For users with multiple Echo speakers, these can even be used as an intercom system throughout the house. There is a “drop-in” feature that allows authorized users to talks through these speakers without additional permission, which allows families to communicate with each other without physically walking over or raising their voice. [49]


Google Home Hub

Google Home Hub is a smart home system with a 7-inch touchscreen that can access and control smart home devices both through the screen or through voice commands. All Google Assistant functions are usable on this device, and it can connect with over 10,000 smart home products including light bulbs, switches, doorbells, outlets, and speakers. It can answer questions the same way any Google Assistant device would, accompanied by visual answers as well such as pulling up photo or trailer of a movie you want to see or showing the weather forecast. Although similar to the Echo Show due to its touchscreen, it does not have a built-in camera meaning there is no option to video call, although regular voice calls can still be made. The camera was purposely left out to ensure a feeling of security so that people can bring the portable device around to places like the bathroom or bedroom without fearing any privacy invasion. The lack of a camera also allows Google to price this device lower as well. [50]







Appliances

The Smart Home Appliance

The following are various types of smart home devices along with a few examples of each device, their capabilities, and their corresponding virtual personal assistant:

Fridge

  • Samsung Family Hub 2.0: Samsung’s smart fridge is Bixby voice activated with a large touch screen on the outside which can be used as a tablet to look at weather, calendar, recipes, and access media. There is a built in camera to see the inside of the fridge, which can be connected to other smart home devices as well to see the fridge from a smartphone while grocery shopping (to determine what groceries are needed). There is an option to make virtual grocery lists, as well as the ability to put expiry date labels on each food item that will automatically update so customers can keep updated with when to buy new groceries[51]

Clock

  • Amazon Echo Spot: Amazon’s Alexa voice controlled Echo Spot also doubles as an alarm clock that is compatible with smart home devices. It has the function to make calls or video calls, and can play videos and show photos as well through a small touch screen. [52]










  • Insignia Voice Smart Bluetooth Speaker and Alarm Clock: Google assistant voice controlled alarm clock with touch buttons to change the volume and the ability to stream music. [53]










Blinds

  • MySmartBlinds: Controlled by both Google Assistant and Alexa, MySmartBlinds is compatible with home hubs and can be voice controlled to open or close. Users can also preset schedules to open or close at a specified time, or track the sun to optimize the amount of light to let into the room. These can be controlled outside of bluetooth range, such as through connected devices when away from home. [54]










Switches

  • Ecobee Switch: Voice controlled by Alexa to switch on or off, play music, or read out the news. This switch can be controlled through sensors as well, and can sense if a person is in a room or not. [55] It can be automated to switch off once the room is empty, or switch on when someone walks into the room.














Lightbulbs

  • Philips Hue: Voice controlled to turn off/on, adjust level of brightness, and just colour and tone. These lightbulbs can be controlled through smartphone or other smart home devices, and can control specific rooms from afar through a command such as “turn on lights in the living room”. This works with most VPAs such as Alexa, Siri, Google Assistant, and Cortana. [56]
















TV

  • Samsung Smart TV: Samsung’s Bixby controlled smart TV’s come with a remote control with a microphone to talk into. This is connected to the home hub as well as mobile devices, and can be voice controlled to change the channel, volume, and play/pause the TV. By connecting with the Smart Fridge as well, users are able to view the fridge from the TV screen as well as other photos and videos from their phone. [57]

Thermostat

  • Echobee3 Smart Thermostat: Uses Alexa, and is voice activated to change the temperature. It can also be controlled from mobile devices from afar, or through room sensors that detect is a user is in a room or not to automatically set the ideal temperature if there are people present. There are also energy reports to offer suggestions for energy saving. [58]










Smart-locks

  • Kwikset Kevo Smart Lock: Kwikset Kevo is a bluetooth enabled smart lock using Alexa. It can sense a nearby phone outside the house and allows access by tapping the lock. It can also sense if a phone is outside or inside the house, and will only allow access if the phone is outside the house so that strangers would not be able to tap the door and walk in. Has the ability to issue temporary eKeys to visitors if the user is away, and can also be voice commanded to lock/unlock the door or check the status of the lock. [59]
  • Schlage Connect Smart Lock: Uses Alexa and Z-wave technology (smart hub) to lock or unlock from anywhere. This locks with a pin and is equipped with an alarm so will ring if the door is experiencing force. Door will also automatically lock after 30 seconds and eKeys can be issued as well.[60]
  • August Smart Lock: Wifi-enabled smart lock that uses Siri, Google Assistant, and Alexa. - Siri, Google Assistant, Alexa. This lock can track the activity of people entering and leaving the house. It also auto-locks after a preset time and temporary eKeys are available.[xi] [61]






The Smart Home Experience

Alexa and Google Routines:

Alexa and Google both includes a “routine” function which can activate and combine multiple smart devices through just one voice command, which enhances the smart home experience through simplifying commands. For example, users could create a command called “good night” that will turn off the lights in all rooms and turn the thermostat up to a preset temperature. These groups of activities can also be set up to be automated at certain times without requiring a voice command [62]

The purpose is to avoid saying each command individually if they are all commonly done at the same time, so these routines are able to make user’s lives easier by allowing them to just give one command to control all the orders at once. Another example is telling your Echo or Google product, “Alexa/Google, I’m going out”, and have an automated string of events such as turning off the thermostat, shutting the blinds, and shutting off the lights. [63]

Enhance Security: With the existence of voice activated smart locks, people are able to track security features such as seeing when people are entering or leaving the house, and are able to open locks just through a close proximity of a phone outside the house. The strengths of this experience is the added convenience of not having to worry about physical locks or forgetting to lock the door. In a sense, it is also more secure because people will not be able to find your lock and walk inside, or pick your lock. However, the weaknesses also relate to security due to the lock’s reliance on phone Bluetooth signals. Malfunctions occur with technology, and there are security issues associated with placing so much trust in having technology guard your house in the case that it fails to work correctly and allows a stranger to walk inside. Overall, the leaving and returning home experience is enhanced through these functions that allows you to quickly leave and enter your home without the concern of locking your door in case you forget or the concern of forgetting your keys.

Adjusting Lighting, Blinds, and Temperature: Now that lightbulbs, light switches, and thermostats can all be voice activated, it reduces the need to physically walk over and turn on or off the light or heat. Instead, users just call out and request for those items to turn on or off, or even rely on the sensors so that they wouldn’t even need to call out orders. These strengths make people’s lives easier by reducing the physical work they need to do. It is also beneficial for the environment by automating some of these processes, so that even if the person forgets to turn off the lights before leaving, they will automatically shut off when the rooms are empty. The weaknesses of these are the reliance of automation and the increased laziness in people. When all these functions are automated or voice controlled, it reduces the need for people to move or remember to do things. However, the home experience of adjusting lighting and temperature is greatly simplified through these processes. Different products may differ in functions offered, such as the Philips Hue lightbulb being able to dim the lights or change colour, while others such as the Echobee Switch using Alexa can only turn the lights on or off. [64]

Video calling: Voice automated calling is especially useful when people have their hands busy but want to make a call, or just don’t want to bother looking through their contacts for the right person. The use of VPAs in. Voice or video calling whether on a mobile device or on a tablet or Echo Show product is made much simpler for users, and also easier to use for those that are less familiar with technology such as grandparents that want to video chat their family. The weakness of this function is the opportunity for mistakes - users could easily accidentally call someone or call the wrong person in case the VPAs pick up the command incorrectly, such as getting the wrong contact name or accidentally calling someone without the user being aware since it is quite likely for them to accidentally give an order. This can enhance home processes such as making calls while cooking or just when the user’s hands are too busy to physically dial someone. These video calls can be made on smart home devices such as the Echo Show, Spot, and Lenovo Smart Display.

Industrial VPA

Hospitality

Marriott and Amazon are working together to bring the hospitality version of the Echo devices into the hotel rooms [65]. This will allow guests to order room service or housekeeping by asking Alexa. This saves the hotel time on simple questions and services, which reduces workload on staff. Hotels like the Marriott are experimenting with voice-enabled devices to help their brand with different channels to reach their end-customers. Virtual personal assistants are becoming more ingrained in everyday life. It is natural to see that VPA are transitioning into the hospitality industry as more people are using them. According to an NPR-Edison Research study, the rate of smart speaker’s adoption eclipse that of smartphones [66]. Additionally, 18% of adults or 43 million individuals in the United States own a smart speaker [67]. Globally, ownership have exceeded 100 million units and is projected to reach 225 million by 2020 [68]. The increase usage of VPAs will have significant implications on marketing. VPAs will serve as a new channel for customers to easily interact with salespeople, customer support, etc. As more consumers regularly use their VPAs for their travel needs, there will be less of an interaction with the traditional industry’s store fronts. VPAs are capable of amassing travel options on command. This can potentially cause disintermediation in the travel industry, eliminating traditional travel agencies in the future [69] The hurdle for traditional hotels will be defending their margins, brand, and relationships with customers. Meeting the expectations of their guests is the key to getting people to return, and increasingly, hotels are seeking advanced analytics on how to meet those expectations[vi] [70]. VPAs will be able to do this with the amount of personal data about customers that the big tech companies have [71].







Human and Automation Balance

Another problem may be the balancing of the human factor and automation efficiency. At the Hen-Na Hotel in Japan, robots are used to automate most of the hotel’s services; from the front-desk, cleaners, and porters. By using robots, the hotel is able to become more efficient and reduce staffing cost. Japan have been the leader in replacing humans with robots for years. Their innovations are driven by the low birth rate and shrinking population of their country [72]. The simple act of removing friction between human and machine can be tempting to fully automate systems and processes to lower cost and improve efficiency. However, finding the balance between human and automation capabilities of a VPA will be crucial as hotels will always need the social element [73]. Hotels should empower staff with analytics. Armed with dashboards on their smartphones, they would enable them to anticipate what a particular guest might expect from their stay, leading to happier customers. [74].








Retail

Most retailers are in the early stages of adopting VPAs. They are currently using chatbots and apps to assist their customers in finding their ideal products and services. Their chatbots help customers with their problems through a simulated conversation and then respond using voice, text, photos, web content and emojis[75]. For example, through the collaboration of The North Face and IBM Watson, their digital shopping tool aggregate relevant coat suggestions for online coat-shoppers based on a simulated conversation. Macys on Call App is a mobile web tool that allows shoppers to navigate their retail stores for specific products, departments, and brand through a simulated conversation. [76]. Although chatbots and apps are useful, they are limited by the context that the user provides. People make purchases every day from thousands of products and services available online. Sorting through every product and services can be a nuance, even routine purchases when conflicted with multiple promotional offers. Unlike chatbots and apps, VPAs continuously improve overtime, understanding their users’ taste and preferences based on their history. Shopping will be more efficient for customers as VPAs can navigate for the ideal product or services through an overwhelming list of items in a matter of seconds. [77]










Growth of Retail


VPA could be a major driver of future retail as they present retailers with another channel for their customers to reach them. For instance, RBC Capital Markets projects that in addition to other revenue platforms, Alexa could become a $10 billion business by 2020, with its voice-driven shopping [78]. According to Stastica, the number of people using virtual personal assistants on a regular basis will grow from 504 million to 1.8 billion [79]. Voice searches on Google are increasing as virtual assistants are gaining more popularity. In fact, Google CEO, Sundar Pichar stated that voice queries on Google already account for 20% of searches on mobile devices [80]. Over the next decade, there will be more competition among retailers to establish themselves on VPA platforms. VPA will transform the traditional channels that customers have use to interact with brands. The growth of smart speaker adoption will make VPAs the primary channel through which people get their products, services, and information, and it will be a competition for their attention.










Automotive

Porsche and Mercedes Benz

Currently, Porsche is using AR technology called Augmented Reality Cubing in their production line to detect any defects in the gaps for the windows and doors of the automobiles they currently produce. They take a photo of the panel they want to inspect and loads it onto the tablet to compare with the laser-perfect version of the panel. This technology allows workers to know of the production details and see where defects are located much easier than before.[81] As a result of this Augmented Reality Cubing, it will lessen the probability of recalls for defective cars.

In the automotive world, where present cars are much more technologically advanced when compared to those produced 20 years ago, problems cannot be solved as easily as before because of the complexity of these cars. The average mechanic will have issues troubleshooting through newer technologies, such as direct injection, navigation, driver assistance technology and hybrid systems, and may result in owners to leave their car at the dealer for a couple days.



To solve this issue, Porsche is providing its technicians in their dealers with AR glasses to help mechanics with this common problem. These glasses will allow mechanics to consult and solve issues with head engineers at the Porsche headquarters in Germany. Using this technology, it will allow mechanics to “… get their customers back into cars with less disruption…” [82] according to Klaus Kellmer, the CEO of Porsche Cars North America. The AR glasses that Porsche uses is the ODR R-7 smartglasses, which are made by a San Francisco-based company called Osterhout Design Group. These glasses “… allow wearers to see the real world while other images can be overlaid in the periphery.” [83]Users can also stream a live video through its’ camera for remote support work from anywhere around the world. Viewers of the stream can then provide suggestions which would appear within the mechanic’s peripheral vision. If there no available mechanics or engineers available, the ODR R-7 can be used to display the manual to aid the user. Through the usage of AR glasses, Porsche has been able to shorten the amount of time customer’s cars are inside the dealer for repairs; therefore increasing customer satisfaction.

With customers expecting top level quality from Porsche, the implementation of this particular technology has greatly increased customer satisfaction of their products. From 2004 – 2016, Porsche has had the highest ranking in customer satisfaction according to J.D. Power[84], a well-known market research company. This study occurs every year and surveys customers 90 days after the delivery of their vehicle. These customers then evaluate the interior and the driving dynamics of their new purchase. With Porsche already being well reputed as manufacturer of cars with great driving dynamics, implementing Augmented Reality Cubing and AR glasses into their business has provided even further satisfaction to its customers.


Porsche and Hyundai have recently invested in WayRay[85], a startup company for AR in Switzerland, in order to adopt AR into their cars for safety and performance driving. At its current stage, it is a AR display for cars when they are driving on the racetrack. It will provide the ideal racing line (the fastest route) for the driver to follow, which would greatly improve their lap times at the track. It will identify specific braking points to ensure the fastest entry and exit speed between corners so that drivers can approach lap times close to professional drivers. This AR technology is similar to the heads up displays available in cars today but projected on the full windscreen of the vehicle rather than a small piece of plastic or a small portion of the windscreen.

Porsche and Hyundai hope that they can further develop this AR system into helping drivers identify crosswalks, warning signs, parking spots, construction zones and navigation directions all within the driver’s line-of-sight. Using this system, drivers will be able to identify areas, such as crosswalks and school zones, where they need to slow down earlier to avoid potential accidents.

            
              



Recently, Mercedes has launched its MBUX (Mercedes Benz User Experience) System in the new A-Class model. Its function is similar to WayRay’s AR system, but on a much simpler and smaller scale. This system only provides directions to its users and is displayed within the car’s infotainment system. Cameras located in front of the car live streams what happens in front of the car and then the AR arrows of navigation are displayed through the car’s system.

Aircraft




AR Glasses






With Boeing, they claim that the wiring production time has decreased by 25% and has reduced the rate of error to nearly 0%[86]. Using this wearable device, they are looking into implementing it into other parts of manufacturing and assembly operations within Boeing. These AR glasses are developed and provided by Upskill, a company that specializes in implementing AR technology to companies to improve the efficiency of business processes.

Upskill also provides AR glasses to companies such as GE Aviation and GE Healthcare. According to the Chief Executive Officer, Brian Ballard, “Upskill’s software platform… …brings data typically stored in books into workers’ tools. The system can be used in manufacturing, field service, and shipping and warehouse logistics…” [87]

The AR glasses from Upskill utilizes an intelligent operation system called Skylight. Skylight operates similar to Apple’s Siri during operations. For example, the user would say: “Skylight, I want to see the wiring manual” and Skylight will project it onto glasses. The AR glasses can also be operated through a touch pad located on the side of the glasses. Users can change pages on their manual by swiping forwards and backwards on the side piece.




Crunchfish, an AR company based in Sweden, is also developing AR glasses for enterprise use. This pair of glasses allows the user to use gesture control when scrolling through or accessing information. This also does not require the user to carry a pocket computer with them to use the functions. Already on sale for companies for implementation. Currently they are partnered with Epson, Vuzix, Optivent and ODG, all of which are well known developers of AR glasses.

AR glasses that are available for consumers are not as complex in comparison to those available for large businesses. Consumer grade AR glasses are usually bulky and requires tactile buttons for its operations. One of these includes the Toshiba dynaEdge AR Smart Glasses. It requires the individual to carry a pocket computer which connects to the glasses, resulting in hindrances for users. Consumer AR glasses also tend to be designed for a single function. The Everysight Rapter is developed for cyclist in mind and only provides the necessary information for its users (speed, heart rate and time).

            
              



Intel announced that they were working on a set of AR glasses, called the Vaunt, that is more stylish and only requires the movement of our eyes to operate them at the beginning of 2018[88]. They were hoping to create a more user-friendly device, where operation is not as difficult as the existing ones on the market. However, five months into the Vaunt project was cancelled [89].

Finance

Internal

In the past several years, AI has been slowly being integrated into the finance industry. With the introduction of AI that is capable of machine learning, companies have been able to improve in their ability to address customers enquiries. Bank tellers and entry level bank jobs have been slowly replaced by AI that can explain/solve problems.

The Fintech Association of Hong Kong has pioneered “… the use of genetic algorithm to generate trading ideas.”[90]Genetic Algorithm provides users the best solution through large and complex data sets. They claim that these algorithms will provide greater predictive accuracy, which would greatly benefit the bank and its clients. These algorithms “… narrow down and mix with evolutionary algorithms and then invest them, which generates growth.”[91]This algorithm will greatly improve and assist its users.

Some internal advancements that AI provides to the finance industry includes, loan approval processes and credit scoring and risk appraisals [92].These processes traditionally take a long time for results, but with the usage of AI the process time will be greatly reduced. Experts also suggest that “… real-time lending decisions will become widespread, as credit scoring and risk appraisals are similarly automated.” [93]. According to Huawei [94], innovations of such financial technologies will most likely be created through technology companies partnering with banks, rather than the banks developing these systems themselves. They also suggested that virtual assistance will rival the physical branch in importance within the banking channel just like how retail businesses have slowly moved into E-commerce models.

External

Technology firms that specialize in the finance industry will become the next big player within this market. Companies such as Emagia, that specializes in digital finance solutions has created a AI and analytics program. This program is called Gia; it is used as a personal finance assistant for finance executives in companies.

Gia can help internal teams process forms and provide conversational support and suggestions to its users. It is able to connect with enterprise accounting and finance software from providers including SAP, PeopleSoft, Oracle, Solix, Teradata and Emagia’s own solutions [95]. Using this access, it will be able to provide accurate and concise support and suggestions to its users. Users can communicate with Gia through chat and voice-based input. After a period of usage, Gia can learn to anticipate its user’s actions with its machine learning technology. Also, Gia is able to send invoices and check on company’s financial reserves.

Some competitors to Gia include Microsoft’s Cortana and Amazon’s Echo (business version). According to industry experts, it is a failing platform since it is currently integrated with Windows 10. They claim that the decoupling process from Windows could take 5-10 years [96]. Amazon is also working on a business version of its Amazon Echo as well. It will be able to assist users with its AI and analytics processing ability to provide users greater efficiency in gathering information.

Another financial assistant application is the Nami Assistant, which was introduced in Hong Kong at the iFX EXPO Asia in March 2018. Using Nami Assistant, “… traders will receive continuous support which minimizes the time spent on low-value-added jobs and they can focus on making smart investment decisions.” [97] Since this is a application for mobile devices, this virtual assistant can alert users whenever there are fluctuations in the market or any changes to the individual’s trading account. Nami is equipped with machine learning and is able to differentiate between languages. It is based off the Facebook Messenger platform, so it is easily accessible by users. They can access it by typing: “ @namipfm” into the Facebook messenger search box and you can being your conversation with Nami. Nami is offered in two versions: free and paid. With the free version users can use functions such as currency rates, commodity index or notifications when the price of a stock has hit a certain threshold. However, with the paid version, users are able to make orders and see if these orders are making profits or losses. With its ease of accessibility, Nami has shown that the financial investment field is heading towards a world where there will be fewer interactions with the traditional broker, but more so with a AI chat robot.

Conclusion

Personal VPA Conclusion

Limitation

Currently, personal assistants still have many limitations on what they can do, but as we’ve seen from Apple’s implementation of the apps “Shortcuts”, and the millions of dollars spent on AI recruitment by the major tech businesses, we can assume that there will be a lot of advance in the coming years for VPAs. The biggest barrier for was privacy concerns by the public, but as the technology becomes more and more useful, people tend to care less about the privacy they give up, as seen in a IDC survey, which reported that only 20 percent of Canadians had privacy conerns regarding connected home automation, which is just a bit above the 16 percent of Canadians who had concerns for security regarding connected home automation.[98]

Which company’s home automation device you use also has a large effect on your daily activities, as results that are sponsored by the specific company tend to come up higher (i.e. if you ask Google Assistant for restaurants nearby, the restaurants that are sponsored tend to come up first). And advertisements have also been a topic that is more and more discussed with regards to audio advertisements over these devices, with Alexa and Google Home taking very cautious steps in implementing ads over their home personal assistant devices.[99] This can further promote “ecosystems” in a person’s life, as on top of all the products must be compatible in order for home automation to be efficient, people may be directed towards specific products by these home personal assistants, creating a true “ecosystem” of products and services for a person, ranging from sponsored toilet paper, to movie theatres and restaurants. Therefore, we must understand the limitations of these personal assistant products and the effects they may bring into our lives and be aware of the monetization element that these products have.

Privacy Concerns

Apple’s Privacy Commitment

Apple has been a company committed to its consumers’ privacy, through several public stints with the FBI in landmark cases fought in the U.S. The result was the FBI withdrawing from the case after continued unsuccess at several courts requiring Apple to unlock the iphones through the creation of a “master key”, and spent approximately $1,000,000 for a hacker to unlock the iphone instead [100] More importantly, Apple’s core business of selling products is different from Google’s core business of producing advertisement and using big data, which although doesn’t mean that Apple doesn’t collect data, at least they don’t sell the data to other people as a main source of revenue.[101] Apple’s still does machine learning through using consumers’ data, but all requests and voice recordings sent to SIRI is under anonymous IDs with no personal identifiers. Apple states that Siri learns through the total cumulative data collected on users’ as a whole without tapping into individuals’ personal data.[102]


Data privacy concerns

There is a common concern regarding smart home devices could be collecting private data by listening to user conversations at home, since they are always listening for a command. Additionally, some products such as the Echo Show, Spot, and Lenovo Smart Display also comes with a camera which comes with the concern that everything a user does will be recorded and in the cloud. Even while virtual assistants perform their normal duty of answering user inquiries or commands, there is still a concern about what the company will do with the data collected. There will always be basic data connected in order for the virtual assistant to perform its duties, but the concern is about whether the company, unknowing to the user, takes more data than agreed upon in the privacy statements.

Companies of voice assistants have tried to address some of these concerns by explaining that the assistant will only start to register information when they hear the trigger words, “Ok Google, Alexa”, and that the device can even be turned off if there is a concern about overhearing private conversations. However, the concern of what the company does with this data still remains. Users with security concerns should refer to the private policy of the device or virtual assistant they are concerned about to develop an understanding of how much of their information if being collected and stored, and what their information will be used for. Any doubts should be questioned and resolved before proceeding on with purchasing the device. Of course, there is always a risk of hackers that can get into company systems and access all user information as well.

Some ways to address privacy or security concerns:

  • Delete past interactions, recordings, inquiries off the history of the device. Once deleted, it will be deleted off the Amazon or Google clouds forever, not just off your device.[103]
  • Not to purposely share information that should be private, meaning do not activate the voice assistant while talking about confidential information
  • Shut off virtual assistant functions when privacy is needed or when the assistant is not needed. There is also a “mute” function on these devices to make sure they are not listening.[104]
  • Enhance security of account through authentication process (for example sending a text to the user’s phone to confirm if they are the rightful owner of the account)

Smart Lock Security Concerns

A common feature in smart locks is to have the door automatically unlock when users approach the door from the outside, determined by the proximity of the smartphone connected to the lock. These locks are able to detect if the phone is indoors or outdoors so that intruders can’t just walk into someone’s home if the owner’s phone is too close proximity to the door from the inside. There are still concerns from people in the case of the lock misinterpreting the location of the phone. If someone is sitting near the front door with their phone, strangers should not be able to unlock the door from the outside since they should only be able to do that if the phone is outside the home as well, but there is always the off chance that there will be a technological malfunction in the lock.[105]

In addition, since smart locks are connected to smartphones, there is a concern of security if the smartphone is ever stolen or lost. Users are able to log onto the website and disable the mobile device connected in case of theft or loss of their phone.[106] However, there is still the time between losing and disabling the phone that could be worrisome for people. Although a security concern, this is also safer than losing a physical key which requires the lock to be replaced altogether.

With smart locks, there is always a concern of hacking, and while these smart locks are secure there are always ways around it. However, if a burglar really were to break into a house, even a secure lock would not necessarily stop them as there are always ways around it such as breaking a window [107] After all, traditional locks with keys can also easily be broken into.

Voice activated locks also possess a concern in case someone other than the user is able to bypass the lock due to technical issues. Most voice activated locks come with the option to add a pin as well for the user to say to ensure correct identity, which makes it the user’s responsibility to mitigate this risk of technology failure.

Industrial VPA Conclusion

Limitations

AR Glasses

The potential of AR glasses will heavily depend on major tech companies; possible companies could include: Apple and Microsoft. Experts claim that for further development of AR technology, adoption of the technology is much needed in order to gain momentum as the next "big thing." AR glasses must become a seamless application in order for users to consider it for daily usage. [108] Currently, AR glasses are only used for specific applications (ex. production processes for automobiles and aircrafts), but experts suggest that if it is more widespread with other parts of business processes AR technology will be able to mature. [109]Some individuals may feel that it might be embarrassing to wear bulky AR glasses around as the public may view them in a negative image. They also suggested that individuals may be drawn to using AR glasses if it were "... worn by style leaders as sleek fashion accessories."[110] (These could include famous celebrities or athletes.) In the end, experts suggest that AR glasses will become the next big innovative technology to be introduced mainstream, just like the personal computer and the Apple iPhone.

Privacy Concerns

Hospitality

The Radisson Hotel Group, an international hotel chain confirmed that it has been a victim of a data breach that affected their members in their loyalty membership plan. Information such as names, email addresses, home address, and phone numbers were taken by a unauthorized person who was able to gain access to the property management system [111]. Hotels and IoT devices gather copious amount of data that could be considered “private”. This combination could lead to potential privacy issues that harm every party that is involved. Under the EU GDPR, hotels need to specifically explain to all guest the ways they are being recorded and have their consent.[112] This is not applicable to the US yet, however there are several initiatives in the pipeline such as the CONSENT act to bring GDPR-style protections to California and the US [113]


Automotive Industry

Recently, there has been news reporting that China is tracking citizens that are driving electric cars from their headquarters located in the Jiading district. The Chinese government claims that they are obtaining this data to ensure "... improve public safety, facilitate industrial development and infrastructure planning, and to prevent fraud in subsidy programs." [114] Automakers are forced to install such systems into their automobiles, otherwise the Chinese government will now allow them to sell them in China. [115] Other countries, such as the United States and Canada, do not have such policies for automobile manufacturers. It has only recently came to the attention of Chinese citizens in an otherwise unknown operation by the Chinese government. They also claim that "...improve public safety and facilitate industrial development and infrastructure planning.”[116]With the introduction of the Social Credit System, the actions of Chinese citizens will be further tracked by the Chinese Government. Automotive manufacturers are now stuck with the task of whether such data collection goes according to their values.[117]


Finance Industry

Early this year, there was a big privacy breach for Canadian banks( Bank of Montreal and Simplii Financial. Both of these financial companies have their own security systems in order to prevent hackers from accessing their customers' private information; however, it was clear that it did not work in this situation. The real question for these companies are whether or not these they did a good enough job to prevent such massive loss of information leak in the first place. These banks claim that they"...have acted swiftly in response, launched full-scale investigations and took immediate action to enhance online security measures to protect customers.”[118] With such a large of collection of customer information, banks must enhance their systems continuously in order to prevent such privacy breaches to occur again in the future.

Authors

Dustin Li Alvin Wong Jessie Chen Randy Huynh
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada
Beedie School of Business
Simon Fraser University
Burnaby, BC, Canada

References

  1. https://www.investors.com/news/technology/artificial-intelligence-ai-spending/
  2. https://www.ibm.com/ibm/history/exhibits/specialprod1/specialprod1_7.html/
  3. https://www.itbusiness.ca/news/history-of-voice-recognition-from-audrey-to-siri/15008/
  4. https://voicebot.ai/2017/07/14/timeline-voice-assistants-short-history-voice-revolution/
  5. https://voicebot.ai/2017/07/14/timeline-voice-assistants-short-history-voice-revolution/
  6. https://www.businessinsider.com/apple-says-siri-has-500-million-users-2018-1/
  7. https://www.businessinsider.com/siri-google-assistant-voice-market-share-charts-2018-6/
  8. https://www.businessinsider.com/siri-google-assistant-voice-market-share-charts-2018-6/
  9. https://slideplayer.com/slide/8926469//
  10. https://www.expertsystem.com/natural-language-process-semantic-analysis-definition//
  11. https://www.scienceabc.com/innovation/what-is-siri-app-working-apple-eyes-free-artificial-intelligence-voice-recognition-natural-language-processing.html/
  12. https://nerdist.com/a-smart-mirror-is-what-future-technology-should-look-like/
  13. https://www.newsweek.com/google-engineer-invents-smart-mirror-inspired-arnold-schwarzenegger-423162/
  14. https://nerdist.com/a-smart-mirror-is-what-future-technology-should-look-like/
  15. https://www.latimes.com/food/dailydish/la-dd-smart-counter-top-singing-fridge-kitchen-future-20140108-story.html/
  16. https://www.cbc.ca/news/business/uber-arizona-1.4676181/
  17. https://www.theatlantic.com/technology/archive/2018/03/can-you-sue-a-robocar/556007//
  18. https://www.hospitalityupgrade.com/getmedia/fa7c556c-1aac-4924-9f14-b971a1f453f4/Angie-Whitepaper_Guest-Room-Tech-Consolidation.pdf?ext=.pdf/
  19. http://www.intel.com/content/www/us/en/internet-of-things/infographics/guide-to-iot.html?lipi=urn%3Ali%3Apage%3Ad_/
  20. http://www.gartner.com/newsroom/id/2839717?lipi=urn%3Ali%3Apage%3Ad_ agship3_pulse_ read%3BU%2FljS34xRQOjFaSF%2B8ChgA%3D%3D/
  21. https://techcrunch.com/2018/06/19/amazon-launches-an-alexa-system-for-hotels//
  22. https://www2.deloitte.com/content/dam/Deloitte/us/Documents/consumer-business/us-cb-2017-travel-hospitality- industry-outlook.pdf/
  23. https://www.hospitalityupgrade.com/getmedia/fa7c556c-1aac-4924-9f14-b971a1f453f4/Angie-Whitepaper_Guest-Room-Tech-Consolidation.pdf?ext=.pdf/
  24. http://www.scmworld.com/googlewalmart-brutal-future-retail-supply-chains//
  25. https://hbr.org/2011/12/the-future-of-shopping/
  26. https://hbr.org/2018/05/marketing-in-the-age-of-alexa/
  27. https://www.scienceabc.com/innovation/what-is-siri-app-working-apple-eyes-free-artificial-intelligence-voice-recognition-natural-language-processing.html/
  28. http://www.who.int/news-room/fact-sheets/detail/falls/
  29. https://electronics.howstuffworks.com/gadgets/high-tech-gadgets/amazon-echo.htm/
  30. https://thewirecutter.com/reviews/what-is-alexa-what-is-the-amazon-echo-and-should-you-get-one//
  31. https://www.digitaltrends.com/smart-home-reviews/amazon-echo-dot-review//
  32. https://www.cnet.com/reviews/amazon-echo-show-review//
  33. https://www.cnet.com/reviews/amazon-echo-show-review//
  34. https://www.stonetemple.com/digital-personal-assistants-study/
  35. https://www.pocket-lint.com/apps/news/google/144422-google-assistant-pretty-please-mode-encourages-kids-to-say-please/
  36. https://www.bestbuy.ca/en-ca/google-store/google-home-mini.aspx/
  37. https://mashable.com/review/google-home-max-review/#uvqE4SbVU5q1/
  38. https://www.businessinsider.com/smartwatches-expensive-stylish-beautiful-2018-7#most-over-achieving-garmin-fenix-chronos-1500-6/
  39. https://www.digitaltrends.com/smart-home-reviews/lenovo-smart-display-review/
  40. https://www.theverge.com/2018/1/25/16931580/samsung-galaxy-s9-prelaunch-mwc-2018/
  41. https://www.the-ambient.com/guides/bixby-voice-assistant-samsung-guide-336/
  42. https://www.mattressfirm.com/blog/sleep-tips/the-benefits-of-tracking-your-sleep//
  43. https://motherboard.vice.com/en_us/article/53d45b/gatebox-holographic-ai-assistant/
  44. https://venturebeat.com/2018/08/02/lines-holographic-gatebox-robot-hints-at-the-virtual-assistants-of-the-future//
  45. https://www.digitaltrends.com/home/gatebox-azuma-hikari-virtual-assistant-news//
  46. https://www.digitaltrends.com/home/gatebox-azuma-hikari-virtual-assistant-news//
  47. https://thewirecutter.com/reviews/what-is-alexa-what-is-the-amazon-echo-and-should-you-get-one/#is-your-echo-spying-on-you/
  48. https://thewirecutter.com/reviews/what-is-alexa-what-is-the-amazon-echo-and-should-you-get-one/#is-your-echo-spying-on-you/
  49. https://www.cnet.com/how-to/amazon-echo-alexa-everything-you-need-to-know//
  50. https://www.digitaltrends.com/home/google-home-hub-specs-roundup//
  51. https://economictimes.indiatimes.com/magazines/panache/samsung-launches-smart-refrigerator-with-bixby-control-touchscreen-at-rs-2-8-lakh/articleshow/65040495.cms/
  52. https://www.tomsguide.com/us/pictures-story/1558-alarm-clocks-with-alexa-google-assistant.html#s10/
  53. https://www.tomsguide.com/us/pictures-story/1558-alarm-clocks-with-alexa-google-assistant.html#s10/
  54. https://www.mysmartblinds.com/pages/my-smart-blinds-works-with-your-home/
  55. https://www.ecobee.com/switch-plus//
  56. https://www2.meethue.com/en-us/friends-of-hue/
  57. https://www.the-ambient.com/guides/bixby-voice-assistant-samsung-guide-336/
  58. https://www.ecobee.com/ecobee3//
  59. https://www.imore.com/smart-home-premium-kwikset-kevo-smart-lock-review/
  60. https://www.allhomerobotics.com/schlage-connect-vs-schlage-sense-comparison-get//
  61. https://august.com/pages/how-it-works/
  62. https://thewirecutter.com/reviews/what-is-alexa-what-is-the-amazon-echo-and-should-you-get-one//
  63. https://www.cnet.com/how-to/everything-you-need-to-know-about-google-home//
  64. https://www.ecobee.com/switch-plus/
  65. https://www.forbes.com/sites/andriacheng/2018/06/19/amazons-marriott-deal-is-way-beyond-alexa-as-your-new-hotel-butler//
  66. http://nationalpublicmedia.com/wp-content/uploads/2018/01/The-Smart-Audio-Report-from-NPR-and-Edison-Research-Fall-Winter-2017.pdf/
  67. https://hbr.org/2018/08/how-smart-speakers-are-poised-to-reinvent-the-travel-industry/
  68. https://venturebeat.com/2018/07/09/smart-speaker-adoption-expected-to-grow-6-times-by-2022-with-apple-trailing//
  69. https://hbr.org/2018/08/how-smart-speakers-are-poised-to-reinvent-the-travel-industry/
  70. https://www.forbes.com/sites/bernardmarr/2016/01/26/how-big-data-and-analytics-changing-hotels-and-the-hospitality-industry//
  71. https://hbr.org/2018/08/how-smart-speakers-are-poised-to-reinvent-the-travel-industry/
  72. /
  73. https://www.britannica.com/technology/automation/Advantages-and-disadvantages-of-automation/
  74. https://www.forbes.com/sites/bernardmarr/2016/01/26/how-big-data-and-analytics-changing-hotels-and-the-hospitality-industry/#108178c41c22/
  75. https://searchcio.techtarget.com/answer/Whats-the-difference-between-chatbots-and-virtual-assistants/
  76. https://www.forbes.com/sites/rachelarthur/2016/07/20/macys-teams-with-ibm-watson-for-ai-powered-mobile-shopping-assistant//
  77. https://hbr.org/2018/05/marketing-in-the-age-of-alexa/
  78. https://www.seattletimes.com/business/amazon/amazons-alexa-could-become-10-billion-business-by-2020-rbc-analysts-estimate//
  79. https://www.statista.com/chart/5621/users-of-virtual-digital-assistants//
  80. https://searchengineland.com/google-reveals-20-percent-queries-voice-queries-249917//
  81. https://www.slashgear.com/porsche-adopts-augmented-reality-for-quality-assurance-13447825//
  82. https://www.usatoday.com/story/tech/talkingtech/2018/05/29/porsche-now-using-augmented-reality-glasses-repair-cars/648673002//
  83. https://www.usatoday.com/story/tech/talkingtech/2018/05/29/porsche-now-using-augmented-reality-glasses-repair-cars/648673002//
  84. https://www.ibm.com/ibm/history/exhibits/specialprod1/specialprod1_7.html/
  85. https://www.autoblog.com/2018/09/19/porsche-hyundai-invest-wayray-augmented-reality//
  86. https://upskill.io/skylight/use-cases//
  87. https://www.reuters.com/article/us-boeing-venture/boeing-forms-venture-group-invests-in-two-tech-startups-idUSKBN1772BE/
  88. https://www.theverge.com/2018/2/5/16966530/intel-vaunt-smart-glasses-announced-ar-video/
  89. https://www.engadget.com/2018/04/19/intel-cancels-smart-glasses-vaunt-superlight//
  90. https://www.forbes.com/sites/huawei/2018/10/23/financial-services-taking-ai-to-the-next-level/#657385037858/
  91. https://www.forbes.com/sites/huawei/2018/10/23/financial-services-taking-ai-to-the-next-level/#657385037858/
  92. https://www.forbes.com/sites/huawei/2018/10/23/financial-services-taking-ai-to-the-next-level/#657385037858/
  93. https://www.forbes.com/sites/huawei/2018/10/23/financial-services-taking-ai-to-the-next-level/#657385037858/
  94. https://https://www.forbes.com/sites/huawei/2018/10/23/financial-services-taking-ai-to-the-next-level/#657385037858/
  95. http://www.digitaljournal.com/tech-and-science/technology/gia-is-an-alexa-rival-for-the-finance-industry/article/512889/
  96. http://www.digitaljournal.com/tech-and-science/technology/microsoft-cortana-is-not-dead-yet/article/512784/
  97. https://nami.io/press/ifx-expo-asia-hong-kong-2018-highlight-nami-assistant-was-worldwide-announced//
  98. https://business.financialpost.com/technology/personal-tech/are-smart-speakers-spying-on-you-privacy-security-fears-grow-as-devices-get-more-popular
  99. https://medium.com/spokenlayer-blog/the-state-of-advertising-on-smart-speakers-286f48b89466
  100. https://www.channelfutures.com/talkin-cloud/microsoft-argues-landmark-cloud-privacy-case-supreme-court/
  101. https://www.investopedia.com/articles/investing/020515/business-google.asp/
  102. https://www.fastcompany.com/40443055/apple-explains-how-its-making-siri-smart-without-endangering-user-privacy/
  103. https://thewirecutter.com/reviews/what-is-alexa-what-is-the-amazon-echo-and-should-you-get-one/#is-your-echo-spying-on-you/
  104. https://www.wired.com/story/amazon-echo-and-google-home-voice-data-delete//
  105. https://www.imore.com/smart-home-premium-kwikset-kevo-smart-lock-review/
  106. https://www.imore.com/smart-home-premium-kwikset-kevo-smart-lock-review/
  107. https://www.howtogeek.com/322576/are-smart-locks-secure//
  108. https://www.sunstar.com.ph/article/1759873/
  109. https://www.sunstar.com.ph/article/1759873/
  110. https://www.sunstar.com.ph/article/1759873/
  111. https://www.komando.com/happening-now/503406/international-hotel-chain-suffers-a-massive-data-breach-has-your-data-been-compromised?utm_medium=nl&utm_source=alerts&utm_content=2018-11-02-article-title/
  112. https://eugdpr.org/the-regulation//
  113. https://www.insideprivacy.com/united-states/congress/senate-democrats-propose-consent-act//
  114. https://www.foxnews.com/auto/chinese-governments-monitoring-of-electric-cars-raises-surveillance-fears/
  115. https://www.foxnews.com/auto/chinese-governments-monitoring-of-electric-cars-raises-surveillance-fears/
  116. https://www.theverge.com/2018/11/30/18120148/car-companies-china-location-data-electric-cars-mercedes-tesla/
  117. https://www.foxnews.com/auto/chinese-governments-monitoring-of-electric-cars-raises-surveillance-fears/
  118. https://www.csoonline.com/article/3276275/data-breach/2-canadian-banks-hacked-90000-customers-data-stolen.html/
Personal tools