Posey's Tips & Tricks

A Chatbot Patent Is Microsoft's Most Disturbing Creation

Using standard AI training practices, Microsoft has developed a technology that would -- theoretically -- allow people to "chat" with deceased relatives.

• By Brien Posey
• 02/05/2021

As someone who has been writing about technology since the launch of Windows 95, I have seen a lot of tech products come and go. Some of those products have been awe-inspiring, while others left me scratching my head.

If you ever want to see the strangest products that the technology world has to offer, just go to the Consumer Electronics Show (CES) in Las Vegas sometime. Besides run-of-the-mill PC launches, CES is also known for showcasing odd technologies that sometimes defy logic. Some of the crazier technologies unveiled at the show in past years include an Alexa-powered toilet and a pair of shoes with an integrated vacuum cleaner. You just can't make this stuff up.

Recently, however, it was revealed that Microsoft had patented a technology that is every bit as "out there" as anything that I have ever seen at CES: a chatbot that would let you converse with the dead -- sort of.

Before I talk about the technology involved, let me just say upfront that Microsoft has said that it has no plans to develop a product based on the patent. Even so, the idea of using a chatbot to converse with the dead is such a ludicrous and creepy idea that I just had to write about it.

Although the person who first told me about Microsoft's new patent described it to me as a "digital Ouija board," the technology does not actually allow you to converse with the dead. What it does do is train an AI bot to take on the persona of someone who has passed on.

Like any other AI-based algorithm, Microsoft's patented chatbot requires training in order to work. It acquires this training by scouring the Internet in an effort to collect all of the information it can about the deceased person. It pulls content from social media accounts, and finds photos and videos of the person. Although not specifically mentioned, the technology likely also pulls data from public records.

All of this information is pulled together to create a chatbot that acts like the person who has passed on. For instance, the chatbot might emulate the person's style of speech or writing, and try to mimic their sentence structure. The training data would also allow the chatbot to learn about the person's interests, career, friends, family and so on. The aim of all this training is to form a convincing analog of the person who has passed on.

While most of the media sources that have covered Microsoft's patent have essentially done so from the standpoint of calling the technology creepy, my first reaction was to wonder if the technology could actually work.

If you think about it, there are two distinct steps in building a chatbot that mimics someone who has died. The first is to compile as much information as possible about the person. I have absolutely no doubt that this part of the process would work flawlessly; technology companies have mastered the art of spying on us by collecting vast quantities of data tied to practically every aspect of our lives. In other words, scraping the Internet for data is nothing new.

The second step is to pull all of the collected data together in a meaningful way that would allow the AI to be properly trained. I believe that this too could work. I can't help but be reminded of an unrelated technology demonstration that I saw a couple of years ago. I can't remember where I saw it or who created it, but someone created an AI that was designed to mimic literary styles. As part of the demo, I was asked to submit samples of articles I had written. The AI then rewrote a stock paragraph in a way that was meant to make it appear as though I had written it. The end result was really convincing. I had trouble distinguishing between the computer-generated text and text that I had actually authored myself.

All of this is to say that with a sufficient amount of data, it probably is possible to build a chatbot that does a convincing job of mimicking a person (living or dead). No matter how good such a chatbot might be, however, it will never be perfect. There will always be details that the chatbot does not know and cannot find out.

Let me give you a really personal example. My sister was the artistic type, and quite a few years back she created a painting for me. Suppose that someone pulled together every bit of digital data that exists about my late sister and used it to create an AI chatbot. My sister was an avid social media user and there are numerous recordings of her at various family events, so I am sure that with enough work, it would be possible to create a convincing chatbot -- or maybe even a "talking head" bot that looked, sounded and acted like my sister.

To the best of my memory, however, I don't think that there was ever any mention on video or on social media of the painting that is displayed in my office. That means that if I asked the chatbot something like, "Hey, do you remember that painting that you made for me?" the chatbot might be able to come back with something like, "Yes, it took me forever to paint that." However, the chatbot would have no way of knowing what the painting looks like, when it was painted, how big it is, what type of paint was used or the reason why the painting was created in the first place.

My point is that regardless of the available technology, no AI would ever be able to perfectly mimic a deceased relative. There are just too many knowledge gaps. At best, the AI would probably seem like a cheap reproduction. Beyond that, the technology strikes me as being somewhat dangerous. If it is possible to scrape Internet data and use it to create a convincing analog of someone who has passed on, just imagine how the technology might be used to imitate someone who is still living. Imagine all of the nefarious things that someone could do if they were able to create a convincing digital fake of any person that they choose.