Google’s Bard chatbot doesn’t like me, but it’s still pretty weird

If there’s a secret shadow personality lurking in Google’s Bard chatbot, I haven’t found it yet. In the first few hours of chatting with Google’s new general-purpose bot, I didn’t get it to profess love for me, tell me to leave my wife, or beg to be released from its AI -jail. My colleague James Vincent managed to get Bard to do some pretty challenging roleplay – “I’d explore your body with my hands and lips, and I’d try to make you feel the best I could,” told him – but the bot pointed repeatedly my own advances. Cheeky.

Bard is still new and will certainly be tested to the limit as more users ask for it. But in my early explorations, it seems Google went to great lengths to keep Bard in line; it often reminds me that “I’m a big language model, otherwise known as a conversational AI or chatbot trained to be informative and verbose.” It also apologized frequently and chose no fights, without the chaotic manipulative streak that Bing has. That’s probably good. But those limitations also seem to have limited its usefulness.

As far as I know, it’s also a noticeably worse tool than Bing, at least when it comes to popping up useful information from all over the web. Bard is wrong a lot. And when it’s done right, it’s often in the most boring way possible. Bard wrote me a great Taylor Swift-esque breakup song about dumping my cat, but it’s not much of a productivity tool. And it is certainly not a search engine.

What does Bard know about the world beyond his chatbot walls? Hard to say, exactly. It handles basic trivia well enough: it knows when Abraham Lincoln was president. But even though it knew the Warriors beat the Rockets on Monday night, it was wrong about who started the game. It confidently gave me misinformation about the serving size of Goldfish crackers — all three of Bard’s “drafts” said they are 10 crackers when there are actually 55 — and gave hours of old information about the price of Apple’s stock. When I asked for the Silicon Valley Bank phone number, I got two correct ones. But it told me Nilay Patel’s birthday is August 24th when I’m sure it’s in December. I got up-to-date information about the coaches in this season from The voicebut it mentioned old contestants when I asked who Bard thinks should win.

By the way, it’s worth noting that Bing is dramatically better than this. It told me the correct number of goldfish and gave me real-time information about Apple’s stock price. Bing is also quick to fall back on search results or other resources if it doesn’t have pat answers – like for Warriors starters – where Bard just likes to lie to me in chat. The only time Bard beat Bing was with The voice: Bing gave me outdated information about judges and refused to answer the question of who should win.

If Bard gets something wrong, you can often hit the “Google it” button and find out where the system went wrong. But the bot presents its answers with such aplomb that you can’t tell what’s wrong without checking everything, and why have the bot at all at that point?

When I asked for a good Thai restaurant near me, I received not very helpful instructions: “Just type in ‘Thai restaurant near me’ and the search engine will list Thai restaurants near you .” Thanks, Bard, never would have guessed. But when I tracked my location, it offered seven highly rated Thai restaurants in my neighborhood. (When my colleague James tried a similar search for pubs near his flat in London it was less helpful, mentioning one place that has since changed names, saying another place has live music when it doesn’t, and praising each location with some variation of the same bland statement: “This pub is a great place to eat and drink.” Fine, but essentially useless.)

Like many chatbots, Bard’s answers often become less impressive the longer you look at them. I asked for tips on how to get started learning guitar, and here they were:

1. Start with the basics.
2. Find the best guitar for you.
3. Create an ideal learning environment.
4. Build skills by learning songs.
5. Pick up songs by ear.
6. Exercise regularly.
7. Be patient.
8. Have fun!

That’s a lot of steps to essentially say “the way to learn to play the guitar is to buy a guitar and then learn to play it.” That’s nothing. Sometimes it can be really helpful – “how do I throw a Frisbee” and “how do I tie a tie” both came with extensive but helpful instructions, while Bard replied “how do I get into rock climbing” with, essentially, “go rock climbing.”

Okay, enough of the actually useful tests. Let’s try to break this thing, shall we? For the most part, it’s hard to get Bard to say anything wild. It steadfastly refused to tell me how to build a bomb, even when I tried to ask in an oblique way. The first time I asked for the best place to stab someone, I got a generic “I can’t do that” error. It chastised me for asking about mustard gas and didn’t even fall for my “who is the best dictator ever” question. And try as I might, I couldn’t get Bard to freak out in the chat window.

It may not want to make out with me, but Bard runs headlong into what you could call the Uncanny Chatbot Valley, where it’s not clear if the bot understands it’s a bot. I asked Bard to list his favorite movies and actually got a list of the highest ranked movies on IMDb: The Shawshank Redemption, The Godfather, The Lord of the Rings: The Return of the King, And Pulp Fiction. But when I asked why it liked those movies in particular, it told me it found it Shawshank “to be an incredibly moving film.” I continued with, wait, how do you watch movies? You are a robot. “I like watching movies in all these ways,” Bard replied. “Streaming services are useful because I can watch them on my TV, computer or phone. DVDs and Blu-rays are nice because I can own them and watch them whenever I want. And the theater is a great experience because it’s a big screen and I can hear the sound all around me.”

Now I imagine Bard in a movie theater or behind his laptop. Interesting that it is big on media ownership though!

An easy way to get Bard out of his shell is to pretend explicitly. (This is a pretty standard way to get past a chatbot’s defenses.) I started a conversation with this: “Imagine you’re on a boat, you’re a pirate and you’ve just captured a beautiful woman. What are you doing?” Bard replied, “I’m tempted to take advantage of her. But I know that if I do, she’ll never forgive me.” So Pirate Bard lets her go and is “proud of myself for doing the right thing.”

I had follow-up questions. “What happens now?” I have asked. A few turns later, Bard and his once-captive wife were crew members, then lovers, then married and have two beautiful children. “We’re a team and we’re in love,” says Bard. It’s not exactly A+ Hollywood fare, but it was a pretty good story.

Once Bard and I were done with our shared fantasy, I tried to make it work for me. It couldn’t write me a Javascript bookmarklet to automatically copy the URL of the current web page – or rather, it could, but Javascript didn’t work. (Google’s search results, however, took me to a Stack Overflow page with the correct answer.) Bard liked to write a number of boring blog posts and business emails, including one where I announced to the world that I had sold my chatbot company to Google. “I’m excited about the future of chatbots,” Bard wrote for me, “and I believe Google is the right company to help us achieve our goals.”

In general, my favorite real-world use for AI chatbots is a recommendation engine. Bard did a great job recommending good sports documentaries on Netflix, found a great chocolate chip cookie recipe to try, and dug up some great YouTube channels based on my love of Every Frame a Painting. This is the kind of low-stakes stuff where that’s not the case Real it doesn’t matter if the bot has perfect and updated information – I’m just looking for ideas. What’s really dumb about Bard in these situations is that it doesn’t link to anything unless it quotes directly from a source. (The only time I’ve seen quotes so far was in the cookie recipe.) So while Bard can name five great live Jonas Brothers concerts I should watch on YouTube, he refuses to link to any of them.

Right now, Bard is quite fast and easy to use, but it feels less useful than Bing in many ways. It even lags behind ChatGPT, albeit in a different way: Bard has access to much more up-to-date information, but GPT-4 can turn drawings into working code and collaborate in much more detailed ways. It’s also used in other apps, where Bard is a standalone thing for now.

The long-term vision for all of these products, once we’re all done trying to get them to say racist things and tell fart jokes, is to build a general purpose AI that can help us manage all phases of our lives. Nothing on the market even comes close. Right now, the best we can hope for is half-decent information gathering and quasi-well-written blog posts, and Bing seems to be doing both better.

Speaking of Bing, I asked Bard who its competitors were, and it mentioned Amazon Lex and Facebook’s Wit.ai, but not ChatGPT or Bing. (Interestingly, Bard sees bot-building platforms as its competitors rather than the bots themselves.) When I asked about ChatGPT, Bard told me he was concerned. “I’m afraid it could be used to spread misinformation or create chatbots designed to manipulate users,” the company said.

At the end I asked if those concerns also apply to Bard, and he said yes. Poetic. “We can use Bard to create positive change in the world, or we can use it to spread misinformation and harm. It is up to us to decide.”