I've been playing with ChatGPT recently, primarily as an alternate way to search the web. Tonight, for example, I recalled that there's a Sherlock Holmes story in which Sherlock remarks to Dr. Watson, "These are very deep waters." I couldn't remember the story containing that quote, so I decided to ask ChatGPT with the following prompt:

One of the Sherlock Holmes stories by Sir Arthur Conan Doyle has a line in it where Sherlock says something to the effect of "these are very deep waters." What is the quote and from what story does it come?

ChatGPT responded that the story that includes this quote is The Adventure of the Engineer's Thumb. Not remembering this particular story, I asked to have the story summarized. The summary jogged my memory, but I did a quick search to double check the answer's accuracy.

Lo and behold, ChatGPT was incorrect! This quote actually appears in The Adventure of the Speckled Band (a terrific story, by the way). I found the story text online, and sure it enough it includes the aforementioned quote. A search of the text for The Engineer's Thumb does not contain this quote.

It's interesting to note that these stories are the 10th and 11th stories, respectively, to be written by Arthur Conan Doyle. I'm guessing that ChatGPT made an "off by one" mistake here, which is certainly easy to do.

I can imagine a future where search via AI allows us to side-step the SEO noise that Google pumps into the ether, but it looks like there is still a ways to go. At the rate things are progressing, however, this future may not be too far off.