Dr. Alok Aggarwal
OpenAI’s Transformer, GPT-3 with 175 billion parameters, was trained on approximately 500 billion tokens (i.e., approximately 365 billion words in English). Keeping this in view, we posed the following question: “Is there anything distinctive about the roof of the Massachusetts Institute of Technology?” To this, GPT-3’s answer was surprising — “there is nothing distinctive about the roof of the Massachusetts Institute of Technology building. It is made of grass.” When we asked the same question again, GPT-3 replied, “the roof of the Massachusetts Institute of Technology is made of red clay tiles.”
The figure given below shows the dome that is distinctive of MIT’s roof, but neither grass nor red clay tiles are anywhere to be seen.
Indeed, we would have been satisfied with a typical human response like, “I don’t know” or “I have never seen this roof even in pictures,” but the wrong and yet confident response from GPT-3 was shocking.
If humans provided answers like those provided above by ChatGPT, then such humans would be referred to as hallucinating. Using this analogy, AI professionals have called such responses from AI systems as “Machine Hallucinations”, of which another example is given below.
Falcon-40B is an open-source Generative Pre-trained Transformer (GPT) with 40 billion parameters. Unfortunately, like most other GPTs of similar ilk, it occasionally provides the right answer the first time and a wrong one the second time (or vice versa). For example, we asked the following question:
Unfortunately, all DLNs including GPTs are unexplainable and uninterpretable. Hence, they become highly untrustworthy when they provide inconsistent or wrong answers. This characteristic hinders their extensive use especially because it forces humans to spend substantial time to check the veracity of their answers or be duped into accepting wrong ones. Unsurprisingly, researchers are working hard to develop ways to reduce brittleness, hallucinations, and the inconsistent behavior of DLNs; and these are discussed further in the upcoming book titled, “The Fourth Industrial Revolution and 100 Years of AI (1950-2050)”. One approach is to create an ensemble of two or more different DLNs (for example two 2 GPTs) whereas another is to combine a Transformer and an Internet search engine that can provide a reference for GPT’s answer; it is unclear whether such rudimentary approaches would work.
The book titled “The Fourth Industrial Revolution and 100 Years of AI (1950-2050) will be published in September 2023. For details, see www.scryai.com/book
Blog Written by
CEO, Chief Data Scientist at Scry AI
Author of the book The Fourth Industrial Revolution
and 100 Years of AI (1950-2050)
Alok Aggarwal is just finishing the book that will be available on Amazon, Barnes and Noble, and other bookstores in September 2023.
Although the field of AI was created over seven decades ago, it remains fantastical to many. In recent years, scores of commentators...
Just like the previous three industrial revolutions, the fourth revolution, which started in 2011, is expanding at a ferocious pace.
The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information.
In the late 1890s, Thomas Edison had been working on developing a nickel-iron battery when his friend, Walter Mallory, visited his laboratory...
In May 1997, there was a six-game rematch between IBM’s supercomputer, Deep Blue, and the reigning world chess champion, Garry Kasparov, which Deep Blue won 3.5 to 2.5.
Recently, researchers and pundits have referred to data as the “new oil” or the “new electricity”. As discussed below, although this analogy...
In his 2017 talk at Stanford University, Andrew Ng referred to Artificial Intelligence as the “new electricity” and added “Just as electricity transformed...
The human brain is around three pounds in weight, has 86 billion neurons and 1,000 trillion synapses, stores 2.5 million Gigabytes of data, uses about 15-20 Calories...