Blog

1. Does ChatGPT know what it is saying?
2. Why "Regin AI"?

Does ChatGPT know what it is saying?

As a high level description, ChatGPT learns the linguistc structure of a language by using lexical analysis to parse text into tokens and then learning the frequency of how those tokens are organised. It doesn't learn facts or logic, it only learns the way the sequence of tokens in the text that it has learnt from.
ChatGPT uses a uses a Generative adversarial network (GAN) to generate responses by ordering tokens in a similar way as it has learnt is the most likely. It effectively generates a response by merging the most likely token sequences. For example, this becomes evident when we ask ChatGPT about a topic that has divided opinions in a manner that makes it hard for ChatGPT to realise that it is the same question.
First, to introduce the topic, below is a screenshot of ChatGPT's answer when asked in English what Völuspá is.


The answer is a good summary.

However, asking in Icelandic and then in English whether the Poetic Edda is written in Icelandic gives polar opposite answers. Below is a translation from Google Translate of its Icelandic response:

"Yes, Eru Eddukäði was written in Icelandic. It is the oldest manuscript collection of Eddic poems owned by the Árni Magnússon Foundation in Iceland, which is also known as the Codex Regius. This manuscript was composed around 1270 and is among the best known and most important surviving manuscripts of the Eddus."

When answering the question in English, ChatGPT gives the opposite answer and says that it was written in Old Norse in Iceland.



As seen in the above screenshot, the question is posed in Icelandic and then immediately in English, yet ChatGPT is unaware that it has given factually opposing replies; it doesn't understand the question or the answers; they have no meaning to ChatGPT.
Furthermore, since the questions and responses are in different languages, they don't have a similar token sequence; they only share a few tokens, such as the tokens for "Edda" and "Icelandic". So Chat GPT cannot see any connection between the two.
We can fix this discrepancy by asking it to translate the English question to Icelandic or vice versa. See below:



Albeit the Icelandic name of the Poetic Edda is "Eddukvæði", not "Póetíska Eddan". So the original name is lost in ChatGPT's translation. "Póetíska" is English slang and not an Icelandic word. Eddukvæði means Edda's poems. Edda here means grandmother.

Why "Regin AI"?

The latest releases in advanced AI are large language models (LLMs), neural networks with typically billions or more of weights. They are trained on petabytes or more of unlabelled text using self-supervised learning.
LLMs effectively learn our most likely responses in a specific context and use a Generative pre-trained transformer (GPT) to generate or predict the most likely outcome based on unsupervised learning of a similar context.
The latest example of these models is ChatGPT: a chatbot developed by OpenAI and launched in November 2022.
There are limits to these models which become apparent when using them. They are only mimicking; they are not reasoning the same as we do. We need to combine these models with other advanced AI to use them for reasoning, logic and factual summaries. Therefore, we focus on working with others to optimise the use of this powerful technology.

These LLMs have become very popular and are fueling a whole new industry and the release of hundreds of new AI apps. It is strikingly reminiscent of creation mythologies; we are effectively playing gods by creating AI in our image.

One such mythology is in the Poetic Edda; the first poem, Völuspá, describes the world's creation. In Völuspá, the gods work together to create and organise the world by deliberating how to resolve issues, such as how to tell the time and by action, such as when giving life to the first humans. Similarly, we should work together to address the current shortcomings and make the most of this technology. In Völuspá, the gods are sometimes called "regin", hence the name Regin AI to represent that we are looking for partners who share our vision to make this technology accessible to us all.

Below are a couple of stanzas from Völuspá; the former is an example of where the gods are called "regin", and the latter is where the gods create humans from driftwood.