A family of conversational neural language models developed by Google.

Introduction

In this blog, we will see what is LaMDA and what is the purpose of developing it. And we will also discuss the claims made by some people about LaMDA.

What is LaMDA ?

LaMDA, short for Language Model for Dialogue Applications was developed by Google to have more fluent and accurate, natural conversations with people. LaMDA can converse with a human on any topic, thanks to the large dataset used to train LaMDA and it is indeed a huge milestone achieved by Google. Google released its first keynote about LaMDA on May 18, 2021. LaMDA understands the concepts and meaning of words with context to sentences from its training data, which helps it carry on a conversation in a realistic way.

Despite being able to make fluent natural conversations, Google has also taken care of the groundedness, offensiveness, informativeness and specificity of the answers given by LaMDA. First LaMDA will produce a certain number of answers during a conversation, and then each answer is ranked by the metrics defined by Google’s AI research team and then the best of it is chosen as the final output. By these kinds of ranking schemes, LaMDA is able to produce engaging, reliable and inoffensive answers. Even though LaMDA is such a big model with 137 Billion parameters, it still can’t answer some questions and gives unrelated answers. It is still lower than human performance in terms of informativeness, sensibleness, groundedness etc. But its performance is much better than other existing models. Here is a comparison of the new LaMDA, its pre-trained version vs a human:

But after seeing the fantastic results by LaMDA, one might wonder about how LaMDA is able to make this level of clear and detailed conversation? The answer is “Transformers”.

LaMDA is based on the Transformers architecture. Yeah, the word ‘Transformers’ does not denote the electrical device or the Transformers movie. It is an architecture proposed by Google to achieve high-level parallelisation, accuracy and to reduce computation cost for NLP-related tasks. Google announced to the world about the Transformers architecture in the 2017 paper, Attention Is All You Need. The transformers architecture is simple, accurate and parallelisable and works well as a replacement to the LSTM — Long Short Term Memory architecture, which was prominent for NLP tasks until 2017.

The drawback in using LSTM is as the sentence length gets longer, it is hard for it to make relation with previously seen words, for example: the pronoun “he” can refer to the subject at the beginning of a sentence. Parallelization is not possible in LSTM as it is a sequence model ( operations occur one after other ), thus increasing the computation cost. Transformers architecture overcomes all of these shortcomings of LSTM by cleverly using attention mechanism with CNNs and achieves a high level of parallelization as well as good accuracy than LSTM. More specifically, it uses self-attention.

Generally, transformers consist of a bunch of decoders and encoders, all encoders share the same architecture and all decoders share the same architecture. Inside the encoder, there are two there two layers one is Feed Forward Neural Network and a Self-attention layer and the Decoder has one more additional layer called encoder-decoder attention. The self-attention path for each word in sequence is dependent but the FNN layer is not dependent on each other, so there we can parallelize the process.

After that, our model creates an attention-matrix, which contains the importance of each word in a sentence with respect to a specific word and it is later used to predict the upcoming word in a sentence or translation from one language to another language etc. In this blog, we will focus only on giving you a feel for why transformer is better than other models and if you are interested in reading about the detailed working of transformers, check here. Apart from self-attention, transformers also use multi-head attention and positional encoding to achieve even better results. Thus by combining all of these techniques and using their advantages, transformers indeed transformed the NLP field. It’s no wonder to see todayalmost all big AI companies like DeepMind, OpenAI using transformers in their NLP models like GPT-2, GPT-3, Gopher etc..

LaMDA sentient crisis :

After seeing the unbelievable performance by LaMDA, it’s no surprise to see people calling it a ‘sentient’. AI is a field where such breakthroughs have become so common over the past 10 years. Yes, there are some claims like LaMDA is sentient, and it also has feelings like a human. But as of now, there is no evidence for these claims other than its intelligent answers which it learned through the huge dataset.

According to many AI researchers and data scientists, “ LaMDA is a model which picks the most probable answer out of available words and gives us an answer. Its performance depends on the dataset it is trained on and the size of the model. While during a conversation or translation it is only choosing the most probable answer by its experience and we can’t call it a sentient hence“. But however, many tech companies have guidelines to follow in developing safer and explainable AI and governments also planning to bring AI guidelines. We can safely say these AI will revolutionise our future and truly make lives easier.

Stay tuned for more, and do follow us on Instagram and YouTube.