NLP Interview Questions and Answers for 2025

The task of image captioning is a complex machine learning task which requires a labelled data containing images and their respective captions. Assuming we have the dataset ready, we can use the above architecture to train such a model. In this architecture we have 3 main components:

1. Convolutional Neural Network Layers: This component receives the image to be captioned as an input and uses multiple CNN layers to generate features for this image. As we know an image is composed of pixels. And the color of each pixel is determined by combination of red, green, and blue color intensities. Thus, the input image can be converted into a 3-D matrix filled with integers representing these color intensities. The CNN Layers generate features by convoluting filters of different sizes in each layer. Once generated, these features are passed on to the next set of layers.
2. Fully Connected Neural Network Layers: This component receives the features from the CNN layers and flattens them into a single vector. It then passes these features through multiple fully connected hidden layers to extract the dependencies within these features.
3. Recurrent Neural Network Layers: This component receives the encoded image from the last component and passes it as an input to the RNN layers. The RNN layers also receive the labelled caption as a sequence of tokens. These tokens are converted into their respective vector representations by using pre-trained embeddings such as GloVe Embeddings.

Machine Translation can be performed using the many-to-many RNN architecture as shown in the image above. This architecture is popularly known as “Encoder Decoder” architecture. As the name suggests, the model is comprised of two components:

• Encoder Layer: This layer receives the text to be encoded as a sequence of tokens. These tokens are converted into their respective vector representations by using pre-trained embeddings such as GloVe Embeddings. And after the entire sentence is passed through the Encoder Layer, we receive the encoding of the entire sentence as a single vector. This vector is then fed to the Decoder Layer.
• Decoder Layer: This layer takes the encoding and passes it though to generate the most probable translated token at each step. In other words, it is decoding the sentence one step at a time. The decoder stops the token generation as soon as it encounters the end-of-sentence (EOS) token.

Vanishing Gradient is very common problem that occurs in case of deep neural networks. As we add more layers containing certain activation function like Sigmoid Activation, during back-propagation we find the gradient of these loss functions approaching zero. And since every neural network requires non-zero gradient of the loss function to train its parameter, the performance of our model becomes stagnant.  

In the above figure, the solid blue line is the sigmoid function, and the dotted red line is for the derivative of this functions. As it is relevant from the graph, the gradient of this cost function is non-zero in very small range of inputs. Now that the inputs become too large or too small, the gradient quickly decays to zero.

There are multiple solutions to this problem. A popular solution which is used in case of RNN is called Batch Normalization. We have discussed it in detail in a separate question. Simply put, it ensures that the inputs to the activation function are always in the desired range between which the gradient is non-zero. As demonstrated in the above figure.  

The Exploding Gradient problem is the inverse of the Vanishing Gradient. As we know, a neural network trains itself by updating its parameters. It uses the error gradient generated in each epoch. Now, while training if large error gradients accumulate, the magnitude of the model parameters become too large and thus it makes the model predictions unstable.

A common technique to deal with exploding gradient is known as “Gradient Clipping”. In this technique, we intentionally limit the values of the gradient between an upper and a lower bound. This prevents the gradients from exploding.

Masked Language Modeling is a machine learning technique in which the Neural Network tries to find the output using unarranged or corrupted inputs. In case of NLP, we can mask the input words from a sentence and ask the machine learning model to correctly predict the masked word. BERT is a popular transformer based neural network which was trained using this technique.

This technique is also used in the “Decoder” branch of a Transformer network. While training a transformer, data is not fed in a sequential manner, rather the entire sentence is fed in one go. Thus, it becomes important to mask the future tokens so that the decoder doesn’t know which words are coming in the later part of the sentence.

Auto-Encoders are a type of feed-forward neural networks in which the input and the output is the same. The idea of this architecture is to ask the model to recreate its inputs as accurately as possible. It is used in NLP for learning Word Embeddings. We start off by converting each sentence into its BOW representation containing the binary values.

We then feed this vector to the underlying network and ask it to recreate the same BOW representation of the sentence. In doing so, in the intermediate “code” layer, the network starts leaning the complex word representations which can be used as word embeddings.

Vanilla RNN models have a major drawback. They use a sequential nature of processing for solving the problems. But there are languages in which the word order doesn’t strictly matter like Polish and Hungarian. And secondly, when the length of the input sentence becomes long, it becomes very difficult for the hidden state vector to capture the entire context of the sentence. In general, as the length of output sentences becomes longer, the performance of our model on metrics like ‘BLEU Score’ decreases.

Attention based model overcomes the above problem. And they work similar to how a human would perform the task of language translation. When translating a sentence, we never memorize an entire sentence. We rather break the sentence into small parts and each output word that we generate is based on the respective part that we pay “Attention” to. In Attention Models, each generated token is not just a function of the final hidden state of the Encoder Network, but a function of all the intermediate hidden states. For each output token, the decoder produces a distinct vector known as a “Context Vector”, denoted by ‘Ci’ in the above diagram. This vector assigns a unique attention weight to each word in the input sentence. If you visualize the amount of “Attention” the network paid to each token in the input sentence the output would look like below. In this heatmap, a lighter color indicates a higher attention being paid for the respective word.

Training deep learning models is not an easy task. It requires huge computation resources and tons of labelled data to perform well. For a majority of problems, we cannot get access to either of these two and this is where Transfer Learning comes to our rescue. In this technique, we take a model which was developed for solving another task and use it as a starting point model for our task. 

It is widely used in Computer Vision and Natural Language Processing. Let us take an example of how we can use Transfer Learning to extract Named Entities from medical reports of customers.  

• Select a pre-trained model which was created to extract named entities from some other datasets. We can use Spacy’s pre-trained models as well.  
• Finetune the previous model using a small dataset which contains labelled Named Entities which are to be extracted from our medical reports. 
• Evaluate the model to check if it’s performing as per the expectations. 

In NLP, we need to convert text data into a vector of numerical feature. Traditional BOW and TF-IDF techniques have a few drawbacks:

• Both of these techniques result in high dimensional datasets. Which makes training the model very difficult.
• They both lose all the semantic and syntactic information stored in the sentence.

To overcome these drawbacks, we use Word Embeddings. These are a low dimensional learned vector representations of words in which words with similar meanings are close to one another. These representations are learned based on the usage of each word in multiple contexts.

For example, the vectors of terms King and Man would be very close to one another and similarly the vectors of terms Woman and Queen would be very close to one another. This makes the below vector addition possible.

Man−Woman+Queen=King

We can interpret the above expression as, if we remove properties of a woman from a man and add the properties of Queen, we get the properties of the King.

Both Skip-Gram and C-Bow are neural network architectures which are used to learn the word embeddings. In the Skip-Gram architecture the training dataset is created such that the feature is the current word, and the target consists of the words around the context of the current word.

While training, we ask the machine learning model to predict the context of the current word. After sufficient iteration, we get the matrix of shape (m * n) where ‘m’ is the total number of words in the vocabulary and ‘n’ is the dimensionality of each vector.

For Continuous Bag-of-Words or C-BOW model, the training dataset is created such that the features are the context around a word and the target is the respective word. While training, we ask the model to predict the word while looking at the context. Similar to filling the blank between the context words.

Negative Sampling technique is used to train machine learning models that have a high amount of negative datapoints as compared to the positive datapoints. In NLP, we use this technique to train Word Embeddings. To understand this technique, consider the sentence “I want to have a glass of orange juice with my meal”.

Now the way Negative Sampling works is that we first pick a context word at random from the corpus let us say “Orange” and then pick a word from the neighborhood of Orange. In this first row of the table shown below, we have picked “Juice” as the word. This pair of Orange and Juice now becomes a positive sample because they occur together in the sentence. And then we randomly pick words from our vocabulary like King, Book, the etc. and each of these pairs are now our Negative Samples.

We do this several times with multiple words to create a training dataset in which we have Context and the Target Word as the features and a binary - 0/1 target column. Later, we train a machine learning model which tries to predict whether a pair of context and target words are valid or not.

Perplexity is a very useful metric which is used for evaluating models in NLP. It is often defined as the “Normalized Inverse Probability of the Test Set”. And is given by the formula:

As we know, a good language model is one that assigns a high probability to sentences that are real and low probabilities to wrong, inaccurate sentences. When we evaluate any model, we feed it the datapoints from the test set. Now assuming that our test dataset contains sentences that are accurate, if our language model is in fact good, it should assign a high probability to the entire test set. Or in other words, the model should be less “Perplexed” by that dataset and hence we use the reciprocal of the probability in the formula.

The nth root is used to normalize the calculation for sentences with varying lengths. This normalization is necessary as a longer sentence has more words and thus it carries more uncertainty. All thing equal, as longer sentence will thus have a lower probability as compared to a shorter sentence and taking the nth root makes the equation independent of the size of the sentence.

BLEU or Bilingual Evaluation Understudy is a metric that is popularly used in machine translation tasks. It is used to evaluate a machine-generated sentence against a reference sentence. To understand the inner working of this score, consider the below two sentences.

• Reference Sentence: The cat is on the mat
• Machine Translation Output: The the the the the the the

If we calculate precision of the above translation, we will check if each word of the translation appears in the Reference text or not. In our example, we only have one word “the”, which appears 7 times in the reference sentence. Thus, the precision becomes 7/7=1.

This precision is misleading as the MT output is completely useless. We can instead use a modified precision score in which we give each word credit equal to the maximum number of times it appears in the reference sentence. So now the word “the” will get a credit of only 2 because it appears only twice in the reference sentence. Thus, score becomes 2/7=0.29. In this example we had only considered unigrams, but we can calculate this score on any n-grams that we want. A generalized formula for it would look like:

Where “clipped_count” is the maximum number of times each n-gram appears in the reference sentence and “count” is the total number of n-grams in the sentence.

The BLEU score is basically an average of the above modified precisions over multiple n-grams and is given by the formula: Here, BP is called “Brevity Penalty” and it’s a penalty which is assigned in case the machine translation output is shorter than the reference sentence.

In NLP, tasks like image captioning and machine translation generate a sequence of tokens. When generating this sequence of tokens, we don’t want to output any random sentence. Rather, we want to output the best possible sentence. Beam Search is the one of the most widely used algorithm to perform this task. This algorithm is an extension of the popular Greedy Algorithm, which greedily chooses the next word such that the overall probability of the sentence is maximized.

This approach often leads to sub-optimal output translations and thus it’s avoided. Beam Search on the other hand expands all the possible outputs at each step and keeps only the ‘k’ most likely. ‘k’ here is an hyperparameter which can be tuned by the programmer.

Text Similarity is used to find closeness of two chunks of text either on the basis of its meaning or the surface. For computation of the similarity, we first need to convert these texts into their respective embedding vectors.

This conversion can be done by using pre-trained word embeddings, such as GloVe vector embeddings. Once these embedding are ready, we can use any one of the following methods for computing the distance.

• Cosine Similarity

This metric measures the angle between the two vectors. It is calculated as:

Cosine of 0∘0° is 1 and is less than 1 for all other angles.

• Euclidean Distance

This metric gives the straight-line distance between the two vectors. And is given by the formula

• Jaccard Distance

Jaccard Distance does not require word embeddings. It computes the distance between the documents based on the words. It is calculated as the ratio of count of words which are common to both the documents to the count of all the unique words from both the documents. It is formulated below.

When training a neural network, one the biggest challenges that we face is of overfitting. Dropout is powerful technique that reduces this problem. In this technique, we randomly drop a small percentage of neurons from the input and the hidden layers.

It changes the weights of all the forward and the backward connection of the dropped nodes to zero, thus creating a new network architecture within the parent network. For example, if a hidden layer has 1000 neurons and we have set the Dropout to 20%, then the model will randomly select 200 neurons and change their weights to zero.

Batch Normalization is an algorithmic technique which is meant to make the training of Deep Neural Networks faster and more stable. During training, the weights in the neural network layers are updated continuously. This causes the inputs to the layers to change abruptly, and the network keeps on chasing a moving target.

Batch Normalization uses a technique called Standardization to stabilize the inputs to these layers. Standardization is scaling technique in which we convert each value into its respective Z-score. A result of this transformation is that the resultant variables now have a mean of 0 and standard deviation of 1.