Deep Learning

This is one of the most frequently asked deep learning coding interview questions and answers for freshers in recent times.

Image classification and tagging, Face detection and Video recognition are use cases of machine vision. So option “i”,”iii”,”iv” are correct answers.

Image search systems use deep learning for image classification and automatic tagging, which allows images to be accessible through a standard search query. For example: companies such as Facebook use deep nets to scan pictures for faces at many different angles, and then label the face with the proper name.

Face detection is a computer technology being used in a variety of applications that identifies human faces in digital images. Deep nets are also used to recognize objects within images which allow for images to be searchable based on the objects within them.

Video recognition systems are important tools for driverless cars, remote robots, theft detection etc. Typically video is an ordered set of frames of same resolution. It has got two parts – video stream and video sequence. Video stream is an ongoing video for online real-time procession. Video sequence is a video of fixed length.

A deep learning platform provides a set of tools and an interface for building custom deep nets. Typically they provide a user with a selection of deep nets to choose from, along with the ability to integrate data from different sources, manipulate data, and manage models through user interface. Some platforms also help with performance if a net needs to be trained with a large dataset. The platform is typically an out of the box application that lets us configure a deep net’s hyper-parameters through an intuitive UI. With a platform, we don’t need to know anything about coding aspects in order to use tools. The downside is that we are constrained by the platform’s selection of deep nets as well as the configuration options. However, for somebody to quickly deploy a deep net, a platform could be the best way to go. Deep learning platforms come in two different forms – software platform and full platform. Examples are H2O.ai, Data Graph etc

Library is a set of functions and modules that we can call through our own programs in order to perform certain tasks. Deep net libraries gives us lot of flexibility with net selection and hyper-parameter configuration. For example: there are not many platforms that let us build a Recursive Neural Tensor Net (RNTN), however we can code our own with the appropriate deep net library. The obvious downside here is the coding experience required to use them. However if we need flexibility, these are great options. Examples are TensorFlow, Theano, Caffe, deeplearning4j, torch etc.

This is one of the most frequently asked deep learning interview questions for freshers in recent times.

Artificial Neural Networks (ANN) can be of two main categories: Feed-forward neural networks and recursive neural networks. They are flexible to many possible variations based on – the learning rule, the inference rule and the architecture.

Feed forward neural network can be described as per below diagram:

Perceptron algorithm is a fundamental computational unit at heart of the Deep learning model. It’s a supervised learning algorithm for binary classification.

There are many packages available for ANNs. In R, some of the packages are nnet, neuralnet, RSNNS, deepnet, darch, caret, RNN, Autoencoder, RcppDL, MXNetR and others for more specific tasks.

Answer: [5]

The answer will be [5].

The program creates a graph by using source operations. These source operations will pass their information to other operations which will execute computations.

Here in order to create two source operations which will output numbers, two constants are defined in a and b. After that, function tf.add() adds two elements and stores the calculation in c.

The with block then prints the result by opening the session.

After running the with block, session will close automatically.

Answer: 0, 1, 2

The result will print 0 to 2.

The variable is first defined using tf.Variable() and it is initialized with 0. Then in order to counter from the initial value, tf.assign() function is used. It takes two arguments – the reference_variable as 1st argument and value_to_update as a 2nd argument.

Variables must be initialized by running an initialization operation after having launched the graph. We first have to add the initialization operation to the graph.

We then start a session to run the graph, first initialize the variables, then print the initial value of the state variable, and then run the operation of updating the state variable and printing the result after each update.

Answer: 7.0

The tf.float32 will define a 32 bit floating point. The first line creates a placeholder and then it is multiplied by 2 and kept in second variable as b.

Now we need to define and run the session, but since we created a "hole" in the model to pass the data, when we initialize the session we are obligated to pass an argument with the data, otherwise we would get an error.

To pass the data into the model we call the session with an extra argument feed_dict in which we should pass a dictionary with each placeholder name followed by its respective data, like the way it was shown in above code snippet.

This is one of the most frequently asked deep learning coding interview questions and answers for freshers in recent times.

Operations are nodes that represent the mathematical operations over the tensors on a graph. These operations can be any kind of functions, like add and subtract tensor or maybe an activation function.

ALL of these four - tf.constant, tf.matmul, tf.add, tf.nn.sigmoid are operations in TensorFlow. These are like functions in python but operate directly over tensors and each one does a specific thing.

While others are straight forward and simple, tf.nn.sigmoid is an activation function, it's a little more complicated, but this function helps learning models to evaluate what kind of information is good or not.

The output will be as follows:

c =: 8
d =: 4

Operations are nodes that represent the mathematical operations over the tensors on a graph. These operations can be any kind of functions, like add and subtract tensor used here. tf.constant function defines the values to variables a and b.

Then c and d performs add and subtract operations respectively based on the given value in the session. Hence it outputs desired result and prints it.

Option c would be the correct answer.

The size of the convoluted matrix is given by C = ((I-F+2P)/S)+1, where

• C is the size of the convoluted matrix.
• I is the size of the input matrix.
• F is the size of the filter matrix.
• P is the padding applied to the input matrix.
• S is the stride applied.

Here I = 12, F = 3, P = 0, S = 1

Therefore the answer is 10 X 10 matrix.

Option c is the correct answer.

Softmax function is of the form in which the sum of probabilities over all i sum to 1.

If we take an input of [1, 2, 3, 4, 1, 2, 3], the softmax of that is [0.024, 0.064, 0.175, 0.475, 0.024, 0.064, 0.175]. The output has most of its weight where the '4' was in the original input. This is what the function is normally used for: to highlight the largest values and suppress values which are significantly below the maximum value. However please note that, softmax is not scale invariant, so if the input were [0.1, 0.2, 0.3, 0.4, 0.1, 0.2, 0.3] (which sums to 1.6) the softmax would be [0.125, 0.138, 0.153, 0.169, 0.125, 0.138, 0.153]. This shows that for values between 0 and 1 softmax, in fact, de-emphasizes the maximum value (note that 0.169 is not only less than 0.475, it is also less than the initial proportion of 0.4/1.6=0.25)

If we execute below code snippet in Python 3.x, then we can get the desired output for the example of [1,2,3,4,1,2,3] that we mentioned above.

Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or approximate gradient) of the function at the current point. If, instead, one takes steps proportional to the positive of the gradient, one approaches a local maximum of that function; the procedure is then known as gradient ascent.

1. The steps to use gradient descent algorithm is as follows:
2. Initialize random weight and bias
3. Pass an input through the network and get values from the output layer
4. Calculate error between the actual value and the predicted value
5. Go to each neurons which contributes to the error and change its respective values to reduce the error
6. Reiterate until we find the best weights of the network

Below is an illustrative diagram of gradient descent on a series of level sets.

Dropout is a regularization technique patented by Google for reducing overfitting in neural networks by preventing complex co-adaptations on training data. It is a very efficient way of performing model averaging with neural networks. The term "dropout" refers to dropping out units (both hidden and visible) in a neural network.

Dropout can be seen as an extreme form of bagging in which each model is trained on a single case and each parameter of the model is very strongly regularized by sharing it with the corresponding parameter in all the other models.

Key points are as follows: Large weights in a neural network are a sign of a more complex network that has overfit the training data. Probabilistically dropping out nodes in the network is a simple and effective regularization method. A large network with more training and the use of a weight constraint are suggested when using dropout.

• Bias – When the machine learning model has a “high bias” then it does not take into account the variation in the data and underfits the data
• Variance – When the machine learning model learns all extraneous information from the data then it tends to overfit the data but does not generalize well.

Goodness of fit is to strike a balance between bias and variance.

Yes, overfitting can occur in a neural network. There are various ways to handle overfitting in a neural network which are as follows:

1. Dropout
2. Regularization
3. Batch normalization

Dropout is a regularization method that approximates training a large number of neural networks with different architectures in parallel. Dropout has the effect of making the training process noisy, forcing nodes within a layer to probabilistically take on more or less responsibility for the inputs.

Some of the key aspects for using dropout regularization are:

• Use with all network types - It can be used with most, perhaps all, types of neural network models, not least the most common network types of Multilayer Perceptrons, Convolutional Neural Networks, and Long Short-Term Memory Recurrent Neural Networks. In the case of LSTMs, it may be desirable to use different dropout rates for the input and recurrent connections.
• Dropout rate - The default interpretation of the dropout hyperparameter is the probability of training a given node in a layer, where 1.0 means no dropout, and 0.0 means no outputs from the layer. A good value for dropout in a hidden layer is between 0.5 and 0.8. Input layers use a larger dropout rate, such as of 0.8.
• Grid search parameters – Instead of guessing at a suitable dropout rate for your network, test different rates systematically. For example, test values between 1.0 and 0.1 in increments of 0.1. This will both help discover what works best for our specific model and dataset, as well as how sensitive the model is to the dropout rate. A more sensitive model may be unstable and could benefit from an increase in size.

Batch normalisation is a technique for improving the performance and stability of neural networks. The idea is to normalise the inputs of each layer in such a way that they have a mean output activation of 0 and standard deviation of 1. This is analogous to how the inputs to networks are standardised.

In Keras, it is implemented using the following code snippet. Note how the BatchNormalization call occurs after each fully-connected layer, but before the activation function and dropout.

Expect to come across this, one of the most important Python deep learning interview questions for experienced professionals in deep learning, in your next interviews.

Yes, CNN has the pooling layer.

• When it is added to a CNN, yes, Translation invariance is induced when you use pooling.
• One of the advantages of max-pooling is translation invariance. It provides a form of translation invariance.

Invariance means that we can recognize an object as an object, even when its appearance varies in some way. It preserves object’s identity, category etc across the changes in the specifics of visual input, like relative positions of the viewer and object.

The translation has a specific meaning in vision, generally borrowed from geometry. It does not refer to any type of conversion, unlike say, a translation from French language to English language or between file formats. Instead, it means that each point/pixel in the image has been moved the same amount in the same direction. Alternately, you can think of the origin as having been shifted an equal amount in the opposite direction.

So typically, CNN + max-pooling ~ Translation invariance