Statistics and Probability will be a part of whichever field you are in. Understanding variables' behaviors and attributes are a significant part of Data Science, which is difficult without knowledge of Distributions. In simplest terms, Probability distribution is a means to show a variable's potential values and corresponding probabilities. The backbone of Data Science and Machine Learning is Probability and Statistics understanding; to properly collect, examine, analyze, and communicate with data, you will need both skills. This calls for you to understand several basic terminologies, what they signify, and how to spot them.
In the real world, several phenomena are considered statistical (i.e., weather data, sales data, financial data, etc.). This indicates that there are numerous occasions where we have been able to create approaches that assist us in simulating nature using mathematical functions that can characterize the properties of data. If you want to get familiar with statistical studies like probability distribution and its contribution to improving Machine Learning solutions. You can start your journey towards being a Data scientist through a complete Data Science Bootcamp.
What is Probability Distribution?
A mathematical function called Probability distribution explains a variable's likelihood of many alternative values. Graphs or Probability tables are frequently used to represent Probability distributions- a statistical function called Probability distribution explains all the potential values and probabilities for a random variable within a specified range. The minimum and highest possible values will be used to limit this range.
However, other circumstances will affect where the potential value would be drawn on Probability distribution. The distribution's skewness, kurtosis, and mean (average) are among these variables. There are two types of distribution in Statistics for Probability that are discrete and continuous, respectively.
According to Wikipedia, a Probability distribution is a mathematical function that estimates the likelihood that several possible outcomes of an experiment will occur. In terms of its sample space and event probability, it is a mathematical description of random phenomena (subsets of the sample space).
For Example
Let us examine the result of rolling two conventional six-sided dice as a straightforward illustration of a Probability distribution. A roll of any number from one to six has a 1/6 chance on each dice. However, the aggregate of two dice will provide the Probability distribution. The most frequent result is seven (1+6, 6+1, 5+2, 2+5, 3+4, 4+3).
Contrarily, two and twelve are much less likely (1+1 and 6+6).
What is Probability Distribution Used For?
Probability Distributions are theoretical since obtaining infinitely large samples in practice is impossible. They are idealized frequency distributions intended to represent the population from which the sample was taken.
Probability distributions are used to characterize the populations of real-world variables, such as coin tosses or the weight of chicken eggs. They are also used to calculate p values in hypothesis testing. Probability distributions are useful in modeling our environment to acquire estimates of the likelihood that a specific event will occur or to determine the variability of occurrence. They are a typical technique to explain and forecast an event's likelihood.
Ways of Displaying Probability Distributions
A formula can describe probability distributions or display them in tables and graphs. For instance, binomial probabilities can be computed using the binomial formula.
For Example, the probability distribution of rotten tomatoes in a tomato packaging business is displayed in the table below. The probability in the second row adds up to 1 if you add them all together (.95 +.02 +.02 + 0.01 = 1).
| Rotten Tomatoes (X) | 0 | 1 | 2 | 3 |
|---|
| Probability P(X) | 0.95 | 0.02 | 0.02 | 0.01 |
The standard normal distribution, perhaps the most popular Probability distribution, is depicted in the graph below. The "bell curve" is another name for the typical normal distribution in data science. Numerous natural phenomena, such as heights, weights, and IQ scores, fit the bell curve. Since the normal curve represents a continuous probability distribution, the total area is one rather than the sum of the individual probabilities under the curve.
Probability Distribution Table
Every result of a statistical experiment is connected to the likelihood that the result will occur shown in a probability distribution table. An experiment's result is recorded as a random variable, typically written with a capital letter (X or Y).
For example, the probable results of tossing a coin three times are:
TTT, HTT, THH, HTH, HHT, and HHH.
If you toss TTT, your odds of obtaining no heads are 1 in 8. There is a 1/8 or 0.125 chance of obtaining three heads, a 3/8 or 0.375 chance of getting only one head with TTH, THT, and HTT, and a 3/8 or 0.375 chance of getting only two heads with THH, HTH, or HHT.
The likelihood that you will receive 0, 1, 2, or 3 heads is shown in the table below, along with the random variable (the number of heads).
| Number of Heads (X) | Probability P(X) |
|---|
| 0 | 0.125 |
| 1 | 0.375 |
| 2 | 0.375 |
| 3 | 0.125 |
| Distribution | Formula | Type of Formula |
|---|
| Binomial |
| Probability mass function |
| Discrete uniform |
| Probability mass function |
| Poisson |
| Probability mass function |
| Normal |
| Probability density function |
| Continuous uniform |
| Probability density function |
| Exponential |
| Probability density function |
If you want to get familiar with statistical studies like probability distribution and its contribution to improving machine learning solutions. You can start your journey towards being a Data scientist with a Data Science course in India.
How Does Probability Distribution Work?
Several extensively used Probability distributions exist. Still, the normal distribution, also known as the "bell curve," is arguably the most used. Usually, the Probability distribution of a phenomenon is determined by the method used to generate the data. The probability density function is the name given to this process.
Cumulative Distribution functions (CDFs), which sum up the probability of occurrences cumulatively and always start at zero and end at 100%, can also be made using Probability distributions.
The probability distribution of a given stock can be determined by academics, financial analysts, and fund managers to assess the potential expected returns that the stock may provide. The analysis will be prone to sampling error since the stock's history of returns, which can be measured over any time period, is probably only made up of a small portion of the stock's returns. The size of the sample can be increased, significantly lowering this inaccuracy.
Key Lessons
- An example of a Probability distribution shows the expected outcomes of potential values for a specific data generation procedure.
- The Mean, Standard deviation, Skewness, and Kurtosis of Probability distributions are used to identify their various shapes and features.
- Probability distributions are used by investors to predict future returns on assets like stocks and to manage risk.
Different Types of Probability Distribution with Examples
There are two Probability distribution types: Discrete Probability distribution and Continuous Probability distribution:
A) Discrete Probability Distributions
Discrete Probability distribution determines the probabilities of outcomes for discrete random variables. In other words, it aids in figuring out the probability that a random variable will take on a specific value within a predetermined range. Discrete Probability distributions include, for instance, the likelihood that ten coin flips will result in a head. A Discrete Probability distribution gives each discrete result of a random variable a probability. The roll of a die or the toss of a coin are examples of events whose outcomes can only be predicted using Discrete Probability distributions.
They can also simulate more complicated phenomena, such as the volume of website visits on a particular day. In general, Discrete Probability distributions are categorized based on the kind of random variable they express. The Binomial distribution, which simulates events with two alternative outcomes, success and failure, is the most prevalent kind of Discrete Probability distribution. Bernoulli, Poisson, Geometric, and Negative Binomial distributions are some further instances of Discrete Probability distribution. A probability mass function can depict Discrete Probability distribution (PMF).
1. Binomial Distribution
It has a binary nature or has two alternative outcomes. It represents the probability distribution of the number of successful trials out of n trials with p success probabilities. Criteria of Binomial Distribution include Fixed and Independent trials, Fixed probability of success, and Two mutually exclusive outcomes.
You can Calculate Binomial Probability Using the following:
Here, p is the probability of success, n is the trials, and x is the expected outcome, like out of n=10 trials getting x=5 heads with p=0.5 which is fixed for getting head or tail in fair coin.
Graph of Binomial Distribution
Example 1: If a coin is tossed five times, find the probability of;
(a) Exactly Two Heads
Solution:
The repeated tossing of the coin is an example of a Bernoulli trial. According to the problem:
Number of trials: n=5
Probability of head: p= 1/2 and hence the probability of tail, q =1/2
For exactly two heads:
x=2
P(x=2) = 5C2 p2 q5-2 = 5! / 2! 3! × (½)2× (½)3
P(x=2) = 5/16
2. Bernoulli’s Distribution
The binomial distribution can be referred to as the Bernoulli distribution for n = 1 (one experiment). When n = 1, the Bernoulli distribution is frequently referred to as a particular case of the binomial distribution.
The probability density function (pdf) for this distribution is:
Example: Probability of getting head if a fair coin tossed once, p(n=1)=0.5.
The probability of a failure is labeled on the x-axis as 0, and success is labeled as 1. In the following Bernoulli distribution, the probability of success (1) is 0.4, and the probability of failure (0) is 0.6:
Bernoulli distribution for p(n=1)=0.4
3. Poisson Distribution
The Poisson distribution gives the probability of an event happening k number of times within a given interval of time or space.
The probability density function (pdf) for this distribution is:
f(x; λ) = P(X = x) = λx × e-λ/x!
λ: Average Success Rate and x=Number of success
Example
Every year, 0.61 soldiers in each Prussian army corps perished from horse kicks. When assuming that the annual number of horse kick deaths follows a Poisson distribution, you wish to determine the likelihood that the VII Army Corps seven soldiers perished in 1898, exactly two of whom perished.
Calculation
The specific army corps (VII Army Corps) and year (1898) don’t matter because the probability is constant.
k= 2 deaths by horse kick
λ= 0.61 deaths by horse kick per year
e= 2.718
f(x; λ) = P(X = x) = λ× e-λ/x!
P(X=2)=0.612 *(2.718-2)/2!
=0.101
The probability that exactly two soldiers died in the VII Army Corps in 1898 is 0.101.
B) Continuous Probability Distributions
Continuous Probability distribution deals with random variables that can have any continuous value within a specific range. Contrary to Discrete Random Variables, which can have only definite, precise values, continuous random variables can take on various values. Like height, weight, and volume, continuous random variables are frequently used in Mathematics. The radioactive decay rate or sound waves' speed are two examples of physical processes often modeled using continuous probability distributions.
Continuous Probability distributions come in various forms, each with its shape. The Normal bell-shaped distribution is the most prevalent. Continuous Probability distributions can represent a wide range of real-world phenomena. A Continuous Probability distribution, for instance, might be used to characterize the height of the students in a classroom. While allowing for the potential of any height being seen, the distribution would account for the fact that some students are taller than others.
Similar to this, a Continuous Probability distribution might be used to characterize the weight of newborn infants. This would allow for the possibility of any weight being detected and the reality that some babies are born heavier than others. In both situations, the distribution's continuous nature reflects the underlying phenomenon's continuous nature. Here we are explaining a few continuous probability distribution types with examples.
1. Normal Distribution
A continuous probability distribution for a real-valued random variable. Most of the observations are centered around the central peak of this symmetric distribution, and the probability for values that are further from the mean taper off equally in both directions.
A bell-shaped density curve with a Mean and Standard deviation represents it. The Gaussian distribution is another name for it. It has the following characteristics: Symmetrical bell forms.
Equal in value, and the middle of the distribution are mean and median.
- The data is within one standard deviation of the mean for 68% of the sample.
- Data is within two standard deviations of the mean for 95% of the sample.
- The data is within three standard deviations of the mean in 99.7% of the sample.
Probability Mass Function of Normal Distribution
F(x) is the probability density function, σ is the standard deviation, and μ is the sample mean
Normal Distribution Probability Examples
In a brand-new test preparation course, you gather the pupils' SAT scores. The data has a Mean score (M) of 1150 and a Standard Deviation (SD) of 150 and is distributed normally.
Using the Empirical Principle
- Between 1,000 and 1,300, or 1 standard deviation above and below the mean, account for about 68% of scores.
- Between 850 and 1,450, or two standard deviations above and below the mean, almost 95% of scores fall.
- Between 700 and 1,600, or 3 standard deviations above and below the mean, are where 99.7% of scores fall.
2. Continuous Uniform Distribution
A probability distribution with a constant probability is known as a Uniform Distribution or a rectangle distribution.
Two factors, a and b, determine this distribution:
- The minimum is a.
- The maximum is b.
The distribution is written as U(a, b).
Distribution for a = one and b = three are displayed in the graph below:
3. Log-Normal Distribution
A probability distribution with a normally distributed logarithm is known as a lognormal (log-normal or Galton) distribution. If a random variable's logarithm has a normal distribution, it is said to be lognormally distributed.
This kind of distribution frequently fits skewed distributions with low mean values, high variation, and only positive values. Since log(x) can only exist for positive values of x, values must be positive.
The probability density function is defined by the mean μ and standard deviation, σ:
The shape of the lognormal distribution is defined by three parameters:
- σ, the shape variable. Additionally, the log-normal Standard Deviation impacts the distribution's overall form. These parameters are often known from past data. You might occasionally be able to estimate it using recent data. The location and height are unaffected by the shape parameter.
- m, the scale parameter (this is also the median). This parameter shrinks or stretches the graph.
- Θ (or μ), the location parameter, tells you where the graph is on the x-axis.
For example, the following phenomenon can all be modeled with a lognormal distribution:
- Milk production by cows
- Lives of industrial units with failure modes characterized by fatigue stress
- Amount of rainfall
- Size distributions of rainfall droplets
- The volume of gas in a petroleum reserve
4. Exponential Distribution
The exponential distribution is a Continuous Probability distribution used in statistics that frequently deals with how long until a particular event occurs. Events occur continually, independently, and at a steady average pace during this process. The crucial characteristic of the exponential distribution is that it has no memory.
Either more little values or fewer larger values can make up the exponential random variable. For instance, a customer's total spending on a single trip to the grocery store has an exponential distribution. The probability density function of Exponential Distribution is shown below:
The continuous random variable, say X, is said to have an Exponential distribution if it has the following Probability density function:
Where λ is called the distribution rate and the mean of the Exponential distribution is 1/λ, and variance is 1/λ2, and the memoryless quality of the exponential distribution is its most significant characteristic.
The density functions of exponential distributions concerning different parameters λ
The following list includes some of the Exponential distribution models fields:
- Equivalent distribution aids in determining the separation of mutations on a DNA strand.
- Figuring out how long it will take the radioactive particle to decay.
- Assists in determining the height of various molecules in a gas at a constant temperature, pressure, and gravitational field.
- Aids in computing the highest monthly and annual amounts of normal rainfall and river outflow volumes.
If you want to get familiar with Statistical studies like Probability distribution and its contribution to improving Machine learning solutions. You can start your journey towards becoming a Data scientist with KnowledgeHut’s complete Data Science Bootcamp.
Relationship Between Various Probability Distributions
1. List of Probability Distributions (Univariate and Discrete)
| Start from Distribution | Do | Obtain |
|---|
| Bernoulli Distribution | Sum of independent Bernoulli random variables. | Binomial distribution |
| Observe a sequence of realizations of independent Bernoulli random variables and record the number of 0s obtained before a one shows up. | Geometric distribution |
| Binomial Distribution | Set parameter for several trials equal to 1. | Bernoulli distribution |
2. List of Probability Distributions (Univariate and Continuous)
| Start from Distribution | Do | Obtain |
|---|
| Exponential Distribution | Sum of independent exponential random variables with common rate parameter. | Gamma distribution |
| Keep summing the realizations of independent exponential random variables while the sum is less than 1. Record the number of variables you have summed. | Poisson distribution |
| Normal Distribution | Take n mutually independent standard normal variables, square them, and sum them. | Chi-square distribution |
| Take a linear combination of n mutually independent normal variables. | Normal distribution |
| Add a constant and multiply by a constant. | Normal distribution |
| Take the exponential. | Log-normal distribution |
| Collect independent normal random variables in a vector. | Multivariate normal distribution |
| Log-normal Distribution | Take the natural logarithm. | Normal distribution |
Examples of Probability Distribution
Types of Continuous Probability Distributions
| Types of Continuous Probability Distribution | Example |
|---|
| Normal | SAT scores |
| Continuous Uniform | The number of times cars wait at a red light |
| Log-normal | The average body weight of different mammal species |
| Exponential | Time between earthquakes |
Common Discrete Probability Distributions
| Types of Discrete Probability Distribution | Example |
|---|
| Binomial | The number of times a coin lands on heads when you toss it five times |
| Discrete Uniform | The suit of a randomly drawn playing card |
| Poisson | The number of text messages received per day |
Probability Distribution Function
The probability distribution function is a function that is used to define the distribution of a probability. We can define these functions based on the types. Additionally, for each given random variable, these functions are employed in terms of Probability density functions.
The probability distribution table can be made based on the random variable and potential outcomes. Imagine that the sample space of a random experiment serves as the domain of the random variable X, which is a real-valued function. The system of numbers is the probability distribution P(X) of a random variable X.
| X | X1 | X2 | X3 | ...... | Xn |
| P(X) | P1 | P2 | P3 | ...... | Pn |
where Pi > 0, i=1 to n and P1+P2+P3+ …….. +Pn =1
Probability Distribution Formulas
| Binomial Distribution | Where a = probability of success b=probability of failure n= number of trials x=random variable denoting success
|
| Cumulative Distribution Function |
|
| Discrete Probability Distribution |
|
Importance of Probability Distribution in Statistics and Data Science
Importance of Probability Distribution in Statistics
The probability distribution's primary benefit is its capacity to calculate the likelihood of any given observation occurring in a sample space. A mathematical model known as Probability distribution determines the likelihood that various potential outcomes of a test or experiment will occur.
Used to provide several random variable types (often discrete or continuous) to base decisions on these models. One can utilize the mean, mode, range, probability, or other statistical methods depending on the type of random variable. In Statistics, Probability distributions are a fundamental concept. They have both theoretical and practical applications of a probability distribution. Probability distributions can be used in the following ways:
- To compute crucial regions for hypothesis tests and confidence intervals for parameters.
- Finding a suitable distributional model for univariate data is frequently helpful.
- Specific distributional assumptions are frequently the foundation of statistical intervals and hypothesis testing. We must ensure that the available data set supports the distributional assumption before constructing an interval or test based on the assumption. In this situation, the distribution need only be adequate to allow the statistical method to produce reliable results rather than having to be the distribution that fits the data the best.
- It is frequently necessary to conduct simulation experiments using random numbers produced using a particular probability distribution.
- In Data Science, it can be used to identify features utilizing hypothesis testing and p-value.
Significance of Probability Distributions in Data Science
In a way, most of the data science and machine learning operations are dependent on several assumptions about the probability of your data. Probability distribution allows a skilled data analyst to recognize and comprehend patterns from large data sets; that is, otherwise, entirely random variables and values. Thus, it makes probability distribution a toolkit based on which we can summarize a large data set. The density function and distribution techniques can also help in plotting data, thus supporting data analysts to visualize data and extract meaning. To know more about linear discriminant analysis in machine learning, check here!
Conclusion
To acquire estimates of the likelihood that a specific event is to occur or to determine the variability of occurrence, Probability distributions are useful in modeling our environment. They are a typical technique to explain and forecast an event's likelihood. Since it is required to determine what distribution should be used to model a specific process, the key problem is defining the properties of the variables whose behavior we seek to describe.
The proper use of a model, such as the standardized normal distribution, which may be used to estimate the likelihood of an occurrence, is made possible by selecting the appropriate distribution. Gain in-depth knowledge of Probability distribution through a Data Science course.
