Preface
In machine learning or deep learning topics, theories of statistical probability are often used frequently to assist in data processing and research. Therefore, it is very necessary to understand and master certain statistical probability knowledge. In the fields of scientific research and urban research, the application of statistical probability theory is also very common. In order to understand the probability distribution well, first look at the following related concepts:
- Random Variable: A random variable is a quantity that can take multiple possible values, which are determined based on a certain probability distribution.
- Density Functions: In continuous random variables, the density function describes the probability density of each value that occurs within the range of possible values of the random variable. It is usually used to calculate probability, expected value, etc.
- Bernoulli Distribution: A discrete probability distribution that describes a random variable that has only two possible values (usually expressed as 0 and 1), such as the result of a coin toss.
- Binomial Distribution: Used to describe the probability distribution of successes in a series of independent repeated trials, with only two possible results per trial and the probability of success is the same.
- Uniform Distribution: Each possible value has the same probability distribution within a certain range, without obvious bias.
- Poisson Distribution: Used to describe the probability distribution of random events occurring in a fixed time or space, and is often used to describe rare events.
- Normal Distribution: Also known as Gaussian Distribution, it is a continuous distribution that is extremely common in statistics, usually with a bell-shaped curve and a symmetrical distribution.
- Long-Tailed Distribution: refers to a relatively long distribution of the tail (larger or smaller values) of the probability distribution, indicating that there is a higher possibility of extreme values.
- Student’s t-test distribution: a distribution used for statistical inference in small samples, mainly used to compare whether there is a significant difference in the mean of the two groups of data.
- Lognormal Distribution: A continuous distribution whose logarithm follows a normal distribution and is often used to describe positive data, such as financial data, stock returns, etc.
- Exponential Distribution: A probability distribution used to describe the time interval waiting for a random event to occur in a continuous time, and is often used to describe time intervals, lifespans, etc.
- Weibull Distribution: A continuous distribution, commonly used to describe time arrival, life span, etc., has a flexible shape that can adapt to different types of data.
- Gamma Distribution: A continuous distribution that is widely used to describe the distribution of positive random variables, and can also be used to describe waiting time, lifespan, etc.
- Chi-square Distribution: A distribution used in statistical inference, usually used to test the fit between an observed value and an expected value.
- Central Limit Theorem: This theorem shows that when a large number of independent random variables are extracted from any distribution and their averages are calculated, the distribution of these averages will be approximately normal distributions and is not affected by the original distribution.
Gaussian distribution
The Gaussian distribution is probably the most commonly heard and familiar distribution. It has several names: some call it a bell curve, because its probability graph looks like a bell, some call it a Gaussian distribution, because the German mathematician who first described it, Karl Gauss, also some call it a normal distribution, because early statisticians noticed it happening again over and over again. The probability density function of a normal distribution is as follows: σ is the standard deviation and μ is the average value of the distribution. It should be noted that in a normal distribution, the mean, mode and median are all equal. When we plot a random variable of a normal distribution, the curve symmetric around the mean - half of the value is on the left side of the center and half on the right side of the center. And, the total area under the curve is 1.
mu = 0
variance = 1
sigma = (variance)
x = (mu - 3*sigma, mu + 3*sigma, 100)
(figsize=(8, 5))
(x, (x, mu, sigma))
("Normal Distribution")
()
For normal distributions. Empirical rules tell us that the percentage of data falls within a certain number of standard deviations of the average. These percentages are:
- 68% of the data fall within one standard deviation of the mean.
- 95% of the data fall within two standard deviations of the mean.
- 99.7% of the data fall within the three standard deviation ranges of the mean.
Logarithmic normal distribution
A log-normal distribution is a continuous probability distribution of random variables whose logarithm is normally distributed. Therefore, if the random variable X is lognormally distributed, Y = ln(X) has a normal distribution. This is a log-normal distribution PDF: Random variables with log-normal distribution only take positive real values. Therefore, a log-normal distribution creates a right-bias curve. Let's draw it in Python:
X = (0, 6, 500)
std = 1
mean = 0
lognorm_distribution = ([std], loc=mean)
lognorm_distribution_pdf = lognorm_distribution.pdf(X)
fig, ax = (figsize=(8, 5))
(X, lognorm_distribution_pdf, label="μ=0, σ=1")
ax.set_xticks((min(X), max(X)))
std = 0.5
mean = 0
lognorm_distribution = ([std], loc=mean)
lognorm_distribution_pdf = lognorm_distribution.pdf(X)
(X, lognorm_distribution_pdf, label="μ=0, σ=0.5")
std = 1.5
mean = 1
lognorm_distribution = ([std], loc=mean)
lognorm_distribution_pdf = lognorm_distribution.pdf(X)
(X, lognorm_distribution_pdf, label="μ=1, σ=1.5")
("Lognormal Distribution")
()
()
Poisson distribution
The Poisson distribution is named after the French mathematician Simon Dennis Poisson. This is a discrete probability distribution, which means it computes events with finite results—in other words, it is a count distribution. Therefore, the Poisson distribution is used to show how many times an event may occur within a specified period. If an event occurs at a fixed rate in time, the probability of timely observing the number (n) of events can be described by Poisson distribution. For example, a customer may arrive at a cafe at an average rate of 3 times per minute. We can use the Poisson distribution to calculate the probability that 9 customers arrive in 2 minutes. Here is the formula for the probability mass function: λ is the rate of event in a unit of time—in our case, it is 3. k is the number of occurrences-in our case, it is 9. Here, Scipy can be used to calculate the probability.
from scipy import stats print((k=9, mu=3))
The results are as follows
0.002700503931560479
The curve of the Poisson distribution is similar to a normal distribution, and λ represents the peak.
X = (mu=3, size=500)
(figsize=(8, 5))
(X, density=True, edgecolor="black")
("Poisson Distribution")
()
Exponential distribution
The exponential distribution is the probability distribution of time between events during Poisson's point process. The probability density function of the exponential distribution is as follows: λ is the rate parameter and x is a random variable.
X = (0, 5, 5000)
exponetial_distribtuion = (X, loc=0, scale=1)
(figsize=(8,5))
(X, exponetial_distribtuion)
("Exponential Distribution")
()
Binomial distribution
The binomial distribution can be considered as the probability of success or failure in an experiment. Some people may also describe it as a probability of tossing a coin. The binomial distribution with parameters n and p is a discrete probability distribution of successes in n independent experimental sequences. Each experiment asks a yes-no question, and each experiment has its own Boolean result: success or failure. Essentially, a binomial distribution measures the probability of two events. The probability of one event occurring is p and the probability of another event occurring is 1-p. This is the formula for the binomial distribution:
- P = binomial distribution probability
- = Number of combinations
- x = number of specific results in n trials
- p = probability of success in a single experiment
- q = probability of failure in a single experiment
- n = Number of experiments
The visual code is as follows:
X = (n=1, p=0.5, size=1000)
(figsize=(8, 5))
(X)
("Binomial Distribution")
()
Student t Distribution
The Student’s t distribution (or t distribution for short) is any member of the continuous probability distribution family that occurs when estimating the mean of the normal distributed population with a small sample size and unknown overall standard deviation. It was developed by British statistician William Sealy Gosset under the pseudonym "student". The PDF is as follows: n is a parameter called "degree of freedom", and it can sometimes be seen as "." For higher n values, the t distribution is closer to the normal distribution.
import seaborn as sns
from scipy import stats
X1 = (df=1, size=4)
X2 = (df=3, size=4)
X3 = (df=9, size=4)
(figsize=(8,5))
(X1, label = "1 ")
(X2, label = "3 ")
(X3, label = "6 ")
("Student's t distribution")
()
()
Chi-square distribution
The chi-square distribution is a special case of the gamma distribution; for k degrees of freedom, the chi-square distribution is the sum of squares of k of some independent standard normal random variables. The PDF is as follows: This is a popular probability distribution, commonly used in the construction of hypothesis tests and confidence intervals. Draw some example diagrams in Python:
X = (0, 6, 0.25)
(figsize=(8, 5))
(X, stats.(X, df=1), label="1 ")
(X, stats.(X, df=2), label="2 ")
(X, stats.(X, df=3), label="3 ")
("Chi-squared Distribution")
()
()
This is the article about the detailed explanation of Python's implementation of probability distribution formulas and visualization. For more related Python probability distribution content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!