Statistics: Law of large numbers and Chebyshev’s inequality

This is part of the course “Probability Theory and Statistics for Programmers”.

Image for post
Image for post
Probability Theory and Statistics For Programmers

Before introducing the law of large number it is better to understand Chebyshev’s inequality first.

Chebyshev’s inequality shows that for any positive number a probability of random variable X to deviate from an expected value not less than by a is equal to:

Image for post
Image for post
Chebyshev’s inequality

For example, what is the probability that after rolling dice result will deviate from an expected value not less than two? At first, we will calculate the expected value.

It is equal to 3.5. Now we can draw all possible values and interval (m-a, m+a).

Image for post
Image for post

For our example, we need to find the probability of getting values out of the range — the probability of getting 1 or 6.

As you can see Chebyshev’s inequality gives an only upper limit of probability deviation. Probability can’t be more than this value no matter what.

The law of large numbers is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value.

The law of large numbers can be proven by using Chebyshev’s inequality. There is a random variable X. Above this value performed n independent experiments and calculated average. As a result, we have random variable Y.

Image for post
Image for post
random variable Y

Let’s find the expected value and variance of an obtained random variable.

Image for post
Image for post
expected value and variance of random variable Y

As you can see expected value is independent of a number of experiments and equal to an expected value of X. Variance decrease as a number of experiments growth. Because variance becomes very small random variable Y becomes not random. Since there is no randomness when you have variance almost equals to zero. Now we came to this inequality.

Image for post
Image for post

And we can apply Chebyshev’s inequality for random variable Y.

Image for post
Image for post

Let’s take a look at an example. As X would be used a continuous random variable with uniform distribution.

Next part ->

Reach the next level of focus and productivity with increaser.org.

Image for post
Image for post
Increaser

Written by

Software engineer, creator of increaser.org. More at geekrodion.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store