BSc CSIT (TU) Science Statistics I (BSc CSIT, STA164) Question Paper 2079 Nepal

Probability and Its Approaches

Probability is a numerical measure of the likelihood that a particular event will occur, lying between $0$ and $1$. If $P(A)=0$ the event is impossible and if $P(A)=1$ the event is certain.

(a) Classical (Mathematical / a priori) Approach

If a random experiment has $n$ equally likely, mutually exclusive and exhaustive outcomes, and $m$ of them are favourable to event $A$, then

$$P(A)=\frac{m}{n}=\frac{\text{number of favourable cases}}{\text{total number of cases}}.$$

It requires outcomes to be equally likely (e.g. a fair coin or die).

(b) Empirical (Statistical / a posteriori / relative-frequency) Approach

If an experiment is repeated $N$ times under identical conditions and event $A$ occurs $f$ times, then

$$P(A)=\lim_{N\to\infty}\frac{f}{N}.$$

It is used when outcomes are not equally likely and is based on observation/experiment.

(c) Axiomatic Approach (Kolmogorov)

For a sample space $S$, a probability $P$ assigns to every event $A$ a real number $P(A)$ satisfying:

1. Non-negativity: $P(A)\ge 0$.
2. Certainty: $P(S)=1$.
3. Additivity: for mutually exclusive events $A_1,A_2,\dots$, $P(A_1\cup A_2\cup\cdots)=\sum_i P(A_i)$.

Addition Theorem of Probability (for two events)

Statement: For any two events $A$ and $B$ of a sample space,

$$P(A\cup B)=P(A)+P(B)-P(A\cap B).$$

Proof: From the Venn diagram, $A\cup B$ can be written as the union of mutually exclusive parts:

$$A\cup B=A\cup(B\cap \bar A),$$

where $A$ and $(B\cap\bar A)$ are disjoint. By the additivity axiom,

$$P(A\cup B)=P(A)+P(B\cap\bar A).\quad(1)$$

Also, $B$ can be split into two disjoint parts $(A\cap B)$ and $(B\cap\bar A)$:

$$B=(A\cap B)\cup(B\cap\bar A)\;\Rightarrow\;P(B)=P(A\cap B)+P(B\cap\bar A),$$

so $P(B\cap\bar A)=P(B)-P(A\cap B).\quad(2)$

Substituting (2) in (1):

$$\boxed{P(A\cup B)=P(A)+P(B)-P(A\cap B).}$$

Corollary: If $A$ and $B$ are mutually exclusive, $P(A\cap B)=0$, so $P(A\cup B)=P(A)+P(B)$.

Normal Distribution

Definition

A continuous random variable $X$ is said to follow a normal distribution with mean $\mu$ and variance $\sigma^2$, written $X\sim N(\mu,\sigma^2)$, if its probability density function is

$$f(x)=\frac{1}{\sigma\sqrt{2\pi}}\,e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2},\qquad -\infty<x<\infty,\;\sigma>0.$$

Properties

1. The curve is bell-shaped and symmetrical about $x=\mu$.
2. Mean = Median = Mode = $\mu$.
3. It is unimodal; the maximum ordinate is $\frac{1}{\sigma\sqrt{2\pi}}$ at $x=\mu$.
4. The curve is asymptotic to the $x$-axis on both sides; total area under it is $1$.
5. It is symmetric, so skewness $=0$ and kurtosis $\beta_2=3$ (mesokurtic).
6. Points of inflexion occur at $x=\mu\pm\sigma$.
7. Empirical rule: about $68.27\%$, $95.45\%$ and $99.73\%$ of values lie within $\mu\pm\sigma$, $\mu\pm2\sigma$, $\mu\pm3\sigma$ respectively.
8. Quartiles: $Q_1=\mu-0.6745\sigma$, $Q_3=\mu+0.6745\sigma$; mean deviation $=0.7979\sigma$.

Standard Normal Variate

The variate

$$Z=\frac{X-\mu}{\sigma}$$

follows the standard normal distribution $N(0,1)$ with mean $0$ and variance $1$, and density $\phi(z)=\frac{1}{\sqrt{2\pi}}e^{-z^2/2}$. Areas under it are tabulated, allowing any normal probability to be evaluated.

Worked Problem

The weights of a large group of students are normally distributed with mean $\mu=50$ kg and standard deviation $\sigma=5$ kg. Find $P(45<X<60)$.

Convert to $Z$:

$$z_1=\frac{45-50}{5}=-1,\qquad z_2=\frac{60-50}{5}=2.$$

From standard normal tables, $P(0<Z<1)=0.3413$ and $P(0<Z<2)=0.4772$. By symmetry,

$$P(-1<Z<2)=P(0<Z<1)+P(0<Z<2)=0.3413+0.4772=0.8185.$$

Hence $P(45<X<60)=0.8185$, i.e. about $81.85\%$ of students.

Rank Correlation

Definition

When the actual numerical values of two variables are not available (or attributes can only be ranked, e.g. beauty, intelligence), the degree of association between the two sets of ranks is measured by Spearman's rank correlation coefficient:

$$\rho=1-\frac{6\sum d_i^2}{n(n^2-1)},$$

where $d_i$ is the difference between the two ranks of the $i$-th individual and $n$ is the number of individuals. It lies between $-1$ and $+1$.

Tied Ranks

When two or more items share equal values, each is assigned the average of the ranks they would otherwise occupy. For every group of $m$ tied observations, a correction factor $\dfrac{m(m^2-1)}{12}$ is added to $\sum d^2$:

$$\rho=1-\frac{6\left[\sum d^2+\sum\frac{m(m^2-1)}{12}\right]}{n(n^2-1)}.$$

Worked Example (with a tie)

Marks of 5 students in two subjects:

Ranks of X: $30\to1,\;35\to2,\;40\&40$ occupy ranks 3 and 4 $\to$ each gets $\frac{3+4}{2}=3.5,\;50\to5$. Ranks of Y: $20\to1,22\to2,25\to3,28\to4,30\to5$.

$\sum d^2=0.5$. There is one tie of $m=2$ in X, correction $=\frac{2(2^2-1)}{12}=0.5$.

$$\rho=1-\frac{6(0.5+0.5)}{5(25-1)}=1-\frac{6}{120}=1-0.05=0.95.$$

Hence $\rho=+0.95$, a very high positive rank correlation.

The axiomatic approach to probability, due to A. N. Kolmogorov, defines probability as a real-valued set function $P$ on the events of a sample space $S$ satisfying three axioms:

1. Non-negativity: $P(A)\ge 0$ for every event $A$.
2. Normalization (certainty): $P(S)=1$.
3. Additivity: if $A_1,A_2,\dots$ are pairwise mutually exclusive events, then $P\!\left(\bigcup_i A_i\right)=\sum_i P(A_i)$.

All other probability results (e.g. $P(\bar A)=1-P(A)$, the addition theorem) are derived from these axioms. It is the most general and rigorous definition and applies even when outcomes are not equally likely.

Properties of the Normal Distribution $X\sim N(\mu,\sigma^2)$:

1. The curve is bell-shaped and symmetrical about the mean $x=\mu$.
2. Mean = Median = Mode = $\mu$; it is unimodal.
3. Total area under the curve is $1$, and the curve is asymptotic to the $x$-axis on both sides.
4. Skewness $=0$ and kurtosis $\beta_2=3$ (mesokurtic).
5. Points of inflexion at $x=\mu\pm\sigma$.
6. Empirical rule: $68.27\%$, $95.45\%$ and $99.73\%$ of observations lie within $\mu\pm\sigma$, $\mu\pm2\sigma$ and $\mu\pm3\sigma$ respectively.
7. Quartile deviation $=0.6745\sigma$ and mean deviation $=0.7979\sigma$.
8. Any linear function of a normal variable is also normal.

Rank correlation is a measure of the degree of association (agreement) between two sets of ranks rather than actual numerical values. It is used when the data are qualitative attributes that can only be ranked (e.g. beauty, honesty, intelligence) or when ranks are easier to assign than exact measurements.

It is measured by Spearman's rank correlation coefficient:

$$\rho=1-\frac{6\sum d_i^2}{n(n^2-1)},$$

where $d_i$ is the difference between the ranks of the $i$-th pair and $n$ is the number of pairs. Its value lies between $-1$ (perfect disagreement) and $+1$ (perfect agreement), with $0$ indicating no association.

A standard normal variate $Z$ is a normal variable that has been standardised to have mean $0$ and variance (standard deviation) $1$. If $X\sim N(\mu,\sigma^2)$, then

$$Z=\frac{X-\mu}{\sigma}\sim N(0,1).$$

Its probability density function is

$$\phi(z)=\frac{1}{\sqrt{2\pi}}\,e^{-z^2/2},\qquad -\infty<z<\infty.$$

Standardising allows the areas (probabilities) of any normal distribution to be read from a single standard normal table.

Correlation vs. Regression

In short, correlation tells us whether and how strongly two variables are related, while regression lets us estimate one variable from the other.

Quartile deviation (also called the semi-interquartile range) is a measure of dispersion equal to half the difference between the third quartile $Q_3$ and the first quartile $Q_1$:

$$Q.D.=\frac{Q_3-Q_1}{2}.$$

It measures the spread of the middle $50\%$ of the data and is unaffected by extreme values. The corresponding relative measure is the coefficient of quartile deviation:

$$\text{Coeff. of Q.D.}=\frac{Q_3-Q_1}{Q_3+Q_1}.$$

Addition Theorem of Probability: For any two events $A$ and $B$ of a sample space,

$$P(A\cup B)=P(A)+P(B)-P(A\cap B).$$

It gives the probability that at least one of the two events occurs.

• If $A$ and $B$ are mutually exclusive ($A\cap B=\varnothing$), then $P(A\cap B)=0$ and $P(A\cup B)=P(A)+P(B)$.
• For three events: $P(A\cup B\cup C)=P(A)+P(B)+P(C)-P(A\cap B)-P(B\cap C)-P(A\cap C)+P(A\cap B\cap C).$

The coefficient of skewness is a relative (unit-free) measure of the degree and direction of asymmetry of a distribution. A symmetrical distribution has coefficient $0$; a positive value indicates a right (positive) skew and a negative value a left (negative) skew.

Karl Pearson's coefficient:

$$S_k=\frac{\text{Mean}-\text{Mode}}{\sigma}\qquad\text{or}\qquad S_k=\frac{3(\text{Mean}-\text{Median})}{\sigma},$$

usually lying between $-3$ and $+3$.

Bowley's (quartile) coefficient:

$$S_k=\frac{Q_3+Q_1-2Q_2}{Q_3-Q_1},$$

which lies between $-1$ and $+1$.

Find the variance of $2,4,6,8,10$.

Number of observations $n=5$.

Mean:

$$\bar x=\frac{2+4+6+8+10}{5}=\frac{30}{5}=6.$$

Deviations and squares:

$\sum (x-\bar x)^2 = 16+4+0+4+16 = 40.$

Variance:

$$\sigma^2=\frac{\sum (x-\bar x)^2}{n}=\frac{40}{5}=8.$$

Hence the variance is $\sigma^2=8$ (and standard deviation $\sigma=\sqrt{8}\approx2.83$).