BSc CSIT (TU) Science Statistics II (BSc CSIT, STA210) Question Paper 2078 Nepal

Theory of Estimation

Estimation is the branch of statistical inference concerned with inferring the value of an unknown population parameter (e.g. mean $\mu$, variance $\sigma^2$, proportion $P$) from a sample statistic computed from sample data. The sample statistic used for this purpose is called an estimator, and a particular numerical value it takes is an estimate.

There are two broad approaches:

Point Estimation vs Interval Estimation

Properties of a Good Estimator

An estimator $\hat{\theta}$ of a parameter $\theta$ is considered good if it possesses the following properties:

1. Unbiasedness: $\hat{\theta}$ is unbiased if its expected value equals the parameter, i.e. $E(\hat{\theta})=\theta$. For example, the sample mean $\bar{x}$ is an unbiased estimator of $\mu$.

2. Consistency: $\hat{\theta}$ is consistent if it converges in probability to $\theta$ as the sample size increases, i.e. $\hat{\theta}\to\theta$ as $n\to\infty$. Larger samples give estimates closer to the true value.

3. Efficiency: Among all unbiased estimators, the one with the minimum variance is the most efficient. If $\hat{\theta}_1$ and $\hat{\theta}_2$ are both unbiased and $Var(\hat{\theta}_1)<Var(\hat{\theta}_2)$, then $\hat{\theta}_1$ is more efficient.

4. Sufficiency: An estimator is sufficient if it utilizes all the information in the sample relevant to the parameter, so that no other statistic can add further information about $\theta$.

A good estimator should ideally be unbiased, consistent, efficient and sufficient.

The Chi-Square ($\chi^2$) Test

The chi-square test is a non-parametric test based on the chi-square distribution, used to test hypotheses about categorical (attribute) data by comparing observed frequencies $(O)$ with expected frequencies $(E)$.

The test statistic is:

which follows a $\chi^2$ distribution with appropriate degrees of freedom. A large value of $\chi^2$ indicates a large discrepancy between observed and expected frequencies, leading to rejection of $H_0$.

Conditions: observations independent, sample reasonably large, total frequency $N>50$, and each expected frequency $\geq 5$ (otherwise pool cells).

Application 1: Test of Goodness of Fit

Used to test whether an observed frequency distribution fits a theoretical/expected distribution (e.g. uniform, Binomial, Poisson, Normal).

• $H_0$: The observed data agree with the assumed theoretical distribution.
• $H_1$: The observed data do not fit the assumed distribution.
• Compute expected frequencies from the theoretical distribution.
• Statistic: $\chi^2 = \sum \dfrac{(O-E)^2}{E}$.
• Degrees of freedom $= n - 1 - k$, where $n$ = number of classes and $k$ = number of parameters estimated from the data.
• If $\chi^2_{cal} > \chi^2_{tab}$ at level $\alpha$, reject $H_0$ (poor fit).

Application 2: Test of Independence of Attributes

Used with a contingency table to test whether two attributes (e.g. gender and preference) are independent or associated.

• $H_0$: The two attributes are independent.
• $H_1$: The two attributes are associated (dependent).
• For an $r \times c$ table, expected frequency of a cell:

• Statistic: $\chi^2 = \sum \dfrac{(O-E)^2}{E}$.
• Degrees of freedom $= (r-1)(c-1)$.
• If $\chi^2_{cal} > \chi^2_{tab}$, reject $H_0$ and conclude the attributes are associated.

Other Uses

• Test for a specified population variance $\sigma^2$.
• Test of homogeneity (whether several populations have the same distribution).

Probability Distribution

A probability distribution is a description that assigns a probability to each possible value of a random variable. For a discrete random variable $X$ taking values $x_1, x_2, \dots$, the function $P(X=x_i)=p_i$ is the probability mass function, satisfying:

For a continuous random variable, it is described by a probability density function $f(x)$ with $f(x)\geq 0$ and $\int_{-\infty}^{\infty} f(x)\,dx = 1$.

Binomial Distribution

A discrete random variable $X$ follows a binomial distribution if it represents the number of successes in $n$ independent trials, each with probability of success $p$ (and failure $q = 1-p$). Its probability mass function is:

where $\binom{n}{x} = \dfrac{n!}{x!(n-x)!}$.

Mean and Variance

• Mean: $E(X) = np$
• Variance: $Var(X) = npq$
• Standard deviation: $\sqrt{npq}$

Since $q<1$, the variance $npq$ is always less than the mean $np$.

Conditions (Assumptions) for Application

The binomial distribution applies when:

1. The experiment consists of a fixed number $n$ of trials.
2. Each trial has only two mutually exclusive outcomes — success and failure.
3. The trials are independent of one another.
4. The probability of success $p$ remains constant from trial to trial.

Example

Tossing a fair coin 5 times and counting the number of heads, where $n=5$, $p=\tfrac12$.

Confidence Interval for a Population Mean

A confidence interval (CI) for the population mean $\mu$ is a range of values, computed from sample data, that is expected to contain $\mu$ with a stated confidence level $(1-\alpha)$, e.g. 95%.

Case 1: $\sigma$ known (or large sample, $n>30$) — use Z

Case 2: $\sigma$ unknown and small sample ($n \leq 30$) — use t

where $s$ is the sample standard deviation and degrees of freedom $= n-1$.

Steps to Construct

1. Compute the sample mean $\bar{x}$ (and $s$ if $\sigma$ unknown).
2. Choose the confidence level $(1-\alpha)$ and find the critical value $Z_{\alpha/2}$ (e.g. $1.96$ for 95%) or $t_{\alpha/2,n-1}$.
3. Compute the standard error $\dfrac{\sigma}{\sqrt n}$ (or $\dfrac{s}{\sqrt n}$).
4. Compute the margin of error $E = Z_{\alpha/2}\dfrac{\sigma}{\sqrt n}$.
5. The interval is $(\bar{x}-E,\ \bar{x}+E)$.

Interpretation

For a 95% CI, if many samples were drawn and an interval computed for each, about 95% of those intervals would contain the true mean $\mu$.

F-Test for Equality of Two Population Variances

The F-test is used to test whether two independent normal populations have equal variances. It is based on the ratio of two independent sample variances, which follows the F-distribution.

Hypotheses

• $H_0: \sigma_1^2 = \sigma_2^2$ (the two population variances are equal)
• $H_1: \sigma_1^2 \neq \sigma_2^2$ (they are unequal)

Test Statistic

From two independent random samples of sizes $n_1$ and $n_2$ with sample variances $s_1^2$ and $s_2^2$:

The larger variance is placed in the numerator so that $F \geq 1$. The unbiased sample variance is

Degrees of Freedom

$\nu_1 = n_1 - 1$ (numerator), $\nu_2 = n_2 - 1$ (denominator).

Decision Rule

Compare $F_{cal}$ with the tabulated $F_{\alpha,(\nu_1,\nu_2)}$:

• If $F_{cal} \leq F_{tab}$, accept $H_0$ — variances are equal.
• If $F_{cal} > F_{tab}$, reject $H_0$ — variances differ significantly.

Assumptions

Both populations are normal, and the two samples are independent and drawn randomly.

Index Numbers

An index number is a statistical measure that expresses the relative change in a variable or a group of related variables (such as prices, quantities, or value) over time, place, or other characteristic, with respect to a chosen base period (taken as 100). It is often called an economic barometer.

A price index measures the average change in the prices of a basket of commodities between the base period (0) and the current period (1).

Laspeyres' Price Index

Uses base-year quantities $(q_0)$ as weights:

It answers: what is the cost now of the base-year basket compared with its base-year cost? It tends to overstate price rises because it ignores substitution away from goods that became dearer.

Paasche's Price Index

Uses current-year quantities $(q_1)$ as weights:

It answers: what would the current basket have cost in the base year versus now? It tends to understate price rises.

Comparison

Fisher's ideal index is the geometric mean of the two: $P_{01}^{F} = \sqrt{P^{L} \times P^{P}}$.

Addition Theorem of Probability

Gives the probability of the union of events (occurrence of at least one event).

For any two events $A$ and $B$:

If $A$ and $B$ are mutually exclusive (cannot occur together, $A\cap B=\varnothing$):

Example: A card is drawn from 52 cards. $P(\text{King}) = \tfrac{4}{52}$, $P(\text{Heart}) = \tfrac{13}{52}$, $P(\text{King of Hearts}) = \tfrac{1}{52}$.

Multiplication Theorem of Probability

Gives the probability of the joint occurrence (intersection) of events.

For any two events $A$ and $B$:

where $P(B\mid A)$ is the conditional probability of $B$ given $A$.

If $A$ and $B$ are independent (occurrence of one does not affect the other):

Example: Two cards are drawn one after another without replacement.

If drawn with replacement (independent): $\tfrac{4}{52}\times\tfrac{4}{52}=\tfrac{1}{169}$.

Poisson Distribution

The Poisson distribution is a discrete probability distribution that gives the probability of a given number of independent events occurring in a fixed interval of time, space, area, or volume, when these events occur at a constant average rate $\lambda$. It is the limiting case of the binomial distribution when $n \to \infty$, $p \to 0$, with $np = \lambda$ finite.

The probability mass function is:

where $\lambda > 0$ is the average number of occurrences and $e \approx 2.718$.

Mean and Variance

A characteristic property is that the mean and variance are equal:

Conditions

• Events occur independently.
• The average rate $\lambda$ is constant.
• $n$ is large, $p$ is small (rare events).

Applications

Used to model the number of rare events, such as:

1. Number of telephone calls received at an exchange per minute.
2. Number of printing/typing errors per page of a book.
3. Number of accidents on a highway per day.
4. Number of defective items in a large batch.
5. Number of customers/packets arriving at a server per unit time (queueing/network traffic).

Random Variable

A random variable is a real-valued function that assigns a numerical value to each outcome (sample point) of a random experiment. It maps the sample space $S$ to the set of real numbers, i.e. $X: S \to \mathbb{R}$.

Example: In tossing two coins, if $X$ = number of heads, then $X$ takes values $0, 1, 2$.

Discrete vs Continuous Random Variables

Examples

• Discrete: Number of students present in a class; number of calls per hour.
• Continuous: The height of a person (e.g. 165.3 cm); the time taken to run a race.

Mathematical Expectation

The mathematical expectation (or expected value) of a random variable is the long-run average value it takes, weighted by probabilities. It is a measure of the central tendency of a probability distribution.

For a discrete random variable $X$ with p.m.f. $P(X=x_i)=p_i$:

For a continuous random variable with density $f(x)$:

Properties (with Proof)

1. Expectation of a constant: $E(c) = c$. Proof: $E(c)=\sum c\,p_i = c\sum p_i = c\cdot 1 = c.$

2. Constant multiplier: $E(cX) = c\,E(X)$. Proof: $E(cX)=\sum c x_i p_i = c\sum x_i p_i = c\,E(X).$

3. Addition (linearity): $E(X + Y) = E(X) + E(Y)$, for any random variables $X,Y$. Proof (discrete): $E(X+Y)=\sum_x\sum_y (x+y)P(x,y) = \sum_x\sum_y xP(x,y) + \sum_x\sum_y yP(x,y) = E(X)+E(Y).$

4. Linear combination: $E(aX + b) = a\,E(X) + b$ (from properties 1–3).

5. Multiplication for independent variables: If $X$ and $Y$ are independent, then $E(XY) = E(X)\,E(Y)$. Proof: For independence $P(x,y)=P(x)P(y)$, so $E(XY)=\sum_x\sum_y xy\,P(x)P(y) = \big(\sum_x xP(x)\big)\big(\sum_y yP(y)\big) = E(X)E(Y).$

t-Test for Difference Between Two Sample Means

This test checks whether the means of two independent small samples ($n_1, n_2 \le 30$) drawn from two normal populations differ significantly, when the population variances are unknown but assumed equal.

Hypotheses

• $H_0: \mu_1 = \mu_2$ (no significant difference in population means)
• $H_1: \mu_1 \neq \mu_2$ (two-tailed)

Test Statistic

where the pooled standard deviation $S$ is

Degrees of Freedom

Decision Rule

Compare $|t_{cal}|$ with the tabulated value $t_{\alpha/2,\,\nu}$:

• If $|t_{cal}| \leq t_{tab}$, accept $H_0$ — means do not differ significantly.
• If $|t_{cal}| > t_{tab}$, reject $H_0$ — the difference is significant.

Assumptions

The two samples are independent and drawn from normal populations with equal (but unknown) variances.

Z-Test for a Single Mean (Large Sample)

For a large sample ($n > 30$), the Z-test is used to test whether the sample mean $\bar{x}$ differs significantly from a specified population mean $\mu_0$. By the Central Limit Theorem, $\bar{x}$ is approximately normally distributed.

Hypotheses

• $H_0: \mu = \mu_0$
• $H_1: \mu \neq \mu_0$ (two-tailed)

Test Statistic

If the population standard deviation $\sigma$ is unknown, the sample standard deviation $s$ is used (valid for large $n$).

Decision Rule (5% level)

Compare $|Z_{cal}|$ with the critical value $Z_{\alpha/2} = 1.96$:

• If $|Z_{cal}| \leq 1.96$, accept $H_0$.
• If $|Z_{cal}| > 1.96$, reject $H_0$.

Example

A sample of $n = 100$ items has mean $\bar{x} = 52$ with $\sigma = 10$. Test whether the population mean is $\mu_0 = 50$ at the 5% level.

Since $|Z| = 2.0 > 1.96$, we reject $H_0$ and conclude that the population mean differs significantly from 50.