BSc CSIT (TU) Science B.Sc. II Year Statistics (STA201) (Model) Question Paper 2075 Nepal

Definition. The negative binomial distribution gives the probability that the $r^{th}$ success occurs on the $(r+k)^{th}$ trial in a sequence of independent Bernoulli trials with success probability $p$ (failure $q = 1-p$). If $X$ is the number of failures before the $r^{th}$ success, its p.m.f. is

Mean and variance (via the m.g.f.). The moment generating function is

Differentiating and evaluating at $t = 0$:

Using $E(X^2) = M_X''(0)$,

Thus Mean $= \dfrac{rq}{p}$ and Variance $= \dfrac{rq}{p^{2}}$. Note that variance $>$ mean (over-dispersion), since $1/p > 1$.

Gamma distribution. A continuous random variable $X$ follows a gamma distribution with shape parameter $\alpha > 0$ (and scale 1) if its p.d.f. is

Its mean is $E(X) = \alpha$ and variance $\operatorname{Var}(X) = \alpha$.

Normal limit as $\alpha \to \infty$. Standardise $X$ by

The m.g.f. of $X$ is $M_X(t) = (1-t)^{-\alpha}$ for $t < 1$. Then

Taking logarithms and expanding $\log(1 - t/\sqrt{\alpha})$ in a Taylor series:

This simplifies to

Hence $M_Z(t) \to e^{t^{2}/2}$, which is the m.g.f. of the standard normal distribution. Therefore, for large $\alpha$, the gamma distribution tends to $N(\alpha, \alpha)$.

Bivariate random variable. Let $(X, Y)$ be a pair of jointly distributed random variables.

Joint distribution function (c.d.f.):

Joint probability density function (continuous case): If $F(x,y)$ is differentiable, the joint p.d.f. is

Equivalently, $F(x,y) = \int_{-\infty}^{x}\int_{-\infty}^{y} f(u,v)\,dv\,du.$

Properties of the bivariate distribution function $F(x,y)$:

1. $0 \le F(x, y) \le 1$.
2. $F$ is monotonically non-decreasing in each argument.
3. $F(-\infty, y) = F(x, -\infty) = F(-\infty, -\infty) = 0$.
4. $F(+\infty, +\infty) = 1$.
5. The marginal distribution functions are $F_X(x) = F(x, +\infty)$ and $F_Y(y) = F(+\infty, y)$.
6. For $a_1 < a_2$ and $b_1 < b_2$:

7. $F(x,y)$ is right-continuous in each variable.

Derivation of Student's $t$-distribution. Let $Z \sim N(0,1)$ and $\chi^{2}$ be a chi-square variate with $n$ degrees of freedom, independent of $Z$. Define

Using the joint density of $Z$ and $\chi^{2}$ and the transformation, integrating out the chi-square variable gives the p.d.f. of $t$:

This is the Student's $t$-distribution with $n$ degrees of freedom.

Relation between $F$ and $t$ when $m = 1$. The $F$ statistic with $(m, n)$ degrees of freedom is

For $m = 1$, $\chi_1^{2} = Z^{2}$ where $Z \sim N(0,1)$. Hence

Therefore $F(1, n) = t_n^{2}$; that is, the square of a $t$-variate with $n$ d.f. follows an $F$ distribution with $(1, n)$ d.f. Equivalently, $t_n = \sqrt{F(1, n)}$.

Method of maximum likelihood estimation (MLE). Let $x_1, x_2, \dots, x_n$ be a random sample from a population with density $f(x; \theta)$. The likelihood function is

The maximum likelihood estimate $\hat{\theta}$ is the value of $\theta$ that maximises $L(\theta)$ (equivalently $\log L(\theta)$). It is obtained by solving

Properties of MLEs:

1. Consistency — MLEs are consistent: $\hat{\theta} \to \theta$ in probability as $n \to \infty$.
2. Asymptotic normality — for large $n$, $\hat{\theta}$ is approximately normally distributed with mean $\theta$ and variance equal to the Cramer-Rao lower bound.
3. Asymptotic efficiency — MLEs attain the minimum possible variance asymptotically.
4. Sufficiency — if a sufficient statistic exists, the MLE is a function of it.
5. Invariance — if $\hat{\theta}$ is the MLE of $\theta$, then $g(\hat{\theta})$ is the MLE of $g(\theta)$.
6. Not always unbiased — MLEs may be biased for small samples, though the bias vanishes as $n \to \infty$.

Parametric vs non-parametric tests:

One-sample run test (test of randomness). A run is a sequence of identical symbols bounded by different symbols (or boundaries). The run test checks whether a sequence of two types of outcomes occurs in a random order.

Procedure:

1. Arrange the observations in the order obtained and classify each into one of two categories (e.g. above/below the median, denoted $+$ and $-$).
2. Let $n_1$ = number of $+$ symbols, $n_2$ = number of $-$ symbols, and $R$ = total number of runs.
3. Hypotheses: $H_0$: the sequence is random; $H_1$: the sequence is not random.
4. For large samples, under $H_0$, $R$ is approximately normal with

5. Compute the test statistic $Z = \dfrac{R - E(R)}{\sqrt{\operatorname{Var}(R)}}$ and compare with the critical value (e.g. $\pm 1.96$ at 5%). Reject $H_0$ if $|Z|$ exceeds the critical value. For small samples, use the run-test tables.

Example: Suppose the sequence of defective (D) and non-defective (N) items is: N N D N D D N N D N. Here $n_1 = 6$ (N), $n_2 = 4$ (D), and the runs are NN | D | N | DD | NN | D | N giving $R = 7$. Compute $E(R) = \frac{2(6)(4)}{10} + 1 = 5.8$ and the variance, then form $Z$ and compare with $1.96$. Since $Z$ is small, we do not reject $H_0$ and conclude the order is random.

The negative exponential distribution has p.d.f.

Moment generating function:

Mean: $E(X) = M_X'(0) = \dfrac{\theta}{(\theta - t)^2}\Big|_{t=0} = \dfrac{1}{\theta}.$

Second moment: $E(X^2) = M_X''(0) = \dfrac{2\theta}{(\theta - t)^3}\Big|_{t=0} = \dfrac{2}{\theta^{2}}.$

Variance: $\operatorname{Var}(X) = E(X^2) - [E(X)]^2 = \dfrac{2}{\theta^{2}} - \dfrac{1}{\theta^{2}} = \dfrac{1}{\theta^{2}}.$

For a beta distribution of the first kind with parameters $m$ and $n$:

• Mean: $\dfrac{m}{m+n} = \dfrac{3}{3+4} = \dfrac{3}{7} \approx 0.4286.$
• Mode: $\dfrac{m-1}{m+n-2} = \dfrac{3-1}{3+4-2} = \dfrac{2}{5} = 0.40$ (valid since $m, n > 1$).
• Variance: $\dfrac{mn}{(m+n)^{2}(m+n+1)} = \dfrac{3 \times 4}{(7)^{2}\,(8)} = \dfrac{12}{49 \times 8} = \dfrac{12}{392} = \dfrac{3}{98} \approx 0.0306.$

Let $U = X_1 X_2$ where $X_1, X_2 \sim U(0,1)$ are independent, so the joint density is $f(x_1, x_2) = 1$ on the unit square.

For $0 < u < 1$, the c.d.f. is

For $x_1 \le u$, $u/x_1 \ge 1$ so the inner probability is $1$; for $x_1 > u$ it equals $u/x_1$. Hence

Differentiating gives the p.d.f. of $U$:

Thus $X_1 X_2$ has p.d.f. $g(u) = -\ln u$ for $0 < u < 1$ (and $0$ otherwise).

Marginal of $X$:

Marginal of $Y$:

Check independence:

Since the joint density factorises into the product of the marginal densities for all $(x, y)$, the random variables $X$ and $Y$ are independent.

Likelihood function. For a random sample $x_1, x_2, \dots, x_n$ from a population with density $f(x; \theta)$, the likelihood function is the joint density viewed as a function of the parameter $\theta$:

Properties:

1. Non-negativity and integration: $L(\theta) \ge 0$, and as a density in the sample it integrates to 1 over the sample space:

2. Score has zero expectation. Differentiating the identity $\int L\, dx = 1$ with respect to $\theta$ (under regularity conditions allowing interchange of integral and derivative):

3. Information identity. Differentiating again gives

the Fisher information, which is positive.

These properties form the basis for maximum likelihood estimation and the Cramer-Rao bound.

For a random sample $x_1, \dots, x_n$ from $N(\mu, \sigma^2)$ (with $\sigma^2$ known), the log-likelihood is

Differentiating with respect to $\mu$:

This is of the form $\dfrac{\partial \log L}{\partial \mu} = A(\mu)\,(T - \mu)$ with $T = \bar{x}$ and $A(\mu) = n/\sigma^2$. By the Cramer-Rao theory, an MVB estimator exists because the score factorises in this form, and the MVB estimator of $\mu$ is the sample mean $\bar{x}$.

Minimum variance bound (Cramer-Rao lower bound):

Thus the MVB estimator of $\mu$ is $\bar{x}$ with minimum variance $\dfrac{\sigma^{2}}{n}$.

Interval estimation. Interval estimation specifies a random interval $(T_1, T_2)$, computed from the sample, that contains the unknown parameter $\theta$ with a stated probability (confidence level) $1 - \alpha$:

The interval $(T_1, T_2)$ is called a confidence interval and $1-\alpha$ the confidence coefficient.

Proof that $T^2$ is biased for $\theta^2$. Since $T$ is unbiased for $\theta$, $E(T) = \theta$. By the definition of variance,

Unless $\operatorname{Var}(T) = 0$, we have

Hence $T^2$ overestimates $\theta^2$ by an amount equal to $\operatorname{Var}(T) > 0$, so $T^2$ is a biased (positively biased) estimator of $\theta^2$.

Statement. To test a simple null hypothesis $H_0: \theta = \theta_0$ against a simple alternative $H_1: \theta = \theta_1$, the most powerful (MP) critical region $w$ of size $\alpha$ is given by

where $k > 0$ is chosen so that $P(x \in w \mid H_0) = \alpha$.

Proof. Let $w$ be the region defined above and $w^{*}$ be any other region of size $\alpha$, so $\int_w L_0\,dx = \int_{w^*} L_0\,dx = \alpha$. The power of $w$ is $\int_w L_1\,dx$. Consider

Inside $w$, $L_1 \ge k L_0$; outside $w$, $L_1 < k L_0$. Therefore

Hence the difference $\ge k\big(\int_{w\setminus w^*} L_0\,dx - \int_{w^*\setminus w} L_0\,dx\big) = k(\alpha - \alpha) = 0.$ Thus the power of $w$ is at least that of $w^*$, proving $w$ is most powerful.

Applications:

1. Construction of most powerful tests for simple hypotheses.
2. Basis for likelihood ratio tests and uniformly most powerful (UMP) tests.
3. Used to derive optimal critical regions for normal, binomial, Poisson, etc.

Let the critical region be $w: x \ge 1$ and the density be $f(x, \theta) = \theta e^{-\theta x}$, $x \ge 0$.

Type I error (rejecting $H_0$ when $H_0: \theta = 2$ is true):

Type II error (accepting $H_0$ when $H_1: \theta = 1$ is true), i.e. $x < 1$ under $\theta = 1$:

Thus the Type I error $\alpha = e^{-2} \approx 0.135$ and the Type II error $\beta = 1 - e^{-1} \approx 0.632$.

Step 1 — Rank all observations (combined, smallest = rank 1):

($T$ = trained, $U$ = untrained.)

Step 2 — Rank sums:

• Trained ($n_1 = 5$): ranks 1, 4, 6, 7, 8 → $R_1 = 26$.
• Untrained ($n_2 = 4$): ranks 2, 3, 5, 9 → $R_2 = 19$.

Step 3 — Compute U statistics:

Check: $U_1 + U_2 = 20 = n_1 n_2$. Take $U = \min(U_1, U_2) = 9$.

Step 4 — Decision. For $n_1 = 5$, $n_2 = 4$ at the 5% level (two-tailed), the critical value of $U$ is $1$. Since the calculated $U = 9 > 1$, we do not reject $H_0$.

Conclusion: There is no significant difference between the average number of trials of trained and untrained rats.

If $p$ is the population proportion and $q = 1 - p$, then for a random sample of size $n$ the sample proportion $\hat{p}$ has standard error

When $p$ is unknown it is estimated by $\hat{p}$, giving $S.E.(\hat{p}) = \sqrt{\dfrac{\hat{p}(1-\hat{p})}{n}}.$

One-tailed test: The critical (rejection) region lies entirely in one tail of the sampling distribution. It is used when the alternative hypothesis is directional, e.g. $H_1: \mu > \mu_0$ (right-tailed) or $H_1: \mu < \mu_0$ (left-tailed).

Two-tailed test: The critical region is split between both tails of the distribution. It is used when the alternative is non-directional, e.g. $H_1: \mu \ne \mu_0$, so that significantly large or small values lead to rejection of $H_0$.

A two-dimensional (bivariate) random variable assigns a pair of real numbers to each outcome of a random experiment.

Example: Select a student at random and record $(X, Y)$ where $X$ = the student's height and $Y$ = the student's weight. Each outcome gives an ordered pair $(x, y)$, so $(X, Y)$ is a two-dimensional random variable. (Another example: tossing two dice and recording the pair of numbers $(X, Y)$ that appear.)

Given $\sigma = 2.4$ kg, $n = 10$, $\bar{x} = 31.4$ kg. Since $\sigma$ is known, use the $z$ value $1.96$ for 95% confidence.

Standard error: $\dfrac{\sigma}{\sqrt{n}} = \dfrac{2.4}{\sqrt{10}} = \dfrac{2.4}{3.162} = 0.759$ kg.

95% confidence limits:

Thus the limits are $(29.91, 32.89)$ kg approximately, i.e. the 95% confidence interval for the population mean is $29.91 \text{ kg} < \mu < 32.89 \text{ kg}$.

Four main features of the $F$-distribution curve:

1. It is a continuous distribution defined only for non-negative values ($F \ge 0$).
2. It is positively skewed (skewed to the right), the skewness decreasing as the degrees of freedom increase.
3. Its shape depends on two parameters — the numerator and denominator degrees of freedom $(m, n)$.
4. The total area under the curve is 1, and it is unimodal; as both degrees of freedom become large it approaches the normal curve.

For a hypergeometric distribution with population size $N$, $M$ successes in the population, and sample size $n$:

Mean: $E(X) = \dfrac{nM}{N} = np$, where $p = M/N$.

Variance: $\operatorname{Var}(X) = n\,\dfrac{M}{N}\left(1 - \dfrac{M}{N}\right)\dfrac{N - n}{N - 1} = npq\left(\dfrac{N - n}{N - 1}\right),$

where $q = 1 - p$ and $\dfrac{N-n}{N-1}$ is the finite population correction factor.

For the chi-square distribution with $n$ degrees of freedom, the central moments $\mu_r$ satisfy the recurrence relation

with $\mu_0 = 1$ and $\mu_1 = 0$. Using this, $\mu_2 = 2n$, $\mu_3 = 8n$, and $\mu_4 = 48n + 12n^2$. (The raw moments satisfy $\mu_r' = n(n+2)(n+4)\cdots(n+2r-2)$.)

Cramer-Rao Inequality. If $T$ is an unbiased estimator of a parameter $\theta$ based on a random sample, then under regularity conditions the variance of $T$ cannot be smaller than the reciprocal of the Fisher information:

The right-hand side is the minimum variance bound (MVB). An estimator attaining this bound is the most efficient (MVB) estimator.

The four characteristics (properties) of a good estimator are:

1. Unbiasedness — $E(\hat{\theta}) = \theta$; on average the estimator equals the parameter.
2. Consistency — $\hat{\theta} \to \theta$ in probability as the sample size $n \to \infty$.
3. Efficiency — it has the smallest variance among all unbiased estimators (minimum variance).
4. Sufficiency — it utilises all the information in the sample relevant to the parameter.

The Cauchy distribution does not possess a moment generating function, because the defining integral $\int_{-\infty}^{\infty} e^{tx} f(x)\,dx$ diverges for every $t \ne 0$ (its moments, including the mean, do not exist).

Instead, the Cauchy distribution is characterised by its characteristic function. For the standard Cauchy distribution with p.d.f. $f(x) = \dfrac{1}{\pi(1 + x^2)}$,

For a general Cauchy with location $\mu$ and scale $\lambda$: $\phi(t) = e^{i\mu t - \lambda |t|}.$