Theory

📊 Statistics 1: The Master Encyclopedia (V5)

Welcome to the ultimate guide for Statistics 1. This document is designed to be the “goated” repository for every concept, formula, and nuance in the course.

---

📔 Volume I: The Genesis of Data & Descriptive Foundations

---

🔬 Week 1: The Anatomy of Statistical Inquiry

1.1 The Epistemology of Statistics

Statistics is the mathematical framework for decision-making under uncertainty. It translates raw observation into actionable intelligence.

• Descriptive Statistics: Summarizing and visualizing the data we have on hand.
• Inferential Statistics: Making predictions or generalizations about a Population based on a Sample.

Key Definitions:

• Population ($N$): The entire collection of interest.
• Sample ($n$): The observed subset.
• Parameter: A characteristic of the population (e.g., Population Mean $\mu$).
• Statistic: A characteristic of the sample (e.g., Sample Mean $\bar{x}$).

1.2 The Taxonomy of Variables

Type	Sub-type	Definition	Example
Categorical	Nominal	Identity only, no order	Blood type, Gender
	Ordinal	Meaningful order, distance unknown	Ranks, Likert scale
Numerical	Discrete	Countable gaps (integers)	No. of children
	Continuous	Infinite values in a range	Height, Weight

1.3 The Scale Hierarchy (S.S. Stevens)

1. Nominal: Identity ($=,\ne$).
2. Ordinal: Ranking ($>,<$).
3. Interval: Meaningful differences ($+,-$). No “True Zero” (e.g., $0^{o}C$).
4. Ratio: True Zero exists ($\times ,÷$). Zero means absence of property (e.g., $0$ kg).

CAUTION

The “True Zero” Trap: Always ask: “If this is 0, does it mean the thing doesn’t exist?” If yes, it’s Ratio. If no, it’s likely Interval.

---

---

🎨 Week 2: Graphical Representations & Frequency Geometry

2.1 Frequency Distributions

Before visualizing, we must structure raw data into Frequency Tables.

• Relative Frequency: $\frac{f_i}{N}$ (Proportion of the total).
• Percent Frequency: Relative Frequency $\times 100$.

2.2 Visualizing Categorical Data

The goal is to show the distribution of observations across categories.

1. The Bar Chart

• Geometry: Rectangular bars where $Height\propto Frequency$.
• Best Use: Comparing specific values across categories.
• Variation: Pareto Chart. A bar chart sorted in descending order of frequency ($80/20$ rule application).

2. The Pie Chart

• Geometry: Circular sectors where $Angle\propto RelativeFrequency$.
• Formula: Sector Angle $\theta =360^o\times RelativeFrequency$.
• Best Use: Highlighting the “part-to-whole” relationship.

2.3 The “Golden Rule” of Visuals: The Area Principle

The visual area of a graphical element MUST be proportional to the value it represents.

• Violation: 3D effects, varying thicknesses, or uneven widths in bar charts. These are designed to mislead and are “Cardinal Sins” in Statistics.

2.4 Case Study: Placement Analysis

• Sector: Software (200), Core (150), Analytics (100), Other (50).
• Total ($N$): 500.
• Software RF: $200/500=0.4$.
• Software Angle: $0.4\times 360^o=144^o$.
• Pareto Logic: Software would be the first bar, followed by Core, then Analytics.

---

📐 Volume II: Measures of Centrality & Linear Relationships

📏 Week 3: The Mechanics of Centrality & Dispersion

Numerical data requires summary metrics that capture where the data “centers” and how much it “leaks” (spreads).

3.1 Measures of Central Tendency

1. Arithmetic Mean ($\bar{x}$):
   • $\bar{x}=\frac{\sum x_i}{n}$.
   • Philosophical Insight: The mean is the “Balance Point” of the data. If you placed weights on a beam at the data points, the mean is where it would balance perfectly.
   • Sensitivity: Extremely sensitive to outliers. One massive value pulls the mean toward it.
2. Median ($M$):
   • The middle value of a sorted dataset.
   • If $n$ is odd: $\text{Value at rank }\frac{n+1}{2}$.
   • If $n$ is even: Average of values at ranks $\frac{n}{2}$ and $\frac{n}{2}+1$.
   • Robustness: The median is a “Robust Statistic.” It ignores outliers.
3. Mode:
   • The most frequent observation.
   • The only measure valid for Nominal data.

3.2 Measures of Dispersion (Spread)

1. Variance ($s^2$ or ${\sigma }^2$):
   • Average squared deviation from the mean.
   • Sample Variance: $s^2=\frac{\sum (x_i-\bar{x}{)}^2}{n-1}$.
   • Why $n-1$? (Bessel’s Correction): Using $n$ would underestimate the true population variance. $n-1$ makes the sample variance an “Unbiased Estimator.”
2. Standard Deviation ($s$ or $\sigma$):
   • $\sqrt{Variance}$. Brings the units back to the original scale.
3. Coefficient of Variation (CV):
   • $CV=\left(\frac{s}{\bar{x}}\right)\cdot 100\%$.
   • Use: Comparing spread across different units (e.g., Variability of height in cm vs weight in kg).

3.3 Percentiles & The Anatomy of a Box Plot

• Percentiles ($P_k$): Value such that $k\%$ of data is at or below it.
• Quartiles: $Q_1=P_{25},Q_2=Median=P_{50},Q_3=P_{75}$.
• IQR (Interquartile Range): $Q_3-Q_1$.
• Outlier Detection (Tukey’s Fences):
  • Lower Fence: $Q_1-1.5\cdot IQR$
  • Upper Fence: $Q_3+1.5\cdot IQR$
  • Values outside these fences are statistically suspected outliers.

---

🔗 Week 4: Bivariate Data (Relationships & Trends)

When we observe two variables $(X,Y)$ for the same case, we search for Correlation—the degree to which they move together.

4.1 Scatter Plots & Visual Trends

• Positive Trend: As $X$ increases, $Y$ tends to increase.
• Negative Trend: As $X$ increases, $Y$ tends to decrease.
• No Trend: Points are scattered randomly with no discernible line.

4.2 Covariance & Pearson’s $r$

1. Covariance ($Cov(X,Y)$):
   • Measures the direction of the relationship.
   • $Cov(X,Y)=\frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{n-1}$.
   • If positive, they move together. If negative, they move oppositely.
2. Correlation Coefficient ($r$):
   • Normalizes covariance to a range of $[-1,1]$.
   • $r=\frac{Cov(X,Y)}{s_x{s}_y}$.
   • Strength:
     • $|r|\approx 1$: Strong linear relationship.
     • $|r|\approx 0.5$: Moderate linear relationship.
     • $|r|\approx 0$: No linear relationship (they might still have a curved relationship!).

4.3 Linear Transformations & Correlation

This is a high-frequency exam topic. If we transform $X\to aX+b$ and $Y\to cY+d$:

• Covariance Result: $Cov(aX+b,cY+d)=ac\cdot Cov(X,Y)$.
• Correlation Result:
  • $r_{new}=r_{old}$ if $a$ and $c$ have the same sign.
  • $r_{new}=-r_{old}$ if $a$ and $c$ have opposite signs.
• Invariance: Adding/Subtracting constants ($b,d$) has zero effect on $r$.

4.4 Scatter Plot Interpretation Strategy

• Step 1: Identify the independent $(X)$ and dependent $(Y)$ variables.
• Step 2: Look for the overall “cloud” shape.
• Step 3: Is it linear? (If it’s a parabola, $r$ might be 0 even if there’s a clear relationship).
• Step 4: Outliers? A single outlier far from the line can tank a high correlation.

---

🎲 Volume III: The Logic of Chance (Combinatorics & Probability)

🔢 Week 5: Countable Assemblies (Combinatorics)

Counting is the backbone of discrete probability. We must master the art of determining the size of the Sample Space ($|S|$) and the Event Space ($|E|$).

5.1 The Multiplication Principle (The “AND” Rule)

If task A can be done in $m$ ways and task B in $n$ ways, doing A AND B takes $m\times n$ ways.

• Example: 3 shirts and 4 pants $⟹12$ outfits.

5.2 Permutations (Order Matters)

Use when the sequence is distinct (e.g., gold/silver/bronze medals).

1. Basic Permutation: $P(n,r)=\frac{n!}{(n-r)!}$.
2. Identical Items (The Anagram Rule): If there are $p$ items of one kind, $q$ of another, total ways to arrange $n$ items:
   • $N=\frac{n!}{p!q!\dots }$
   • Example: “STATISTICS” (10 letters: 3 S, 3 T, 2 I). Ways $=\frac{10!}{3!3!2!}$.
3. Circular Permutations: If $n$ items are in a circle, ways $=(n-1)!$.

5.3 Combinations (Order Doesn’t Matter)

Use when only the selection counts (e.g., a committee of 3).

• Formula: $\left(\genfrac{}{}{0ex}{}{n}{r}\right)=\frac{n!}{r!(n-r)!}$.
• Symmetry: $\left(\genfrac{}{}{0ex}{}{n}{r}\right)=\left(\genfrac{}{}{0ex}{}{n}{n-r}\right)$ (Choosing $r$ to take is the same as choosing $n-r$ to leave).

5.4 The “Bundle & Gap” Strategy

• Bundle Method: When items must be together. Bundle them as 1 item, arrange the bundle internal elements, then arrange the new set.
• Gap Method: When items must NOT be together. Arrange the other items first, then place the restricted items in the “gaps” between them.

---

🌩️ Weeks 6-8: The Calculus of Uncertainty (Probability)

6.1 The Axiomatic Foundation

Probability is a measure of belief or frequency assigned to an event $E\subset S$.

• Axiom 1: $0\le P(E)\le 1$.
• Axiom 2: $P(S)=1$.
• Axiom 3: If $A$ and $B$ are Mutually Exclusive ($A\cap B=\varnothing$), then $P(A\cup B)=P(A)+P(B)$.

6.2 The “Independence” vs. “disjoint” Trap

• Mutually Exclusive (Disjoint): Events CANNOT happen at the same time ($P(A\cap B)=0$).
• Independent: One event happening does NOT change the probability of the other.
  • Test: $P(A\cap B)=P(A)\cdot P(B)$ or $P(A|B)=P(A)$.
• Crucial Fact: If $P(A),P(B)>0$, then they cannot be both independent and mutually exclusive.

7.1 Conditional Probability: The “Reduced Sample Space”

$P(A|B)$ is the probability of $A$ restricted to the universe of $B$.

• Formula: $P(A|B)=\frac{P(A\cap B)}{P(B)}$.
• Multiplication Rule: $P(A\cap B)=P(A|B)\cdot P(B)$.

8.1 Total Probability & Partitions

If states $B_1,B_2,\dots$ cover the entire space and don’t overlap (A Partition), the probability of any event $A$ is:

• $P(A)=P(A|B_1)P(B_1)+P(A|B_2)P(B_2)+\dots$
• Visual: Think of $B_i$ as different factories and $A$ as a “defective item.”

8.2 Bayes Theorem (Belief Revision)

Bayes allows us to “invert” conditional probability.

• Formula: $P(B_k|A)=\frac{P(A|B_k)\cdot P(B_k)}{P(A)}$.
• Interpretation:
  • $P(B_k)$: Prior (What we knew before).
  • $P(A|B_k)$: Likelihood (How often the evidence happens in this state).
  • $P(B_k|A)$: Posterior (What we know after the evidence).

TIP

The Table Method for Bayes:

State	Prior	Likelihood	Product	Posterior (Product / Sum)
$B_1$	…	…	…	…
$B_2$	…	…	…	…
Sum	1.0	-	$P(A)$	1.0

---

📈 Volume IV: Random Variables & Statistical Blueprints

🧬 Week 9: Discrete Random Variables & PMFs

A Random Variable ($X$) is a function that maps outcomes of a random experiment to real numbers. It translates “Heads/Tails” into “1/0”.

9.1 The Probability Mass Function (PMF) - $f(x)$

For a discrete RV, the PMF gives the probability of $X$ taking a specific value $x$.

• Notation: $f(x)=P(X=x)$.
• Existence Constraints:
  1. $f(x)\ge 0$ for all $x$.
  2. ${\sum }_{allx}f(x)=1$.

9.2 The Cumulative Distribution Function (CDF) - $F(x)$

The “Running Total” of probability.

• Notation: $F(x)=P(X\le x)={\sum }_{x_i\le x}f(x_i)$.
• Critical Properties:
  • $F(x)$ is non-decreasing.
  • ${\lim }_{x\to -\infty }F(x)=0$ and ${\lim }_{x\to \infty }F(x)=1$.
  • $P(a<X\le b)=F(b)-F(a)$.

9.3 Support of a Random Variable

The set of all values $x$ for which $f(x)>0$.

• Finite Support: $X\in \{1,2,3\}$.
• Countably Infinite Support: $X\in \{0,1,2,\dots \}$.

---

⚖️ Week 10: The Algebra of Expectation & Variance

To summarize a distribution, we use its Expected Value (The “Center”) and Variance (The “Spread”).

10.1 Expected Value ($E[X]$)

• Definition: The long-term weighted average of outcomes.
• Formula: $E[X]={\sum }_{x}x\cdot f(x)$.
• Law of the Unconscious Statistician (LOTUS): To find the expectation of a function $g(X)$:
  • $E[g(X)]={\sum }_{x}g(x)\cdot f(x)$.

10.2 Linearity of Expectation (The “God” Rule)

Expectation is a linear operator. This holds regardless of whether variables are independent!

• $E[aX+b]=aE[X]+b$.
• $E[X+Y]=E[X]+E[Y]$.

10.3 Variance ($Var(X)$)

• Definition: The expected value of the squared deviation from the mean.
• Master Identity: $Var(X)=E[X^2]-(E[X]{)}^2$.
• Transformation: $Var(aX+b)=a^2\cdot Var(X)$.
• Std Dev: $SD(X)=\sqrt{Var(X)}$.

10.4 Independent Variables

If $X$ and $Y$ are Independent:

• Expectation: $E[XY]=E[X]\cdot E[Y]$.
• Variance: $Var(X\pm Y)=Var(X)+Var(Y)$. (Note: It is always addition because squaring terms removes negative signs).
• Covariance: $Cov(X,Y)=0$.

---

🏗️ Week 11: Statistical Blueprints (Specific Discrete Distributions)

Certain experiment types appear so frequently that their distributions are pre-calculated.

11.1 The Bernoulli Trial ($X\sim Ber(p)$)

A single trial with two outcomes: Success (1) or Failure (0).

• PMF: $P(X=1)=p,P(X=0)=1-p$.
• Stats: $E[X]=p$, $Var(X)=p(1-p)$.

11.2 The Binomial Distribution ($X\sim Bin(n,p)$)

The count of successes in $n$ Independent Bernoulli trials.

• PMF: $P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$.
• Conditions:
  1. Fixed number of trials $n$.
  2. Each trial is independent.
  3. $P(Success)$ is constant ($p$).
• Stats: $E[X]=np$, $Var(X)=np(1-p)$.

11.3 The Poisson Distribution ($X\sim Pois(\lambda )$)

Used for counts of “rare” events happening in a fixed interval (Time, Space, Volume).

• PMF: $P(X=k)=\frac{e^{-\lambda }{\lambda }^k}{k!}$.
• Rate Scaling: If $\lambda$ is for time $T$, the rate for time $kT$ is $k\cdot \lambda$.
• Stats: $E[X]=\lambda$, $Var(X)=\lambda$.

IMPORTANT

The Binomial $\to$ Poisson Approximation: When $n\ge 100$ and $p\le 0.05$, the Binomial distribution $Bin(n,p)$ can be modeled as $Pois(\lambda =np)$.

---

🌊 Week 12: Continuous Blueprints (Uniform & Exponential)

In the continuous domain, we deal with “densities” rather than discrete chunks of probability.

12.1 The Probability Density Function (PDF) - $f(x)$

For a continuous RV, $P(X=x)$ is always 0. We only measure probability over an interval $[a,b]$.

• Core Relation: $P(a\le X\le b)={\int }_a^{b}f(x)dx$.
• Constraint: Area under the entire curve ${\int }_{-\infty }^{\infty }f(x)dx=1$.

12.2 The Continuous Uniform Distribution ($X\sim U(a,b)$)

Equally likely outcomes over a range $[a,b]$.

• PDF: $f(x)=\frac{1}{b-a}$ for $x\in [a,b]$.
• Stats: $E[X]=\frac{a+b}{2}$ (Center), $Var(X)=\frac{(b-a{)}^2}{12}$.

12.3 The Exponential Distribution ($X\sim Expo(\lambda )$)

Models the time between events in a Poisson process.

• PDF: $f(x)=\lambda e^{-\lambda x}$ for $x\ge 0$.
• CDF: $F(x)=1-e^{-\lambda x}$.
• Stats: $E[X]=1/\lambda$, $Var(X)=1/{\lambda }^2$.

TIP

The Memoryless Property: $P(X>s+t|X>s)=P(X>t)$. “The probability that the car survives another $t$ hours given it has already survived $s$ hours is the same as the probability a brand new car survives $t$ hours.” Only the Exponential distribution has this property.

12.4 The Poisson-Exponential Duality

• Poisson: Counts of events ($N$ events in time $T$).
• Exponential: Time until the next event ($T$ time between events).
• If events occur at rate $\lambda$ (Poisson), the time between them follows $Expo(\lambda )$.

---

🏁 Final Conclusion

This guide covers the entire spectrum of Statistics 1. From the raw classification of data to the sophisticated modeling of continuous processes, you now possess the complete theoretical toolkit for mastery.