NumPy — Arrays, Broadcasting, Indexing & Linear Algebra

Why NumPy Exists — The Performance Problem

Python lists store pointers to arbitrary Python objects. Every arithmetic operation involves Python object overhead, type checking, and memory indirection. For numerical computing, this is devastating.

NumPy's ndarray stores values in a contiguous block of typed memory — exactly like a C array. Operations are dispatched to compiled C/Fortran routines that process entire arrays with no Python overhead per element.

The ndarray — Internals

An ndarray has four key attributes that define its structure:

• dtype — the type of each element (int32, float64, bool, etc.)
• shape — a tuple of dimension sizes, e.g. (3, 4) for a 3×4 matrix
• strides — bytes to step in each dimension; enables reshaping without copying
• data — pointer to the raw memory buffer

Creating Arrays — Every Method

Data Types — Complete Reference

Indexing and Slicing — Every Pattern

Universal Functions (ufuncs) and Aggregations

Broadcasting — The Most Powerful Feature

Broadcasting lets NumPy work with arrays of different shapes without copying data. The rules:

1. If arrays have different ndim, prepend 1s to the shorter shape
2. Dimensions of size 1 are stretched to match the other
3. If shapes don't match after stretching → error

Shape Manipulation

Random Number Generation

Linear Algebra

Performance: Vectorization Patterns

Project: Gradient Descent from Scratch

Implement linear regression and gradient descent using only NumPy — no scikit-learn.

Exercises

Exercise 1 — Array Creation and Inspection

Exercise 2 — Indexing and Masking

Exercise 3 — Broadcasting and Vectorization

Exercise 4 — Statistics from Scratch

Exercise 5 — Image Processing with Arrays

Exercise 6 — PCA from Scratch

Exercise 7 — Moving Averages and Time Series

Exercise 8 — Neural Network Forward Pass

Key Takeaways

• NumPy stores data in contiguous typed memory — this is why it beats Python loops by 10-100x for numerical work
• strides let NumPy reshape, transpose, and slice arrays without copying data — always check if you have a view or a copy
• Avoid Python loops over array elements — every operation has a vectorized NumPy equivalent
• axis=0 reduces along rows (gives column results); axis=1 reduces along columns (gives row results); always add keepdims=True when the result must broadcast back
• Broadcasting follows three rules: prepend 1s, stretch size-1 dims, error on incompatible dims — internalize these to write clean vectorized code
• Boolean indexing returns a copy; slicing returns a view — modifying a slice modifies the original
• np.where(cond, a, b) is the vectorized ternary operator — use it instead of masked assignment when creating a new array
• @ is matrix multiply; * is element-wise — never confuse them
• np.linalg.solve(A, b) is faster and more numerically stable than inv(A) @ b
• Use dtype=np.float32 instead of float64 for ML training — half the memory, same accuracy for most tasks