Chapter 1: Introduction to NumPy
What is NumPy?
NumPy (Numerical Python) is the foundational open-source library for scientific computing and high-performance multi-dimensional array operations in Python. Created in 2005 by Travis Oliphant by merging two legacy engines (Numeric and Numarray), it represents the central pillar supporting the entire Python data science, machine learning, and AI ecosystem.
Why NumPy was Created
Standard Python lists are highly flexible container objects: they can hold mixed data types and are dynamically resized. However, this flexibility incurs a devastating performance penalty. Each element in a Python list is a complete individual object requiring type-checking and memory lookup wrappers. NumPy solves this by allocating elements as contiguous, homogeneous blocks at the C-programming layer, enabling CPUs to load and execute mathematical tasks in batches (Vectorization).
Imagine a Python List as a general-purpose moving box containing a mix of books, toys, and apples—each item individually wrapped in layers of bubble wrap. It is highly flexible, but extremely slow to count or sort. A NumPy Array is like a custom carton of identical eggs: packed tightly in a contiguous grid. Counting, indexing, or shifting them is near-instantaneous because we know the exact physical dimension boundaries without unwrapping each item.
import numpy as np
import time
# Create list and array with 1,000,000 elements
size = 1000000
python_list = list(range(size))
numpy_array = np.arange(size)
# Python List element-wise addition
start = time.time()
python_list_result = [x + 2 for x in python_list]
list_time = time.time() - start
# NumPy Array vectorized addition
start = time.time()
numpy_array_result = numpy_array + 2
numpy_time = time.time() - start
print(f"Python list took: {list_time:.5f} seconds")
print(f"NumPy array took: {numpy_time:.5f} seconds")
print(f"NumPy vectorization is {list_time/numpy_time:.1f}x faster!")Chapter 2: Installation & Environment Setup
Before computing, you must prepare a clean virtual environment. This prevents global version collision errors between scientific packages.
Installing via PIP
PIP is Python's native packet manager. In your terminal shell, execute:
pip install numpyInstalling via Conda
Anaconda / Miniconda is the preferred distribution bundle for scientific computing because it installs optimized binaries (like Intel's MKL or OpenBLAS libraries) automatically:
conda install numpyAlways verify that your IDE is mapped to the same active virtual environment or interpreter that contains your installed packages. Run import numpy as np in Python to confirm.
Chapter 3: NumPy Basics: Metadata & Inspecting
Every NumPy array (ndarray) holds critical metadata describing its shape, bytesize, and structures. Accessing these attributes consumes zero processing time because they are cached instantly upon creation.
Essential Array Attributes
- shape: A tuple representing the length along each coordinate axis (e.g.
(3, 4)for 3 rows and 4 columns). - ndim: The integer count of axes (e.g.
2for a flat sheet,3for a volume block). - dtype: The data type of the storage elements (e.g.
int32,float64). - size: The total element count (product of the shape coordinates).
- nbytes: Total computer RAM in bytes occupied by the array buffer.
import numpy as np
matrix = np.array([[10, 20, 30], [40, 50, 60]], dtype=np.int32)
print("Shape:", matrix.shape)
print("Dimensions:", matrix.ndim)
print("Data Type:", matrix.dtype)
print("Total elements:", matrix.size)
print("Item bytesize:", matrix.itemsize)
print("Total memory (bytes):", matrix.nbytes)Chapter 4: Creating Arrays (The Full Toolkit)
Manually constructing lists and casting them is highly inefficient. NumPy supplies premium templates to populate structures in specialized shapes natively.
Standard Factory Methods
np.zeros(shape): Fills the specified shape coordinates with float 0.0.np.ones(shape): Fills coordinates with 1.0.np.arange(start, stop, step): Fills consecutive ranges (excludes stop endpoint).np.linspace(start, stop, num): Generates exactlynumpoints evenly spread between start and stop. Perfect for calculus graphing.np.eye(n): Creates a square identity matrix (diagonal 1s).
import numpy as np
print("Zeros matrix:\n", np.zeros((2, 3)))
print("Ones array:\n", np.ones(3))
print("Linspace 0 to 1 with 5 points:\n", np.linspace(0, 1, 5))
print("Identity Matrix:\n", np.eye(3))Chapter 5: Array Data Types & Memory Allocation
Choosing proper bitwidths prevents integer overflow and dramatically optimizes RAM and processor cache utilization, especially in deep learning architectures.
Common NumPy dtypes
np.int8, np.int32, np.int64: Signed integers supporting various ranges.np.uint8: Unsigned integers (0-255). Standard format for pixel values in computer vision (OpenCV).np.float32: Single-precision float. The standard numerical precision in Deep Learning models.np.float64: Double-precision float (standard python float).
import numpy as np
# Initialize float64 array
arr_64 = np.array([1.5, 2.7, 3.9])
# Cast down to integer (truncates fractional digits)
arr_int = arr_64.astype(np.int32)
print("Original Data:", arr_64, "Dtype:", arr_64.dtype)
print("Cast Data:", arr_int, "Dtype:", arr_int.dtype)Chapter 6: Array Indexing: Positive, Negative, Boolean, Fancy
Accessing data effectively is the core of numerical querying. NumPy supports standard offset indexing, coordinate queries, and advanced logical filters.
Boolean Indexing (Masking)
Applying relational operators yields a boolean mask. Passing this mask back into subscripts filters the array, extracting matching coordinates into a flat 1D sequence.
import numpy as np
scores = np.array([45, 82, 90, 31, 75])
# Generate boolean mask
mask = scores > 70
print("Mask:", mask)
# Filter elements
passing_scores = scores[mask]
print("Filtered passing:", passing_scores)Chapter 7: Array Slicing: Views vs Copies
Slicing retrieves subsets of axes using start:stop:step syntax. However, to save time and memory, slicing returns a View pointing to the original buffer.
If you modify elements in a sliced view, you modify the original parent array! Slices share the same memory buffer. To decouple elements, you must explicitly write arr[slice].copy().
import numpy as np
parent = np.array([10, 20, 30, 40, 50])
# Slice a view of the first 3 indices
view_slice = parent[0:3]
view_slice[0] = 999
print("View Slice changed first index to 999.")
print("Original parent is ALSO changed!:", parent)
# Slice an independent copy
copy_slice = parent[0:3].copy()
copy_slice[1] = 888
print("Original parent remains protected:", parent)Chapter 8: Reshaping & Matrix Transposition
You can reorganize dimensions easily as long as the total element size remains constant. Reshaping returns views whenever possible, leaving performance unhindered.
import numpy as np
vector = np.arange(12)
# Reshape into a 3x4 grid. Using -1 forces auto-calculation of that column axis.
matrix = vector.reshape(3, -1)
print("3x4 Matrix:\n", matrix)
# Transpose columns to rows
print("Transposed (4x3):\n", matrix.T)Chapter 9: Mathematical Operations & Ufuncs
All basic operators (+, -, *, /) act element-wise on matching sizes. This is achieved through compiled C functions called Universal Functions (Ufuncs).
import numpy as np
a = np.array([1, 2, 3])
b = np.array([10, 20, 30])
print("a + b =", a + b)
print("a * b =", a * b)
print("Square root of a:", np.sqrt(a))
print("Exponential (e^x) of a:", np.exp(a))Chapter 10: Broadcasting: Arithmetic on Unequal Shapes
Broadcasting allows element-wise arithmetic between arrays of different dimension lengths without wasting RAM copying duplicated records.
The Core Broadcasting Rules
Starting from trailing dimensions (rightmost axes) and moving left, dimensions are compatible if:
- They are exactly equal in size.
- One of the dimensions is exactly 1.
import numpy as np
matrix = np.zeros((3, 3))
row_vector = np.array([10, 20, 30])
# row_vector shape is (3,) which has trailing dimension compatible with matrix shape (3,3)
# It is stretched vertically to match 3 rows automatically
result = matrix + row_vector
print("Broadcast Result:\n", result)Chapter 11: Aggregation & Axial Reductions
Aggregation reduces dimensions using summaries (sum, mean, median, min, max, std, var). The axis parameter controls orientation:
- axis=0: Collapses rows vertically (operates down each column).
- axis=1: Collapses columns horizontally (operates across each row).
import numpy as np
arr = np.array([[1, 2, 3], [4, 5, 6]])
print("Total global sum:", np.sum(arr))
print("Sum down columns (axis=0):", np.sum(arr, axis=0))
print("Sum across rows (axis=1):", np.sum(arr, axis=1))Chapter 12: Sorting & Partitioning
NumPy includes lightning fast sorting routines built directly inside compiled modules. You can retrieve complete sorted arrays or intermediate indices using argsort.
import numpy as np
unsorted = np.array([40, 10, 30, 20])
# Retrieve sorting offsets
indices = np.argsort(unsorted)
print("Original elements:", unsorted)
print("Sorting offsets:", indices)
print("Sorted via offsets:", unsorted[indices])Chapter 13: Searching: np.where & Filtering
Beyond simple masking, np.where(condition, x, y) operates like a vectorized ternary operator: if condition is true, output x, otherwise output y.
import numpy as np
arr = np.array([12, 45, 9, 80, 23])
# Set elements over 20 to 1, otherwise set to 0
binary_labels = np.where(arr > 20, 1, 0)
print("Thresholded labels:", binary_labels)Chapter 14: Stacking & Splitting Arrays
Join multiple arrays using hstack (horizontal), vstack (vertical), or stack (new axis). Split grids using np.split.
import numpy as np
a = np.array([1, 2])
b = np.array([3, 4])
print("Vertical stack (Rows):\n", np.vstack((a, b)))
print("Horizontal stack (Cols):\n", np.hstack((a, b)))Chapter 15: Random Module & Distributions
NumPy's random library simulates stochastic variables. Set random seeds to ensure computational experiments can be reproduced reliably.
import numpy as np
# Set reproducible random seed state
np.random.seed(42)
# Normal (bell-curve) distribution points
bell_samples = np.random.normal(loc=0.0, scale=1.0, size=3)
print("Reproducible Normal Samples:", bell_samples)Chapter 16: Linear Algebra (linalg)
Execute matrix math (dot products, determinants, inverses, solving algebraic equations) instantly using np.linalg wrappers mapped to optimized LAPACK packages.
import numpy as np
matrix = np.array([[1, 2], [3, 4]])
# Compute Inverse Matrix
inv = np.linalg.inv(matrix)
print("Inverse:\n", inv)
# Determinant
det = np.linalg.det(matrix)
print("Determinant:", det)Chapter 17: Statistics & Quantiles
Examine statistics within datasets: evaluate covariance, standard deviation spreads, correlation structures, and percentiles.
import numpy as np
data = np.array([10, 20, 25, 30, 45, 50])
print("75th Percentile:", np.percentile(data, 75))
print("Correlation Matrix:\n", np.corrcoef(data, data * 2))Chapter 18: File Handling: Saving & Loading
Save massive multi-dimensional elements quickly using NumPy binary formatting (.npy) or tabular texts (.csv).
import numpy as np
matrix = np.eye(3)
# Save as compiled binary file
np.save("identity.npy", matrix)
# Load binary array back
loaded_matrix = np.load("identity.npy")
print("Successfully loaded identity matrix:\n", loaded_matrix)Chapter 19: Performance Optimization & Vectorization
To write elite clean code, eliminate explicit for loops in Python. Vectorized operations execute in fast hardware registers simultaneously.
import numpy as np
arr = np.random.randn(100000)
# Unoptimized loop approach
%timeit [np.sin(x) for x in arr]
# Optimized vectorized approach
%timeit np.sin(arr)Chapter 20: NumPy Internals: Memory Layout
At the C-level, multi-dimensional elements are stored sequentially. C-order (row-major) stores row blocks consecutively in RAM, while F-order (column-major) stores column strides consecutively.
import numpy as np
matrix = np.array([[1, 2], [3, 4]], dtype=np.int32)
# Inspect internal memory strides mapping coordinate changes
print("Strides info:", matrix.strides)Chapter 21: Error Handling & Debugging
Common failures include ValueError: shape mismatch when attempting math on arrays that do not satisfy broadcasting rules, or TypeError when passing invalid dtypes.
Error: ValueError: operands could not be broadcast together with shapes (3,3) (2,).
Fix: Confirm shapes with print(arr.shape) and pad axes using np.newaxis to ensure alignment rules are satisfied.
Chapter 22: Best Practices & Code Readability
Maintain elegant, clean code rules to scale scientific products successfully:
- Import NumPy explicitly using
npalias:import numpy as np. - Explicitly declare custom
dtypesignatures to prevent unneeded 64-bit float allocations. - Keep dimensions clear: document inputs and output matrices explicitly inside docstrings.
Chapter 23: Real-World Industry Projects
Project 1: Sales Analytics System (Retail Case)
Analyze transaction registers across global stores. Compute daily maximum bounds, median thresholds, and filter poor store registers to maximize profits.
import numpy as np
# rows: 4 days, columns: 3 branches
sales = np.array([
[1200, 1500, 950],
[1300, 1600, 1100],
[900, 1400, 800],
[1500, 1800, 1200]
])
print("Total global revenues:", np.sum(sales))
print("Average daily revenue per store branch:", np.mean(sales, axis=0))
print("Best sale day per branch (row indices):", np.argmax(sales, axis=0))Chapter 24: Machine Learning Connections
Why NumPy is the ultimate backbone of modern AI:
- TensorFlow & PyTorch: Tensors share immediate memory blocks and design semantics directly with NumPy arrays.
- Pandas: Series and DataFrames store records inside optimized 1D and 2D ndarrays under the hood.
- OpenCV: Digital graphics are processed as 3D matrices of size (height, width, color channels) natively.
🎓 NumPy Certification Exam (10 Questions)
Take this comprehensive testing module to measure your mastery of NumPy. Score 80% or higher to earn elite certified recognition.
No comments:
Post a Comment