🧷Intro to Scientific Computing Unit 3 – Numerical Representation & Error Analysis

What's the Big Deal?

• Computers process and store data using binary number system consists of only 0s and 1s
• Binary representation is fundamental to how computers operate at the most basic level
• Understanding binary and how it relates to decimal number system is crucial for scientific computing
• Floating-point representation used by computers to approximate real numbers introduces potential for errors and inaccuracies
  • Rounding errors occur when real numbers are represented with finite precision
  • Truncation errors happen when an infinite series is approximated by a finite number of terms
• Precision and accuracy are key concepts in scientific computing refer to how close a value is to the true value and how consistently a value can be reproduced
• Error analysis is critical for understanding the limitations and reliability of computational results
• Real-world applications such as weather forecasting, financial modeling, and engineering design rely heavily on numerical computations and are impacted by these concepts

Binary & Decimal: The Basics

• Binary number system uses base-2 only has two digits: 0 and 1
• Decimal number system uses base-10 has ten digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9
• In binary, each digit represents a power of 2 ($2^0$, $2^1$, $2^2$, etc.)
  • Example: 1101 in binary = 1 x $2^3$ + 1 x $2^2$ + 0 x $2^1$ + 1 x $2^0$ = 13 in decimal
• In decimal, each digit represents a power of 10 ($10^0$, $10^1$, $10^2$, etc.)
  • Example: 1234 in decimal = 1 x $10^3$ + 2 x $10^2$ + 3 x $10^1$ + 4 x $10^0$
• Converting between binary and decimal involves summing the products of each digit and its corresponding power of the base
• Computers use binary because it aligns with the two states of electronic switches (on/off) and simplifies circuitry design

Floating-Point: How Computers Handle Decimals

• Floating-point representation used by computers to approximate real numbers
• Consists of a sign bit, exponent, and mantissa (also called significand)
  • Sign bit indicates whether the number is positive (0) or negative (1)
  • Exponent represents the power of the base (usually 2) that the mantissa is multiplied by
  • Mantissa represents the significant digits of the number
• Formula for floating-point representation: $(-1)^{sign} \times mantissa \times base^{exponent}$
• IEEE 754 standard defines the format for single-precision (32-bit) and double-precision (64-bit) floating-point numbers
• Floating-point representation has limited precision introduces rounding errors when real numbers cannot be exactly represented
• Special values such as infinity and NaN (Not a Number) are used to handle exceptional cases like division by zero

Precision vs. Accuracy: What's the Difference?

• Precision refers to the level of detail or number of significant digits in a measurement or calculation
  • Example: A value of 3.14159 has higher precision than 3.14
• Accuracy refers to how close a measured or calculated value is to the true value
  • Example: If the true value of pi is 3.14159265359, then 3.14 is more accurate than 3.15
• High precision does not guarantee high accuracy measurements can be precise but inaccurate if there is a systematic error or bias
• High accuracy requires both high precision and low measurement error
• In scientific computing, it's important to consider both precision and accuracy when evaluating results and making decisions based on them
  • Using appropriate data types (e.g., single vs. double precision) can help balance precision and computational efficiency
  • Validating results against known values or experimental data can help assess accuracy

Rounding & Truncation: The Nitty-Gritty

• Rounding involves replacing a number with an approximation that has a shorter, simpler, or more explicit representation
  • Rounding to the nearest integer: 3.14 rounds to 3, 3.85 rounds to 4
  • Rounding to a specific number of decimal places: 3.14159 rounded to three decimal places is 3.142
• Truncation involves discarding the less significant digits of a number
  • Truncating to an integer: 3.14 truncates to 3, 3.85 truncates to 3
  • Truncating to a specific number of decimal places: 3.14159 truncated to three decimal places is 3.141
• Rounding and truncation can introduce errors in calculations especially when performed repeatedly
• In floating-point arithmetic, rounding is often performed automatically by the computer based on the available precision
• Truncation errors occur when an infinite series or process is approximated by a finite number of steps
  • Example: Approximating the exponential function using a Taylor series truncated to a finite number of terms

Error Types: Where Things Go Wrong

• Absolute error is the magnitude of the difference between the true value and the approximate value
  • Formula: $|true value - approximate value|$
  • Example: If the true value is 10 and the approximate value is 9.5, the absolute error is $|10 - 9.5| = 0.5$
• Relative error is the ratio of the absolute error to the magnitude of the true value
  • Formula: $\frac{|true value - approximate value|}{|true value|}$
  • Example: Using the same values as above, the relative error is $\frac{0.5}{10} = 0.05$ or 5%
• Round-off errors occur due to the limited precision of floating-point representations
  • Example: The decimal number 0.1 cannot be exactly represented in binary, leading to round-off errors in calculations
• Truncation errors occur when an infinite process is approximated by a finite number of steps
  • Example: Approximating an integral using a finite number of rectangles in the Riemann sum method
• Propagation of errors occurs when errors in input data or intermediate calculations accumulate and affect the final result
  • Example: In a complex calculation with multiple steps, errors from each step can compound and lead to a larger overall error

Error Analysis: Keeping Track of Mistakes

• Error analysis involves quantifying and tracking the errors that occur in numerical computations
• Forward error analysis starts with the input data and tracks how errors propagate through the computation to affect the final result
• Backward error analysis starts with the computed result and determines the perturbation in the input data that would produce the same result
• Condition number measures the sensitivity of a problem to small changes in the input data
  • A problem with a high condition number is ill-conditioned and more sensitive to input errors
  • A problem with a low condition number is well-conditioned and less sensitive to input errors
• Stability of an algorithm refers to how errors in the input data affect the errors in the output
  • A stable algorithm produces a result with an error that is not much larger than the error in the input data
  • An unstable algorithm can produce a result with a much larger error than the input data error
• Error bounds provide limits on the size of the error in a computed result
  • Can be expressed as absolute bounds (e.g., $|error| < 10^{-6}$) or relative bounds (e.g., $|relative error| < 0.1$)

Real-World Applications: Why This Stuff Matters

• Weather forecasting uses numerical models to predict future weather conditions based on current data
  • Accuracy of predictions depends on the precision of input data, model equations, and computational methods
  • Small errors in initial conditions can lead to large differences in long-term predictions (butterfly effect)
• Computer graphics and animation rely on floating-point calculations to render realistic images and motion
  • Rounding errors can cause visual artifacts or inconsistencies in the final output
  • Techniques like anti-aliasing and texture filtering help minimize the impact of these errors
• Financial modeling and analysis use numerical computations to value assets, assess risk, and make investment decisions
  • Rounding and truncation errors can lead to inaccurate valuations or risk assessments
  • Regulations like the Global Investment Performance Standards (GIPS) provide guidelines for handling these errors
• Engineering design and simulation use numerical methods to analyze and optimize complex systems
  • Finite element analysis (FEA) and computational fluid dynamics (CFD) are examples of numerical techniques used in engineering
  • Errors in these simulations can lead to incorrect design decisions or product failures
• Scientific research in fields like physics, chemistry, and biology relies on accurate and precise numerical computations
  • Propagation of errors can affect the reliability and reproducibility of experimental results
  • Techniques like uncertainty quantification and sensitivity analysis help researchers assess the impact of errors on their conclusions