Math 371 - Alexiades
                  Norms on vectors and matrices

Let V be a vector space. A real-valued function ∥ . ∥ on V is called a norm on V if:
(i)   ∥ x ∥ ≥ 0 for any x in V, and ∥ x ∥ = 0 iff x = 0
(ii)  ∥ λ x ∥ = |λ| ∥x∥ for any scalar λ , any x in V
(iii) ∥ x + y ∥ ≤ ∥x∥ + ∥y∥ for any x, y in V   (triangle inequality)

Several norms can be defined on vectors in ℝⁿ. Most useful :

p-norm, p≥1 :   ∥ x ∥_p = ( ∑ |x_i|^p )^1/p   in particular:  

1-norm: ∥x∥₁ = ∑ |x_i| ,

2-norm: ∥x∥₂ = ( ∑ |x_i|² )^1/2 = √x•x   = Euclidean norm

max-norm :  ∥ x ∥_∞ = max( |x_i| )   (amazingly   lim_p→∞ ∥x∥_p = ∥x∥_∞ )
e.g. x = ( 1, −3, 2 ):   ∥x∥₁ = 6 ,   ∥x∥₂ = √14 ,   ∥x∥_∞ = 3
Theorem: All vector norms on a finite-dimensional vector space are equivalent. They define the same sense of convergence.

Matrix norm: Let A be m×n matrix. The (operator) norm induced on A by a vector norm ∥ ∥ is
    ∥A∥ = max{ ∥Ax∥ / ∥x∥ } = amplification factor under the action of A on any vector = maximum stretching
    ∥ I ∥ = 1
Easiest to compute (and most used):

1-norm: ∥A∥₁ = max absolute column sum

2-norm: ∥A∥₂ = (no easy way) = max |eigenvalue| = spectral norm

∞-norm: ∥A∥_∞ = max absolute row sum

Frobenius-norm: ∥A∥_F = ( ∑₁^m ∑₁ⁿ |a_ij|² )^1/2 = tr( A^TA )
e.g. A = [1 , −3 ; −2 , 2]:   ∥A∥₁ =max(3,5)=5 , ∥A∥₂ ≈ 4.1306 , ∥A∥_∞ = max(4,4)=4 , ∥A∥_F = (1+9+4+4)^1/2 = √18 ≈ 4.2426

Very desirable property for a matrix norm (which all induced norms satisfy):

∥Ax∥ ≤ ∥A∥ ∥x∥   ,   ∥AB∥ ≤ ∥A∥ ∥B∥
  (but not all norms do this, and those that don't are not useful...)

Cauchy-Swartz Inequality: |x • y| ≤ ∥x∥₂ ∥y∥₂   for any vectors x, y

---

                  Function norms

The set of functions f(x) defined on an interval [a, b] constitutes a vector space (with usual pointwise addition and multiplication by a scalar).
Many useful norms can be defined, the most usual are:

p-norm, p≥1 :   ∥ f ∥_p = ( ∫_a^b |f(x)|^p dx )^1/p.   The space of functions with finite p-norm is L^p(a, b).
in particular:  

1-norm: ∥f∥₁ = ∫_a^b |f(x)| dx.   The space of functions with finite 1-norm is L¹(a, b) of integrable functions on (a,b).

2-norm: ∥f∥₂ = ( ∫_a^b |f(x)|² dx )^1/2.   The space of functions with finite 2-norm is L²(a, b) of square-integrable functions on (a, b).
        The 2-norm is the only p-norm that comes from an inner product: <f , g> := ∫_a^b f(x)g(x) dx   (generalization of dot product), ∥f∥₂ = √<f,f>
        This makes L²(a, b) into a Hilbert space, by far the most useful function space. Orthogonality can be defined:   f ⊥ g means <f , g> = 0.

inf-norm or sup-norm or max-norm:  ∥ f ∥_∞ = sup{ |f(x)|: a≤x≤b }
        The space of continuous functions with finite inf-norm is ℭ[a, b], next most useful to L² space.
        It is a Banach space (complete normed space) but not an inner product space.

All these generalize to functions defined in a multidimensional region Ω by replacing the interval [a,b] by Ω.