Orthonormal bases

Email completed labs to npflueger@amherst.edu by Wednesday night. Include all team members’ last names in the attachment file name (e.g. BrillNoether-Lab5.nb)

On Problem Set 6 (supplementary problem 3), you explored a bit about special sets called orthonormal sets. These sets are particularly useful for approximations, because they reduce otherwise-difficult optimization problems to computing a few dot products. In this lab, we’ll think about sets that are both orthonormal and also bases.
   
Terminology: An orthonormal basis is a set B of vectors in (R^n) such that:
	1) B is a basis.
	2) Every vector in B has length 1.
	3) Any two vectors in B are orthogonal.

Syntax note: you can compute dot products in Mathematica using a period between two vectors, as in the cell below.

{1,2,3} . {4,5,6}

Exercise 1. All three of the sets below are bases for (R^3). Which one(s) are orthonormal bases? For each basis, show some Mathematica calculations to verify that it is orthonormal, or show some calculations that show it is not. (Note: in some of your calculations, you may see some dot products that appear to be extremely small but nonzero. These numbers way be regarded as actually being 0; the tiny numbers are just due to rounding error.)

(a) {(({{1}, {-1}, {0}})),(({{1}, {1}, {1}})),(({{1}, {1}, {-2}}))}
(b) {(({{0.6`}, {0.48`}, {0.64`}})),(({{0.8`}, {-0.36`}, {-0.48`}})),(({{0.`}, {-0.8`}, {0.6`}}))}
(c) {(({{(3/5)}, {(4/5)}, {0}})),(({{-(3/5)}, {(4/5)}, {0}})),(({{(1/3)}, {(1/3)}, {(2/3)}}))}

Coordinates in orthonormal bases

Orthonormal bases have several advantages in applications. One is that it is particularly easy to compute coordinates in an orthonormal basis, due to the following fact.

Fact: If B an orthonormal basis for (R^n), then the coordinates of any vector v in (R^n) in basis B can be found by taking the dot product of v which each basis vector from B.

Exercise 2. Find the coordinates of (({{30}, {-20}, {40}})) in the basis from exercise 1(b) (which you should have found to be orthonormal). Verify that your answer is correct by computing  a linear combination of the basis vectors and checking that it is equal to (({{30}, {-20}, {40}})).

The Gram-Schmidt procedure

OK, so we know orthonormal bases are handy when you have them. We now discuss an efficient algorithm to create orthonormal bases in practice, called the Gram-Schmidt procedure.

The basic idea is to take each vector in the given basis and subtract from it the parts that duplicate the previous vectors.  Please work through the following example that shows how to do the Gram-Schmidt procedure before doing the exercises.

Consider the basis { (({{({{1}, {0}})}, {1}})) , (({{({{1}, {1}})}, {0}})) , (({{({{2}, {1}})}, {2}})) } for (R^3).  (Take my word for it that this is a basis)

Enter the 3 vectors into Mathematica:

v1={1,0,1};
v2={1,1,0};
v3={2,1,2};

Now we’ll create a new orthogonal basis {w1,w2,w3}. 

Terminology: the projection of one vector onto another is defined by the following formula.
	(Subscript[proj, (OverVector[u])])(OverVector[v]) = (((OverVector[u]) ·(OverVector[v]))/((OverVector[u]) · (OverVector[u])))(OverVector[u])
The meaning of the projection is: it is the multiple of (OverVector[u]) that is closest to (OverVector[v]).

At each step, take the next vector in the given basis and subtract off the projections of that vector onto all previous vectors.

1) Start with v1 as the first vector w1.
2) Next take v2 and subtract off the projection of v2 onto w1 to obtain w2.
3) Then take v3 and subtract off the projections of v3 onto w1 and onto w2 to obtain w3.\

The resulting set {w1,w2,w3} will be orthogonal (any two vectors in it are orthogonal), but may not yet be orthonormal (not all vectors have length 1). Execute the cell below to do these computations.

w1=v1

proj1=(v2.w1)/(w1.w1) *  w1;
w2=v2-proj1

proj1=(v3.w1)/(w1.w1) * w1;
proj2=(v3.w2)/(w2.w2) * w2;
w3=v3-proj1-proj2

Finally, we divide each vector by its length to obtain vectors u1, u2, u3 that will have length 1. Recall that length is equal to the square root of the dot product of a vector with itself. Execute the cell below to perform this computation.

u1=w1/Sqrt[ w1 . w1]
u2=w2/Sqrt[w2 . w2]
u3=w3/Sqrt[w3 . w3]

u1.u2       (* to check that u1 and u2 are orthogonal *)
u1.u3       (* to check that u1 and u3 are orthogonal *)
u2.u3       (* to check that u2 and u3 are orthogonal *)

The resulting orthonormal basis for (R^3) is {(({{(1/(Sqrt[2]))}, {0}, {(1/(Sqrt[2]))}})),(({{(1/(Sqrt[6]))}, {(Sqrt[(2/3)])}, {-(1/(Sqrt[6]))}})),(({{-(1/(Sqrt[3]))}, {(1/(Sqrt[3]))}, {(1/(Sqrt[3]))}}))}.

Exercise 4.  Apply the Gram-Schmidt procedure to the basis { (({{1}, {1}})) , (({{3}, {2}}))} to obtain an orthonormal basis for (R^2).  (If necessary, use the Simplify command to clean up messy results.)

Exercise 5.  Apply the Gram-Schmidt procedure to the basis { (({{({{0}, {3}})}, {4}})) , (({{({{-20}, {27}})}, {11}})) , (({{({{-14}, {-4}})}, {-2}})) } to obtain an orthonormal basis for (R^3).  Check that the vectors are indeed orthogonal by evaluating the dot products of each pair of vectors (so you’ll check 3 dot products).

Exercise 6.  Apply the Gram-Schmidt procedure to the basis { (({{({{1}, {0}})}, {({{0}, {-1}})}})) , (({{({{1}, {2}})}, {({{0}, {-1}})}})) , (({{({{3}, {1}})}, {({{1}, {-1}})}})) , (({{({{0}, {1}})}, {({{1}, {-1}})}})) } to obtain an orthonormal basis for (R^4).  Check that the vectors are indeed orthogonal by evaluating the dot products of each pair of vectors (so you’ll check 6 dot products).