1. Prove that if vector b is orthogonal to the column space of matrix A then v=0 is a least squares solution of the system Av=b. From the WebNotes, we know that a vector v is a least squares solution of the system Av=b if and only if v is the solution of the system ATAv=ATb. Let A be considered a row of the column vectors c1,...,cn. The fact that b is orthogonal to the column space of A tells us, by the definition of orthogonality, that the dot product of b with every column vector of A is zero. That is, (ci,b)=0 for i=1,...,n. We can say that AT is a column of the row vectors r1,...,rn, where ri is identical to ci rewritten as a row vector (corresponding entries are equal). So
A =
[ c1 ]
....
[ cn ]
b = ((c1*b), ... , (cn*b))
But all of the (ci,b) are equal to zero, so ATb=0. Thus our system is simplified to ATAv=0. By inspection, v=0 is obviously a solution to this system. So v=0 is thus the least squares solution of Av=b.

=========================================================================

2. Show that functions sin(x), xsin(x) are linearly independent in C[0,1] and find the projection of the function x2 onto the subspace spanned by these functions. What is the distance from x2 to this subspace?

Given a set of n functions, each of which has the first n-1 derivatives, we know from the theorem about the Wronskian that if the Wronskian of a set of functions is not identically zero, then the set is independent.

Let f(x)=sin(x) and g(x)=xsin(x). Then the Wronskian of the set {f(x),g(x)} is the determinant of the matrix
[ f(x) g(x) ]
[ f'(x) g'(x) ]
Use Maple to find the Wronskian:


> f:=x->sin(x);g:=x->x*sin(x);

f := sin

g := x -> x sin(x)

> matrix([[f(x),g(x)],[diff(f(x),x),diff(g(x),x)]]);
[ sin(x) x sin(x) ]
[ cos(x) sin(x) + x cos(x) ]

> det(");

sin(x)2


This Wronskian is not identically zero, so the set is linearly independent.

---------

The WebNotes give us that the projection v of a vector w onto a vector space having an orthogonal basis {v1,...,vk} is 
           (w*v1)         (w*v2)               (w*vk)
      v = --------- v1 + --------- v2 + ... + --------- vk.
          (v1*v1)       (v2*v2)             (vk*vk)

However, while {sin(x),xsin(x)} is a basis of the set it spans (since the two vectors are linearly independent), it is not orthogonal. To find a suitable orthogonal basis, we use the Gram-Schmidt algorithm as given in the Notes to find an orthogonal basis {v1,v2}: 
                   v1 = sin(x)
                                   (xsin(x),sin(x))
                   v2 = xsin(x) - ---------------- sin(x).
                                   (sin(x),sin(x))

Use Maple to find v2 (note that the dot product of two functions f(x) and g(x) in C[0,1] is equal to int(f(x)*g(x),x=0..1)):


> a:=int(sin(x)*x*sin(x),x=0..1);

a := 1/4 + 1/4 sin(1)2 - 1/2 cos(1) sin(1)


> b:=int(sin(x)*sin(x),x=0..1);
b := - 1/2 cos(1) sin(1) + 1/2


> v2:=simplify(x*sin(x)-a/b*sin(x));
sin(x) (2 x cos(1) sin(1) - 2 x + 2 - cos(1)2 - 2 cos(1) sin(1))

v2 := 1/2 ---------------------------------------------------------------- cos(1) sin(1) - 1


We can now find the projection p of x2 on the subspace with orthogonal basis {v1,v2}: 
                       (x2*v1)       (x2*v2)
                   p = --------- v1 + --------- v2.
                       (v1*v1)       (v2*v2)
Use Maple:
> c:=int(x2*sin(x),x=0..1);

c := 2 sin(1) + cos(1) - 2


> d:=int(x2*v2,x=0..1);
d :=

                            3                      
2                          2
        - 10 sin(1) + cos(1)  + 8 cos(1) - 6 cos(1)  sin(1) + 10 cos(1) 
sin(1)
  - 1/2 
-----------------------------------------------------------------------
                                   cos(1) sin(1) - 1

           2
     cos(1)  - 2 + 2 cos(1) sin(1)
   + -----------------------------
           cos(1) sin(1) - 1

> f:=int(v2*v2,x=0..1);

f := - 1/24 (12 cos(1) sin(1)3 - 12 sin(1)2 + 12 cos(1)3 sin(1) - 4 cos(1) sin(1) + 3 cos(1)4 - 18 cos(1)2 + 10)/(cos(1) sin(1) - 1)



> p:=simplify(c/b*sin(x)+d/f*v2);

p := - 4 sin(x) (12 x - 10 - 12 cos(1)  x + 28 sin(1) - 77 cos(1) + 6 cos(1)

                4           5            3
      + 6 cos(1)  + 6 cos(1)  - 12 cos(1)  sin(1) + 16 cos(1) sin(1)

                 2                          4                          3
      + 55 cos(1)  sin(1) - 33 sin(1) cos(1)  - 30 sin(1) x - 39 cos(1)  x

                              2              4
      + 84 cos(1) x + 6 cos(1)  x + 27 cos(1)  x sin(1) - 24 x cos(1) sin(1)

                   2                  3                     3            5
      - 72 x cos(1)  sin(1) + 6 cos(1)  x sin(1) + 55 cos(1)  - 18 
cos(1)  x)

        /                                 3                   
2            4
       /  (2 - 10 cos(1) sin(1) - 6 cos(1)  sin(1) + 14 cos(1)  - 11 cos(1)
      /

                5
      + 3 cos(1)  sin(1))
We now know the projection of x2 onto the subspace spanned by {sin(x), xsin(x)}.

-------- As explained in the WebNotes, the vector v'=x2-p is the normal component of x2, and ||v'|| is the distance from x2 to the subspace spanned by v1 and v2.

Maple can compute ||v'||=sqrt(v'*v') for us:

> sqrt(int(vprime*vprime,x=0..1));
                       2       4                           9       3
(- 1/5 (- 273240 sin(1)  cos(1)  + 316184 %1 - 25740 cos(1)  sin(1)

                    6       4                      4               
5       4
     - 151560 cos(1)  sin(1)  + 46080 cos(1) sin(1)  - 96720 cos(1)  sin(1)

                   6       3                3       3               
8       3
     + 63120 cos(1)  sin(1)  - 214000 sin(1)  cos(1)  - 30960 cos(1)  sin(1)

                   7               3                      2
     - 85200 cos(1)  + 84640 cos(1)  + 16160 cos(1) sin(1)

                    2
     + 142880 cos(1)  sin(1) + 208280 %2 + 25440 sin(1) - 62240 cos(1)

                   2               2       2               8
     - 34640 sin(1)  - 49620 sin(1)  cos(1)  + 94080 cos(1)  sin(1)

                    7                      9                       2
     + 320884 cos(1)  sin(1) + 57426 cos(1)  sin(1) - 235856 cos(1)


                   2       3              9                6       2
     - 54080 sin(1)  cos(1)  - 5040 cos(1)  + 333864 cos(1)  sin(1)

                   5                       3                5
     + 50560 cos(1)  - 163200 cos(1) sin(1)  - 798144 cos(1)  sin(1)

                    7       2                2       8               
10       2
     - 105360 cos(1)  sin(1)  - 144744 sin(1)  cos(1)  + 38511 cos(1)   
sin(1)

                   6                              5       
2                4
     + 82880 cos(1)  sin(1) - 4004 + 101520 cos(1)  sin(1)  + 361888 cos(1)

                   3                       4                3       4
     - 25440 sin(1)  - 314320 sin(1) cos(1)  + 146000 sin(1)  cos(1)

                   6                8               10               12
     - 56192 cos(1)  - 108161 cos(1)  + 17040 cos(1)   + 12960 cos(1)

                   11                      10                      11
     - 29520 cos(1)   sin(1) - 30960 cos(1)   sin(1) + 17280 cos(1)

                    4       2                5       3                
2       3
     + 162000 sin(1)  cos(1)  + 433240 cos(1)  sin(1)  - 168320 cos(1)  
sin(1)


                   4       3               4               7       4
     + 77760 sin(1)  cos(1)  + 33600 sin(1)  + 20880 cos(1)  sin(1)

                   4       4               4       8               
9       2
     + 73200 sin(1)  cos(1)  + 35640 sin(1)  cos(1)  + 38160 cos(1)  sin(1)

                   7       3
     - 48960 cos(1)  sin(1) )

       /                              2            4           5        2
      /  (2 - 10 %2 - 6 %1 + 14 cos(1)  - 11 cos(1)  + 3 cos(1)  sin(1))  
+ 16
     /

                     4                    2
    (33 sin(1) cos(1)  + 12 %1 - 55 cos(1)  sin(1) - 16 %2 - 28 sin(1)

               5           4            3           2
     - 6 cos(1)  - 6 cos(1)  - 55 cos(1)  - 6 cos(1)  + 77 cos(1) + 10)

       /                              2            4           5
      /  (2 - 10 %2 - 6 %1 + 14 cos(1)  - 11 cos(1)  + 3 cos(1)  sin(1)))1/2
     /

%1 :=                          cos(1)  sin(1)

%2 :=                           cos(1) sin(1)
This is ||v'||, the distance from x2 to the subspace spanned by sin(x) and xsin(x). Numerically, we have that ||v'||=.04240917118.
============================================================================
3. Find a basis of the subspace in R4 spanned by the vectors (1,2,3,4), (2,3,4,5), (4,5,6,7), (2,2,2,2), (5,4,3,2).

In the WebNotes, just below the theorem about the rank of a matrix, we have that we can find a basis of a subspace spanned by some vectors by finding a core of the vectors. The theorem about the core of a set of vectors in Rn tells us that we can do this by constructing a matrix whose columns are the vectors spanning the subspace whose basis we want and then finding the reduced-row echelon form of the matrix. The columns of the original matrix corresponding to the columns of the reduced matrix which contain leading ones are then a basis of the subspace.


> matrix([[1,2,3,4],[2,3,4,5],[4,5,6,7],[2,2,2,2],[5,4,3,2]]);
[ 1 2 3 4 ]
[ 2 3 4 5 ]
[ 4 5 6 7 ]
[ 2 2 2 2 ]
[ 5 4 3 2 ]

> transpose(");
[ 1 2 4 2 5 ]
[ 2 3 5 2 4 ]
[ 3 4 6 2 3 ]
[ 4 5 7 2 2 ]

> gaussjord(");
[ 1 0 -2 -2 -7 ]
[ 0 1 3 2 6 ]
[ 0 0 0 0 0 ]
[ 0 0 0 0 0 ]
Only the first two columns contain leading ones; thus, the first two columns of the unreduced matrix are a basis of the subspace. So our basis is the set

{(1,2,3,4),(2,3,4,5)}.


Alternatively, we can throw out vectors in the set which are linear combinations of other vectors in the set (we know that we will have to do this anyway; a basis of a subspace of R4 cannot contain more than four vectors). Notice that 

                  (2,2,2,2) = 2*(2,3,4,5) - 2*(1,2,3,4)

                  (4,5,6,7) = 3*(2,3,4,5) - 2*(1,2,3,4)

                  (5,4,3,2) = 6*(2,3,4,5) - 7*(1,2,3,4)
So we can throw out (2,2,2,2), (4,5,6,7) and (5,4,3,2). The vectors (2,3,4,5) and (1,2,3,4) we know to be linearly independent because they are non-proportional. So, again, {(2,3,4,5),(1,2,3,4)} is a basis of our subspace.
=============================================================================
4. Find the rank of the matrix
[ t 1 1 1 1 ]
[ t 1 1 1 t ]
[ 1 t t t 1 ]
[ t 1 1 t t ]
depending on t.

If we can find a reduced-row echelon form of the matrix, then we know from the WebNotes that the rank of the matrix is simply equal to the number of leading 1's. We must avoid gaussjord to eliminate division by t:

> A:=matrix([[t,1,1,1,1],[t,1,1,1,t],[1,t,t,t,1],[t,1,1,t,t]]);
A := [ t 1 1 1 1 ]
[ t 1 1 1 t ]
[ 1 t t t 1 ]
[ t 1 1 t t ]

> addrow(",1,2,-t):addrow(",1,3,-t):B:=addrow(",1,4,-t);
B := [ 1 t t t ] 1
[ 0 -t2+1 -t2+1 -t2+1 ] 0
[ 0 -t2+1 -t2+1 -t2+1 -t+1 ]
[ 0 -t2+1 -t2+1 -t2+t 0 ]

> addrow(",2,3,-1):C:=addrow(",2,4,-1);
C := [ 1 t t t 1 ]
[ 0 -t2+1 -t2+1 -t2+1 0 ]
[ 0 0 0 0 -t+1 ]
[ 0 0 0 -1+t 0 ]

> D:=addrow(C,4,2,1+t):

> for i from 1 to 4 do for j from 1 to 5 do D[i,j]:=simplify(D[i,j]) od od:

> print(D);
[ 1 t t t 1 ]
[ 0 -t2+1 -t2+1 0 0 ]
[ 0 0 0 0 -t+1 ]
[ 0 0 0 -1+t 0 ]
At this point, take t not equal to 1 or -1, so that we may divide by 1-t, t-1, and 1-t2:
> mulrow(D,2,1/(-t2+1)):mulrow(",3,1/(-t+1)):F:=mulrow(",4,1/(-1+t));
F :=1 [ 1 t t t 1 ]
[ 0 1 1 0 0 ]
[ 0 0 0 0 1 ]
[ 0 0 0 1 0 ]

> addrow(",2,1,-t):addrow(",3,1,-1):addrow(",4,1,-t);
[ 1 0 0 0 0 ]
[ 0 1 1 0 0 ]
[ 0 0 0 0 1 ]
[ 0 0 0 1 0 ]
So, if t is not equal to 1 or -1, the reduced-row echelon form of the matrix has four columns containing leading 1's, and thus has rank four.

If t is equal to 1, we return to matrix D:


> t:=1:

> for i from 1 to 4 do for j from 1 to 5 do D[i,j]:=simplify(D[i,j]) od od:

> print(D);
[ 1 1 1 1 1 ]
[ 0 0 0 0 0 ]
[ 0 0 0 0 0 ]
[ 0 0 0 0 0 ]
In this case, there is only one leading 1, so the rank of the matrix is 1.

If t is equal to -1, we again use matrix D (reset to its dependence on t):


> t:=-1:

> for i from 1 to 4 do for j from 1 to 5 do D[i,j]:=simplify(D[i,j]) od od:

> gaussjord(D);
[ 1 -1 -1 0 0 ]
[ 0 0 0 1 0 ]
[ 0 0 0 0 1 ]
[ 0 0 0 0 0 ]
So, if t = -1, there are three leading 1's, and the rank of the matrix is three.