# Linear Combinations and Spans

Throughout this discussion I am assuming we are dealing with a real vector space V.  Every time I mention a collection of vectors I will assume that these vectors are in V.  Vectors will usually be in bold for example w or v.  Scalars will be in italics for example k or c.  Remember for real vector spaces scalars are just real numbers.   For complex vector spaces scalars are complex numbers.  Typical vector spaces are:

• Rn={all n-tuples with real entries}={ (x1,x2, … ,xn)  |  xi is a real number}
• Mmn={all m by n matrices with real entries}
• Pn={all polynomials of degree less than or equal to n}
• C(a,b)={set of all continuous functions on the interval (a,b)}
• Cr(a,b)={set of all functions which have r continuous derivatives on (a,b)}

Subsets are denoted by Í.  Subspaces will be denoted £.

Definition:  A vector w is a linear combination of the vectors v1, v2, … , vr, if it can be expressed in the form  where k1, k2, … , kr are scalars If r = 1 then we have w= k1v1, and we say that w is a scalar multiple of v1.

Lemma 1:  If a nonzero vector w is a scalar multiple of v then v is a scalar multiple of w.

Proof:

 Think Write Remember in a proof always start the proof by feeding back the given which is called the hypothesis Since w is a scalar multiple of w then w= kv for some scalar k. Check to make sure have used all of the given.  We never used that w is nonzero.  What does that tell us? Since w is nonzero then k must be nonzero because if k were zero we would have:  w= kv=0v=0. We want to write v as a scalar multiple of w.  This means we need to write v equals. Since k is nonzero we can write v=(1/k)w Make sure to state your conclusion. Therefore v is a scalar multiple of w.

Ex. 1:  -14+9x+4x2 is a linear combination of 2+3x and 5-x2 since
-14+9x+4x2=3(2+3x)+-4(5-x2).

Ex. 2:  -14+8x+4x2 is not a linear combination of 2+3x and 5-x2 since if
-14+8x+4x2= k1(2+3x)+ k2(5-x2) then this could be rearranged to
-14+8x+4x2=(2k1+5k2)+(3k1)x+(-k2)x2.  Two polynomials are equal if their coefficients are equal.  So the above equation becomes 3 equations and two unknowns.

 2 k1 + 5 k2 = -14 3 k1 8 -1 k2 1

This system has no solution (you can eyeball it or use matrices).  So there are scalars that work therefore -14+8x+4x2 is not a linear combination of 2+3x and 5-x2.

Since the scalars are found by solving a system of equations it is possible to have infinite solutions for the choice of scalars.

Ex 3:  since w=2v1+v2+v3=  -1v1+2v2+2v3.

When a vector can be written as two different linear combinations of a set of vectors then we say {v1,v2,v3} form a dependent set.  We will come back to this later.

Because saying "vector w is/is not a linear combination of the vectors v1, v2, … , vr" is so wordy, mathematicians of course invented another word to use instead of this phrase.

Definition:  Span{v1, v2, … , vr} is the set of all linear combinations of v1, v2, … , vr.

To save me typing let S be the set of vectors v1, v2, … , vr.  In the definition above span is used as a noun.  It is a set too.  Notice that viäSpan{S} for 1£ I £ r  so S Í Span{S} Í V.  You can also take the span of V, but Span{V}=V.  Now if vector w is a linear combination of the vectors v1, v2, … , vr we will write wÎSpan{S}.  If vector w is not a linear combination of the vectors v1, v2, … , vr we will write wÏSpan{S}.

Theorem 2:  If S is the set of vectors v1, v2, … , vr in V then,

1. Span{S} is a subspace of V.
2. If any subspace W of V contains S then W also contains Span{S}.  ie. If W£V and SÍW then Span{S}£W.

Because of the second part of the Theorem Span{S} is said to be "the smallest subspace of V that contains S".  If we let U=Span{S} then we say v1, v2, … , vr span U.  Here the first span in Span{S} is used as a noun, but the second span is used as a verb.

Proof:  Part a.  To prove a subset is a subspace you must only show three things.

 Think Write You must show it is nonempty.  Can you show zero is in Span{S} Since 0=0v1+0v2+ … +0vr we have 0 is a linear combination of the vectors in S so it is in Span{S}.  Thus Span{S} is nonempty. You need to check closure.  Make sure to start with a hypothesis telling where things come from. Let w1 and w2 be vectors in Span{S}.  Let a be a real number. Interpret your hypothesis. This means that w1 and w2 are linear combinations of the vectors in S. Sometimes you need to further interpret.  Notice that you must use different letters for the scalars since if you used the same letters you would be implying that the scalars were equal which would mean the vectors would be equal. Therefore there exists scalars k1, k2, … , kr such that w1=k1v1+k2v2+ … + krvr and there are scalars m1, m2, … , mr such that w2=m1v1+m2v2+ … +mrvr. Now you need to show that w1 +a w2 is in Span{S}  What does this mean?  It means that this new vector can be written as a linear combination of the vectors in S.  This normally involves plug and chug. w1 +a w2=(k1v1+k2v2+ … + krvr) + a( m1v1+m2v2+ … +mrvr)=(k1+am1)v1 + (k2+am2)v2 + … + (kr+amr)vr .  The last equality followed because vectors satisfy all of our normal associative, commutative, and distributive rules under vector addition and scalar multiplication. Make a conclusion about closure. Thus w1 +a w2ÎSpan{S} so Span{S} is closed under vector addition and scalar multiplication. Make a conclusion about the Theorem Since Span{S} is nonempty and closed under vector addition and scalar multiplication it follows that Span{S} is a subspace of V.

Proof Part b.

 Think Write Remember in a proof always start the proof by feeding back the given which is called the hypothesis Let W be a subspace of V that contains S. To show that W contains Span{S} means you are showing Span{S}ÍW.  To do this you must start with an element in Span{S} and show it is also in W. Let w be a vector in Span{S}. Interpret Therefore there exists scalars k1, k2, … , kr such that w=k1v1+k2v2+ … + krvr. Somehow connect to given.  What does the given mean?  It means W is closed under vector addition and scalar multiplication.  Now this was for two vectors by induction if you add up r vectors in W the resulting vector is in W. Since the vectors viÎW and W is closed under vector addition and scalar multiplication it follows that w is in W. Conclude Therefore Span{S}ÍW.

Ex 4: Describe Span{(1,2)}.
Span{(1,2)}={ a(1,2) where a is a scalar}.  This is the straight line through the origin with slope 2.

Ex 5: Show that Span{ (1,0,3,4),  (3,0,-1,3) } Ì Span{ (1,0,3,4),  (3,0,-1,3),  (2,1,5,6) }
Since (1,0,3,4) and (3,0,-1,3) are in Span{ (1,0,3,4),  (3,0,-1,3),  (2,1,5,6) } part b of the theorem above tells us that Span{ (1,0,3,4),  (3,0,-1,3) } Í Span{ (1,0,3,4),  (3,0,-1,3),  (2,1,5,6) }.  We must still show that the sets cannot be equal.  To do this you must find one element of the larger set that is not in the smaller set or you must show that the "size" of the larger set is bigger than the "size" of the lower set.  Notice that (2,1,5,6) is not in Span{ (1,0,3,4),  (3,0,-1,3) } so the sets are not equal. If the third vector in the larger set had a 0 in the second component it would not be as easy to say it was not in the smaller set you would have to test whether or not it was like in Ex 1 and 2.

Ex 6:  Show Span{ (1,-2,3) , (5,6,-1), (3,2,1) }=Span{ (5,6,-1), (3,2,1) }
Like above it follows immediately from Theorem 2 that Span{ (1,-2,3) , (5,6,-1), (3,2,1) } Ê Span{ (5,6,-1), (3,2,1) }.  To finish the proof you must show that Span{ (1,-2,3) , (5,6,-1), (3,2,1) } Í Span{ (5,6,-1), (3,2,1) } too.  (Don't forget how to show subsets!)  Let wÎ Span{ (1,-2,3) , (5,6,-1), (3,2,1) }.  (Interpret what does this mean?) Therefore there exists scalars k1, k2, k3 such that w=k1(1,-2,3)+k2(5,6,-1)+ k3(3,2,1).  We need to show that we can find scalars that will be functions of k1, k2, k3 such that w=m1(5,6,-1)+m2(3,2,1).  This changes to a system:

 k1 + 5k2 + 3k3 = 5m1+3m2 -2 k1 + 6 k2 + 2 k3 = 6m1+2m2 3 k1 - k2 + 1 k3 = -1m1+1m2

Despite its backward form this is a system of 3 equations in two unknowns m1 and m2.  The left side of each of the above equations is just a real number; it is not unknown since the scalars are not unknown.  Solving this system we get m1= -k1+k2 and m2= 2k1+k3.  Since there is a solution to our system we could find the scalars thus wÎ Span{ (5,6,-1), (3,2,1) }.  This implies that Span{ (1,-2,3) , (5,6,-1), (3,2,1) } Í Span{ (5,6,-1), (3,2,1) }.  We therefore can conclude that Span{ (1,-2,3) , (5,6,-1), (3,2,1) }=Span{ (5,6,-1), (3,2,1) }

# Independent and Dependent Sets

When the span of a set of vectors does not change by removing one of its members we say that the larger set of vectors is said to be a dependent set or the vectors themselves are said to be dependent.  So the larger set of vectors { (1,-2,3) , (5,6,-1), (3,2,1) } in Example 6 form a dependent set.  To determine if the smaller set was a dependent set you would have to try removing a vector and see if the span changes.  Be careful if you remove one and the span does not change that does not mean it is not dependent.  You would have to then try removing a different one and check again.  Etc. until you have tried removing everyone.  If the span never changes then the set is not dependent.  This is a lousy way to check for dependency.

Before I can give you a better way I need to remind you about notation.  A subset T of a set S is called proper if it is nonempty and is not all of the vectors in S.  If you know T is a proper subset of S you should write TÌS; if you don't know if it is proper or you don't care you write TÍS.

Definition: A set S of vectors v1, v2, … , vr is a dependent set of vectors if there exists a choice of scalars k1, k2, … , kr with at least one scalar not being zero such that k1v1+k2v2+ … + krvr=0.

Sometimes it is easier to use the following:  The set S is a dependent set if any of the following occur.

a.      The Span{S}=Span{T} where T is some proper subset of S.  OR

b.      If one vector in S can be written as a linear combination of the others.  OR

c.      S contains the zero vector.  OR

Choice a and b could have actually have been used for the definition; In other words the definition is equivalent to bullet a which is equivalent to bullet b.  Bullet c is not equivalent to the definition.  Bullet c says if you know your set contains the zero vector you know it is dependent, but if you know it does not contain the zero vector you don't know anything.

• Bullet c implies definition.  If S contains the zero vector then for the purpose of this proof we can assume that the zero vector is v1 so we have 50+0v2+ … +0vr=0.
• definition implies bullet b.  In the definition assume that the nonzero scalar is kd.  Then the equation in the definition can be rewritten to vd=(k1/kd)v1+ …+(kd-1/kd)vd-1 +(kd+1/kd)vd+1+ … +(kr/kd)vr  . So vd is a linear combination of the others.
• b implies definition.  Assume that vd is a linear combination of the others. So vd=m1v1+ …+md-1vd-1 +md+1vd+1+ … +mrvr.  Rearranging we obtain m1v1+ …+md-1vd-1 +(-1)vd+md+1vd+1+ … +mrvr=0.
• definition implies bullet a.  Since the definition implies bullet b it suffices to show that bullet b implies bullet a.  Assume that vd is a linear combination of the others.  Let T be the set of all vectors in S except for vd.  By Theorem 2 we have that Span{T}ÍSpan{S}.  We must show that Span{S}}ÍSpan{T}.    Let wÎSpan{S} then there exists scalars k1, k2, … , kr such that w=k1v1+k2v2+ … + krvr.  Substituting our equation for vd into this last equation gives w as a linear combination of the vectors in T.  In other words wÎSpan{T}.  We conclude that Span{T}=Span{S}.
• Bullet a implies definition.  Since the bullet b implies the definition it suffices to show that bullet a implies bullet b.  So we are given that there exists some proper subset T of S with Span{T}=Span{S}.  Since T is proper there is at least one vector in S that is not in T.  Assume it is vd.  Since vdÎSÍSpan{S}=Span{T} it follows that vd is a linear combination of the vectors in T.  But since the vectors in T are also in S it follows that this linear combination can also be viewed as a linear combination of vectors in S.  This is just the statement in bullet b.  So we are done.

Definition:  A set is called independent if it is not dependent.  In other words a set is independent if

1. The Span{T}ÌSpan{S} for every proper subset T of S OR
2. If no vector in S can be written as a linear combination of the others.  OR
3. If you try to solve k1v1+k2v2+ … + krvr=0 for the scalars the only solution is k1=k2=…=0.

Now don't forget all of Chapter 1 and 2.  If the equation in bullet c in the definition of independent translates to r equations and r unknowns it has a unique solution iff the determinant is nonzero; thus if the determinant is nonzero the set is independent and if the determinant is zero the set will be dependent.  If instead it translates to m equations and r unknowns where r is bigger than m then there are infinite solutions so the set will be dependent.

Ex 7:  Determine if the set of vectors { (1,-2,3), (5,6,-1), (3,2,1)} is independent or dependent a different way then in Example 6.
Assume we can find scalars k1, k2, k3 such that k1(1,-2,3)+k2(5,6,-1)+ k3(3,2,1)=0=(0,0,0).  This becomes (k1+5k2+3k3, -2k1+6k2+2k3, 3k1-k2+k3)=(0,0,0). Which translates to three equations and three unknowns by doing first coordinate equals zero, second coordinate equals zero, etc.

 k1 + 5k2 + 3k3 = 0 -2 k1 + 6 k2 + 2 k3 = 0 3 k1 - k2 + 1 k3 = 0

So we can use determinants:  .  Thus the vectors are dependent.  Much easier, right?  Please notice that the vectors given appear as columns in our matrix or determinant not as rows.

Ex 8:  Determine if the set of vectors {sin2x, cos2x, 5} is a independent or dependent set.
We start the same way.  Assume we can find scalars k1, k2, k3 such that k1sin2x+k2cos2x+k35=0 regardless of the value of x.  You cannot change the scalars if x changes.  In other words you actually have infinite equations: k1sin2p+k2cos2p+k35=0, k1sin22p+k2cos22p+k35=0,  k1sin20+k2cos20+k35=0 k1sin2(p/2)+k2cos2(p/2)+k35=0  etc. Even though we used four values of x we only really got two different equations.  This is not a good approach.  Maybe you remember your trig and "see" the answer as dependent because 5sin2x+5cos2x+(-1)5=0.  If you don't and the functions are differentiable you can try to see if the Wronskian is helpful.

Theorem 3:  Given a set of functions f1,f2,…,fr in C(n-1) (,¥).  If the set is dependent then

Proof:  To same me typing I will assume n=2 if n is bigger the same argument works but with more typing.  Since the set is dependent we can find scalars k and m at least one of which is nonzero such that kf+mg=0  The 0 vector is the function Zero(x)=0 for all real x in the interval (,¥).  So our equation becomes kf(x)+mg(x)=0 for all real x in the interval (,¥).  Since either k or m is nonzero the system has a nontrivial solution for every x in the interval (,¥).  This implies that for every x in (,¥) the coefficient matrix is not invertible, or equivalently that its determinant the Wronskian is zero for every x in (,¥).  Thus if the functions are linearly dependent in C1(,¥) then W(x)=Zero(x)=0 for every x in the interval (,¥).

Lemma 4:  If there exists an x such that W(x)¹0 then the functions are independent.

Proof:  If they were dependent then W(x)=0 contradiction.

Remark:  Although Theorem 3 and Lemma 4 are often helpful you must remember that the converse of Theorem 3 is not true.  In other words If W(x)= 0 for all x in an interval [a,b] it does not mean the functions are dependent in Cr[a,b].

Ex 9:  Show that if f=sin2x and g=|sinx|sinx then W(x)=0 on the interval [-1,1] but the vectors are independent on [-1,1].

.  Remember to think in radians.  Since on the interval [0,1] sinx and sin2x are both positive the absolute values are not needed and then it is clear that W(x)=0.  On the interval [-1,0) sinx is negative and sin2x is negative so the expression becomes W(x)=sin2x(-sin2x)-sinx sin2x(-sinx)=0.  So we have W(x)=0.  Now if we just look at kf(x)+mg(x)=0 when x=1 and when x= -1 we obtain the two equations k+m=0 and k-m=0 which imply k=m=0.  Thus the functions are independent.

Ex 10:  Show that f=sin2x, g=cos2x, h=5 is an dependent set and W(x)=0.
We already showed dependence in Ex 8.

Ex 11:  Determine if the vectors f=x and g=sinx form a dependent set or independent set of vectors in C1(,¥).
Calculate the Wronskian.  .  This function has value -p when x=p so it is not identically zero.  Lemma 4 tells us that the functions must therefore be independent.