A set of vectors is said to be linearly independent provided the equation
has no solution except .
This can be used to show that when you pick one of the vectors , it can not be expressed as a sum over the rest. There usually is a largest number of independent vectors:
The dimension of a space is the largest possible number of linearly independent vectors which can be found in the space.
Any set of linearly independent vectors in an -dimensional space is said to form a complete set of basis vectors, since one can show that any vector in the space can be expanded in the form
(2.1) |
where the numbers are called the components of in the basis .
Example 2.4:
Show that the vectors , and are linearly independent. Find the component of a general vector in this basis.
Solution:
We get the three coupled equations
These have a unique solution if the determinant is not zero,
This is true, since the determinant equals . The components are found by solving the equations (Gaussian elimination is quite easy in this case):
Theorem 2.1. The decomposition ( 2.1 ) is unique.
Proof. Suppose that there is a second decomposition, . Subtract the left- and right-hand sides of the two decompositions, collecting terms:
Linear independence of the vectors implies that , which contradicts our assumption of a second decomposition, and thus it is unique. □
Let us look at an example in an infinite dimensional space:
Example 2.5:
The Fourier decomposition of a function defined only on the interval is given by
This means that for the Fourier series the basis functions are:
It is highly non-trivial (i.e., quite hard) to show that this basis is complete! This is a general complication in infinite dimensional spaces.
For any two vectors we can define a scalar product 1 [the mathematicians’ preferred notation for ], which satisfies:
together with
(2.4) |
where the equality holds for only.
Note: Mathematically the inner product is a mapping from !
Note: In functional spaces we physicists often use the Dirac bra-ket notation for . We can use the inner product to define the norm (a more correct description of the length) of the vector ,
One can define a length without an inner product. A good example is the so-called “1-norm” of a vector, , which is used quite commonly in numerical analysis of linear algebra.
One of the important uses of an inner product is to test whether two vectors are at straight angles:
The triangle inequality is a trivial statement that the length of the sum of two vectors is less than the sum of the lengths,
From the triangle inequality we can prove the Cauchy Schwartz inequality
Proof. The proof is simple. Consider , which is . Minimise this with respect to , and we find a minimum for . At that point the function takes on the value . Multiply this by and take the square root at both sides for the desired result. □
There are two ways to turn an arbitrary set of vectors into an orthogonal set–one where every pair of vectors is orthogonal–, or even better orthonormal set–an orthogonal set where each vector has length one.
The most traditional approach is the Gramm-Schmidt procedure. This procedure is defined recursively. In the th step of the algorithm one defines the vector that is orthonormal to the orthonormal vectors defined in previous steps. Thus
The first line above removes all components parallel to the previous normalised and orthogonal vectors (check!), the second step normalises the result, so that is normalised.
A more modern approach (based on extensive use of numerical linear algebra) is based on the construction of the “overlap matrix” (also called norm matrix, which is why we use the symbol) , with entries . This matrix is Hermitian (or symmetric if the basis is real), and we can now define a matrix , such that . This can then be used to define the orthonormal basis
For a
real symmetric matrix
(and similarly for a Hermitian one, but we shall
concentrate on the first case here) we can define
matrix powers in a simple and unique way by requiring that the
powers are also symmetric matrices.
The easiest way to get the result is
first to diagonalise the matrix
, i.e., to find its eigenvalues
and eigenvectors
. We can then write
, with
the matrix with the normalised eigenvectors as columns,
. The eigenvectors are orthonormal, and thus
. The matrix
has the eigenvalues
on the diagonal, and is zero elsewhere. (Convince
yourself that
and
.)
We then define arbitrary powers of
by
Orthonormal basis functions: For discrete 2 vector spaces one can always choose an orthonormal set of basis functions satisfying
(2.5) |
Here we have introduced the Kronecker delta , defined for integer . This object is zero unless , when it is 1.
(2.6) |
Use the definition of independence to show that Eq. ( 2.1 ) holds for any set of independent functions.