STK 500. Pengantar Teori Statistika. Eigenvalues and Eigenvectors

STK 500 Pengantar Teori Statistika Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors Example 1: if we have a matrix A: A  2 4   4 -4 

then

A  I   2 4    1 0   4 -4  0 1  4  2   4    16  0  2-   4 -4 -   or  2  2  24  0

which implies there are two roots or eigenvalues : =-6 and =4.

Eigenvalues and Eigenvectors  For

a square matrix A, let I be a conformable identity matrix. Then the scalars satisfying the polynomial equation |A - I| = 0 are called the eigenvalues (or characteristic roots) of A.  The equation |A - I| = 0 is called the characteristic equation or the determinantal equation.

Eigenvalues and Eigenvectors 

For a matrix A with eigenvectors , a nonzero vector x such that Ax = x is called an eigenvector (or characteristic vector) of A associated with .

Example 1 if we have a matrix A:

A  2 4   4 -4 

with eigenvalues  = -6 and  = 4, the eigenvector of A associated with  = -6 is Ax  x   2 4   x1   6  x1   4 -4   x 2  x 2   2x1  4x 2  6x1  8x1  4x 2  0 and  4x1  4x 2  6x 2  4x1  2x 2  0

Fixing x1=1 yields a solution for x2 of –2.

Example 1 Note that eigenvectors are usually normalized so they have unit length, i.e., e

x

x'x

For our previous example we have: e

x x'x



 1 -2

 1  1  -2  5    -2  5  1  1 -2   5  -2

Thus our arbitrary choice to fix x1=1 has no impact on the eigenvector associated with  = -6.

Example 1 For matrix A and eigenvalue  = 4, we have Ax  x   2 4   x1   4  x1   4 -4   x 2  x 2   2x1  4x 2  4x1  2x1  4x 2  0 and  4x1  4x 2  4x 2  4x 1  8x 2  0

We again arbitrarily fix x1=1, which now yields a solution for x2 of 1/2.

Normalization of Eigenvectors Normalization to unit length yields

e

x x'x



 1 1   2

 1 1  2    5 1 1   11  4 2 2   

 1 1   2   2   5  1  5  5  2

Again our arbitrary choice to fix x1=1 has no impact on the eigenvector associated with  = 4.

Example 2 

Find the eigenvalues and corresponding eigenvectors for the matrix A. What is the dimension of the eigenspace of each eigenvalue?  2 1 0 A   0 2 0 0 0 2 Characteristic equation:  2 I  A 

Eigen value :

0 0 2

1 0  2 0  (  2)3  0 0  2

Example 2 The eigenspace of λ = 2: 0 1 0   x1  0 ( I  A)x  0 0 0   x2   0  0 0 0   x3  0  x1   s  1 0  x2   0   s 0   t 0  , s , t  0         x t  3   0 1

 1   0         s 0  t 0 s , t  R       : the eigenspace of A corresponding to   2  0  1        

Thus, the dimension of its eigenspace is 2.

Eigenvalues and Eigenvectors (1) If an eigenvalue 1 occurs as a multiple root (k times) for the characteristic polynominal, then 1 has multiplicity k. (2) The multiplicity of an eigenvalue is greater than or equal to the dimension of its eigen space.

Example 3 Find the eigenvalues and corresponding eigenspaces for 1 3 0  A   3 1 0  0 0 2 

0    1  3 I  A    3   1 0   (  2)2 (  4)  0   0   2  0  eigenvalues 1  4, 2  2 The eigenspaces for these two eigenvalues are as follows.

B1  {(1, 1, 0)}

Basis for 1  4

B2  {(1,  1, 0), (0, 0, 1)}

Basis for 2  2

Example 4  Find

the eigenvalues of the matrix A and find a basis for each of the corresponding eigenspaces 0  1 0 0 A  0 1 5  10  1 0 0 0 0  1 0 2 3  1 0 0 0   1 5 10 I  A  1 0  2 0 Characteristic equation: 1

0

0

 3

 (  1) 2 (  2)(  3)  0

Eigenvalues: 1  1, 2  2, 3  3 According to the note on the previous slide, the dimension of the eigenspace of λ1 = 1 is at most to be 2  For λ2 = 2 and λ3 = 3, the dimensions of their eigenspaces are at most to be 1

Example 4 0 0 (1) 1  1  (1 I  A)x    1   1

0 0 0   x1  0  0 5 10   x2  0   0 1 0   x3  0      0 0 2   x4  0 

 0    2    x1   2t  0  2      x   s  1   0  1  0   2        s    t   , s,t  0  0,  2    x3   2t  0   2                0  1   x t 0  1   4  

is a basis for the eigenspace corresponding to 1  1

The dimension of the eigenspace of λ1 = 1 is 2

Example 4 1 0 (2) 2  2  (2 I  A)x    1   1

0 0 0   x1  0  1 5 10   x2  0     0 0 0 0  x3     0 0 1  x4  0 

 x1   0  0  x  5t  5    2     t  , t  0  x3   t  1         x4   0  0  0       5      is a basis for the eigenspace 1  corresponding to 2  2     0  

The dimension of the eigenspace of λ2 = 2 is 1

Example 4 2 0 (3) 3  3  (3 I  A)x    1   1

0 0 0   x1  0  2 5 10   x2  0     0  0 1 0  x3     0 0 0   x4  0 

 x1   0   0   x   5t   5   2     t  , t  0  x3   0   0         x4   t   1   0         5    is a basis for the eigenspace  0   corresponding to 3  3  1  

The dimension of the eigenspace of λ3 = 3 is 1

Eigenvalues and Eigenvectors Theorem 1. Eigenvalues for triangular matrices If A is an nn triangular matrix, then its eigenvalues are the entries on its main diagonal 

Finding eigenvalues for triangular and diagonal matrices

2 0 0 (a) A   1 1 0   5 3 3

 2 (a)  I  A 

1 5

0 0   1 0  (  2)(  1)(  3)  0 3   3

 1  2, 2  1, 3  3

Eigenvalues and Eigenvectors 

Finding eigenvalues for triangular and diagonal matrices  1 0  (b) A   0  0  0

0 2 0 0 0

0 0 0 0 0 0 0 4 0 0

0 0  0  0 3 

(b) 1  1, 2  2, 3  0, 4  4, 5  3

Diagonalization Definition 1: A square matrix A is called diagonalizable if there exists an invertible matrix P such that P–1AP is a diagonal matrix (i.e., P diagonalizes A) Definition 2: A square matrix A is called diagonalizable if A is similar to a diagonal matrix

Diagonalization Theorem 2: Similar matrices have the same eigenvalues If A and B are similar nn matrices, then they have the same eigenvalues A and B are similar  B  P1AP

For any diagonal matrix in the form of D = λI, P–1DP = D

Considering the characteristic equation of B:

 I  B   I  P 1 AP  P 1 IP  P 1 AP  P 1 ( I  A) P  P 1  I  A P  P 1 P  I  A  P 1P  I  A  I  A Since A and B have the same characteristic equation, they are with the same eigenvalues

Example 5 

Eigenvalue problems and diagonalization programs Characteristic equation:  1

 I  A  3 0

1 3 0  A  3 1 0    0 0  2  

3 0  1 0  (  4)(  2) 2  0 0 2

The eigenvalues : 1  4, 2  2, 3  2

1    (1)   4  the eigenvector p1  1   0 

Example 5 (2)   2  the eigenvector

P  [p1 p 2

If

P  [p 2

1 0 p 2   1 , p3  0   0  1 

1 1 0  4 0 0  p3 ]  1 1 0  , and P 1 AP  0 2 0  0 0 1  0 0 2 p1 p3 ]

 1 1 0  2 0 0    1 1 0   P 1 AP   0 4 0   0 0 1   0 0 2   The above example can prove Thm. 2 numerically since the eigenvalues for both A and P–1AP are the same to be 4, –2, and –2  The reason why the matrix P is constructed with eigenvectors of A is demonstrated in Thm. 3 on the next slide

Diagonalization Theorem 3: An nn matrix A is diagonalizable if and only if it has n linearly independent eigenvectors  Note that if there are n linearly independent eigenvectors, it does not imply that there are n distinct eigenvalues. It is possible to have only one eigenvalue with multiplicity n, and there are n linearly independent eigenvectors for this eigenvalue  However, if there are n distinct eigenvalues, then there are n linearly independent eigenvectors and thus A must be diagonalizable

Since A is diagonalizable, there exists an invertible P s.t. D  P 1 AP is diagonal. Let P  [p1 p 2 p n ] and D  diag (1 , 2 , , n ), then

()

1 0 0  2 PD  [p1 p 2 pn ]    0 0  [1p1 2p 2 np n ]

0 0    n 

Diagonalization AP  PD (since D  P 1 AP ) [ Ap1 Ap 2 Ap n ]  [1p1 2p 2  Api  i pi , i  1, 2,

np n ]

,n

(The above equations imply the column vectors pi of P are eigenvectors of A, and the diagonal entries i in D are eigenvalues of A)

Because A is diagonalizable  P is invertible  Columns in P, i.e., p1 , p 2 , , p n , are linearly independent (see p. 4.101 in the lecture note or p. 246 in the text book) Thus, A has n linearly independent eigenvectors

() Since A has n linearly independent eigenvectors p1 , p 2 , with corresponding eigenvalues 1 , 2 , n , then Let P  [p1 p2  Api  i pi , i  1, 2, , n

pn

pn ]

Diagonalization AP  A[p1 p 2  [1p1 2p 2  [p1 p 2

Since p1 , p 2 ,

p n ]  [ Ap1 Ap 2

Ap n ]

np n ] 1 0 0  2 pn ]    0 0

0 0   PD   n 

, p n are linearly independent

 P is invertible (see p. 4.101 in the lecture note or p. 246 in the text book) AP  PD  P 1 AP  D  A is diagonalizable (according to the definition of the diagonalizable matrix on Slide 7.27)

Note that pi 's are linearly independent eigenvectors and the diagonal entries i in the resulting diagonalized D are eigenvalues of A

Example 6 

A matrix that is not diagonalizable Show that the following matrix is not diagonalizable

1 2  A  0 1  

 1

2 I  A   (  1)2  0 0  1 The eigenvalue 1  1, and then solve (1I  A)x  0 for eigenvectors

Characteristic equation:

0 2 1  1I  A  I  A    eigenvector p1     0 0  0 

Since A does not have two linearly independent eigenvectors,

A is not diagonalizable.

Diagonalization 

Steps for diagonalizing an nn square matrix: Step 1: Find n linearly independent

eigenvectors

p1 , p2 ,

pn for A

with corresponding eigenvalues 1 , 2 , , n pn ] Step 2: Let P  [p1 p2 Step 3: 1 0  0 

0 P AP  D     0 1

0     n 

2   0

where Api  i pi , i  1, 2,

,n

Example 6 

Diagonalizing a matrix  1  1  1 A 1 3 1    3 1  1 Find a matrix P such that P 1 AP is diagonal.

Characteristic equation:  1

 I  A  1 3

1 1   3 1  (  2)(  2)(  3)  0 1   1

The eigenvalues : 1  2, 2  2, 3  3

Example 6 1 1 1 1 0 1  G.-J. E. 1  2  1 I  A   1 1 1   0 1 0   3 1 3  0 0 0   x1   t  x    0   2    x3   t 

2

 1  eigenvector p1   0   1 

1 1 0  14   3 1 G.-J. E. 1   2  2 I  A   1 5 1   0 1 4  0 0 0   3 1 1  x1   14 t  1  x     1 t   eigenvector p   1 2  2  4     x3   t   4 

Example 6 2 1 1 1 0 1  0 1 1 G.-J. E. 3  3  3 I  A   1 0 1        3 1 4  0 0 0   x1   t  x    t   2    x3   t  P  [p1

p2

2 P 1 AP   0  0

 1  eigenvector p3   1   1 

 1 p3 ]   0  1 0 0 2 0  0 3 

1 1 4

1 1  and it follows that 1 

Diagonalization Note: a quick way to calculate Ak based on the diagonalization technique 1 0 0  2 (1) D     0 0

0 1k  0  0 k   D      n   0

(2) D  P 1 AP  D k  P 1 AP P 1 AP

0

2k 0

P 1 AP  P 1 Ak P

repeat k times

1k  0 k k 1 k  A  PD P , where D     0

0

2k 0

0  0  k n 

0  0  k n 

Diagonalization Theorem 4: Sufficient conditions for diagonalization If an nn matrix A has n distinct eigenvalues, then the corresponding eigenvectors are linearly independent and thus A is diagonalizable. Proof : Let λ1, λ2, …, λn be distinct eigenvalues and corresponding eigenvectors be x1, x2, …, xn. In addition, consider that the first m eigenvalues are linearly independent, but the first m+1 eigenvalues are linearly dependent, i.e., xm1  c1x1  c2 x2   cm xm , (1) where ci’s are not all zero. Multiplying both sides of Eq. (1) by A yields Axm1  Ac1x1  Ac2 x2   Acm xm

m1xm1  c11x1  c22 x2 

 cm m xm

(2)

Diagonalization On the other hand, multiplying both sides of Eq. (1) by λm+1 yields

m1xm1  c1m1x1  c2m1x2 

 cmm1xm

(3)

Now, subtracting Eq. (2) from Eq. (3) produces

c1 (m1  1 )x1  c2 (m1  2 )x2 

 cm (m1  m )xm =0

Since the first m eigenvectors are linearly independent, we can infer that all coefficients of this equation should be zero, i.e.,

c1 (m1  1 )  c2 (m1  2 ) 

 cm (m1  m )=0

Because all the eigenvalues are distinct, it follows all ci’s equal to 0, which contradicts our assumption that xm+1 can be expressed as a linear combination of the first m eigenvectors. So, the set of n eigenvectors is linearly independent given n distinct eigenvalues, and according to Thm. 3, we can conclude that A is diagonalizable.

Example 7 

Determining whether a matrix is diagonalizable 1  2 1  A  0 0 1 0 0  3

Because A is a triangular matrix, its eigenvalues are 1  1, 2  0, 3  3.

According to Thm. 4, because these three values are distinct, A is diagonalizable.

Symmetric Matrices and Orthogonal Diagonalization A square matrix A is symmetric if it is equal to its transpose: A  AT 

Example symmetric matrices and nonsymetric matrices  0 1  2 A 1 3 0     2 0 5   4 3 B 3 1 3 2 1 C  1  4 0   1 0 5  

(symmetric) (symmetric) (nonsymmetric)

Symmetric Matrices and Orthogonal Diagonalization Thm 5: Eigenvalues of symmetric matrices If A is an nn symmetric matrix, then the following properties are true. a) A is diagonalizable (symmetric matrices are guaranteed to has n linearly independent eigenvectors and thus be diagonalizable). b) All eigenvalues of A are real numbers c) If  is an eigenvalue of A with multiplicity k, then  has k linearly independent eigenvectors. That is, the eigenspace of  has dimension k. The above theorem is called the Real Spectral Theorem, and the set of eigenvalues of A is called the spectrum of A.

Example 8 

Prove that a 2 × 2 symmetric matrix is diagonalizable. a c  A  c b

Proof: Characteristic equation: I  A 

 a c

c  2  (a  b)  ab  c 2  0  b

As a function in , this quadratic polynomial function has a nonnegative discriminant as follows (a  b) 2  4(ab  c 2 )  a 2  2ab  b 2  4ab  4c 2  a 2  2ab  b 2  4c 2  (a  b) 2  4c 2  0

Example 8 (1) (a  b) 2  4c 2  0  a  b, c  0

a c  a 0 A  itself is a diagonal matrix.    c b 0 a  (2) (a  b) 2  4c 2  0

The characteristic polynomial of A has two distinct real roots, which implies that A has two distinct real eigenvalues. According to Thm. 5, A is diagonalizable.

Symmetric Matrices and Orthogonal Diagonalization 

Orthogonal matrix : A square matrix P is called orthogonal if it is invertible and P1  PT (or PPT  PT P  I )

Thm. 6: Properties of orthogonal matrices An nn matrix P is orthogonal if and only if its column vectors form an orthonormal set. Proof: Suppose the column vectors of P form an orthonormal set, i.e.,

P  p1 p2

pn  , where pi  p j  0 for i  j and pi  pi  1.

 p1T p1 p1T p 2 p1T p n   p1  p1 p1  p 2  T   T T p 2  p1 p 2  p 2 p p p p p p T 2 1 2 2 2 1   P P      T   T T p n p n  p n  p1 p n  p 2 p n p1 p n p 2 It implies that P–1 = PT and thus P is orthogonal.

p1  p n  p 2  p1   In   pn  pn 

Example 9 

Show that P is an orthogonal matrix.  13  2 P 5  2 3 5

2 3 1 5 4 3 5

  0 5  3 5 2 3

If P is a orthogonal matrix, then P1  PT  2 2   1 2 2  1 3 1 5 3 5 3 3 3     T 1 2 1 4 PP   25 0     0 3 5 5 3 5  2  2  0 5 5 4  3 5 3 5 3 5   3 0 3 5  

PPT  I

0 0 1 0   I 0 1 

Symmetric Matrices and Orthogonal Diagonalization  1   2  2   3  3   3    2 1 Moreover, let p1   5  , p 2   5  , and p3   0  ,  2   4   5  3 5   3 5   3 5  we can produce p1  p 2  p1  p3  p 2  p3  0 and p1  p1 

p 2  p 2  p3  p3  1. So, {p1 , p 2 , p3} is an orthonormal set. (Thm. 7.8 can be illustrated by this example.)

Symmetric Matrices and Orthogonal Diagonalization Thm. 7: Properties of symmetric matrices Let A be an nn symmetric matrix. If 1 and 2 are distinct eigenvalues of A, then their corresponding eigenvectors x1 and x2 are orthogonal. (Thm. 6 only states that eigenvectors corresponding to distinct eigenvalues are linearly independent

1 (x1  x2 )  (1x1 )  x2  ( Ax1 )  x2  ( Ax1 )T x2  (x1T AT )x2

Proof :

because A is symmetric



(x1T A)x2  x1T ( Ax2 )  x1T (2 x2 )  x1  (2 x2 )  2 (x1  x2 )

The above equation implies (1  2 )(x1  x2 )  0, and because

1  2 , it follows that x1  x2  0. So, x1 and x2 are orthogonal.  For distinct eigenvalues of a symmetric matrix, their corresponding eigenvectors are orthogonal and thus linearly independent to each other  Note that there may be multiple x1 and x2 corresponding to 1 and 2

Symmetric Matrices and Orthogonal Diagonalization A matrix A is orthogonally diagonalizable if there exists an orthogonal matrix P such that P–1AP = D is diagonal. Thm.8: Fundamental theorem of symmetric matrices Let A be an nn matrix. Then A is orthogonally diagonalizable and has real eigenvalues if and only if A is symmetric. Proof: () A is orthogonally diagonalizable  D  P 1 AP is diagonal, and P is an orthogonal matrix s.t. P 1  PT  A  PDP 1  PDPT  AT  ( PDPT )T  ( PT )T DT PT  PDPT  A

() See the next two slides

Symmetric Matrices and Orthogonal Diagonalization Let A be an nn symmetric matrix. (1) Find all eigenvalues of A and determine the multiplicity of each. According to Thm. 7, eigenvectors corresponding to distinct eigenvalues are orthognoal (2) For each eigenvalue of multiplicity 1, choose a unit eigenvector. (3) For each eigenvalue of multiplicity k  2, find a set of k linearly independent eigenvectors. If this set {v1, v2, …, vk} is not orthonormal, apply the Gram-Schmidt orthonormalization process.

Symmetric Matrices and Orthogonal Diagonalization (4) The composite of steps (2) and (3) produces an orthonormal set of n eigenvectors. Use these orthonormal and thus linearly independent eigenvectors to form the columns of P. i. According to Thm. 7, the matrix P is orthogonal ii. Following the diagonalization process, D = P–1AP is diagonal iii. therefore, the matrix A is orthogonally diagonalizable

Example 10 

Determining whether a matrix is orthogonally diagonalizable Symmetric Orthogonally matrix diagonalizable 1 1 1 A1  1 0 1   1 1 1

 5 2 1 A2   2 1 8     1 8 0

 3 2 0 A3   2 0 1 0 0  A4   0  2

Example 11 

Orthogonal diagonalization Find an orthogonal matrix P that diagonalizes A. 2 A   2  2

2 2  1 4  4 1

Sol: (1) I  A  (  3) 2 (  6)  0 1  6, 2  3 (has a multiplicity of 2) v1 (2) 1  6, v1  (1,  2, 2)  u1   ( 13 , v1

(3) 2  3, v2  (2, 1, 0), v3  (2, 0, 1)

2 3

,

2 3

)

Verify Thm. 7 that v1·v2 = v1·v3 = 0 Linearly independent but not orthogonal

Example 11 Gram-Schmidt Process:

v3  w 2 w 2  v 2  (2, 1, 0), w 3  v3  w 2  ( 52 , 54 , 1) w2  w2 w3 w2 2 1 u2   ( 5 , 5 , 0), u3   ( 325 , 3 45 , 3 55 ) w2 w3

Verify Thm. 7 that after the Gram-Schmidt orthonormalization process, i) w2 and w3 are eigenvectors of A corresponding to the eigenvalue of 3, and ii) v1·w2 = v1·w3 = 0 2  13   2 15 P3 5  23 0  u1 u2

2 3 5 4 3 5 5 3 5

u3

    

  6 0 0  P 1 AP   0 3 0   0 0 3  

Beberapa Teorema Akar Ciri dan Vektor Ciri Jika λ adalah akar ciri matriks A dengan vektor ciri padanannya x, maka untuk fungsi tertentu g(A) akan mempunyai akar ciri g(λ) dengan x vektor ciri padanannya.  Kasus khusus : 1. Jika λ adalah akar ciri matriks A, maka c‫ ג‬adalah akar ciri matriks cA dengan c≠0 sebarang skalar Bukti : c A x = c λ x x vektor ciri padanan λ dari matriks A x vektor ciri padanan c λ dari matriks cA 

Beberapa Teorema Akar Ciri dan Vektor Ciri 2. Jika ‫ ג‬adalah akar ciri matriks A, dengan x vektor ciri padanannya, maka c‫ג‬+k adalah akar ciri matriks (cA+kI) dengan x vektor ciri padanannya. Bukti : c A x + k x = c λ x + k x (c A + k I)x = (c λ + k) x (tidak dapat diperluas untuk A + B, dengan A , B sebarang matriks n x n ) 3. λ2 adalah akar ciri dari matriks A2 (dapat diperluas untuk Ak) Bukti : A x = λ x A(A x) = A(λ x ) A2 x= λ Ax= λ(λ x )= λ 2 x

Beberapa Teorema Akar Ciri dan Vektor Ciri 4. 1/ λ adalah akar dari matriks A-1 Bukti : A x = λ x A-1 (A x) = A-1(λ x ) x= λ A-1x A-1x = λ-1 x 5. Kasus (1) dan (2) dapat digunakan untuk mencari akar ciri dan vektor ciri dari polinomial A Contoh : (A3 + 4 A2 -3 A + 5 I ) x = A3x + 4 A2 x -3 Ax + 5 x = λ 3x + 4 λ 2 x -3 λ x + 5 x =(λ3 + 4λ 2 -3 λ + 5)x  λ3 + 4 λ 2 -3 λ + 5 adalah akar ciri dari A3 + 4 A2 -3 A + 5 I dan x vektor ciri padanannya

Beberapa Teorema Akar Ciri dan Vektor Ciri Sifat (5) dapat diperluas untuk deret tak hingga. ☺ Misal : akar ciri A adalah λ, maka (1- λ) adalah akar ciri dari (I-A). ☺ Jika (I-A) nonsingular, maka (1- λ)-1 adalah akar ciri dari (I-A)-1 . ☺ Jika -1< λ <1, maka (1- λ)-1 =1+ λ + λ2 +.... ☺ Jika akar ciri A memenuhi -1< λ <1, maka (I-A)-1 .= I +A + A 2 +A 3 +....

Beberapa Teorema Akar Ciri dan Vektor Ciri 6. Jika matriks A berukuran (n x n) dengan akar ciri λ1, ..., λn maka a. ǀAǀ=∏ λi b. tr(A)=∑ λi Bukti : a11   a12 a13 a21 a31

a22   a23  0 a32 a33  

(-λ) 3 + (-λ)2 tr1(A) +(-λ) tr2(A) + ǀAǀ=0 Dengan tri(A)= jumlah minor utama, tr1(A)= tr (A) tr2(A )=ǀa11 a22ǀ+ ǀa11 a33ǀ + ǀa22 a33 ǀ tr3(A )=ǀa11 a22 a33ǀ

Beberapa Teorema Akar Ciri dan Vektor Ciri Bukti (lanjutan): Secara umum (-λ)n +(-λ)n-1 tr1(A) +(-λ) n-2 tr2(A) + ... +(-λ)trn-1(A) +ǀAǀ=0 Jika λ1, ..., λn akar ciri dari persamaan tersebut maka (λ1-λ) (λ2-λ)... (λn-λ)=0 (-λ)n +(-λ)n-1 ∑ λi+(-λ) n-2 ∑i ≠j λi λj +...+ ∏ λi =0 Sehingga ∑ λi = tr(A) dan ∏ λi =ǀAǀ

Beberapa Teorema Akar Ciri dan Vektor Ciri ♪ Jika A dan B berukuran (n x n) atau A berukuran (n x p) dan B berukuran (p x n), maka akar ciri (tak nol) AB sama dengan akar ciri BA. Jika x vektor ciri AB, maka Bx vektor ciri BA ♪ Jika A berukuran (n x n), maka 1. Jika P (n x n) nonsingular, maka A dan P-1AP mempunyai akar ciri yang sama 2. Jika C (n x n) matriks ortogonal, A dan C’AC mempunyai akar ciri yang sama

Teorema Matriks Simetrik 1. a. Akar ciri λ1, ..., λn adalah real b.Vektor ciri x1, ..., xn bersifat ortogonal Bukti (1a): Ambil λ bilangan kompleks dengan x vektor ciri padanannya Jika λ= a+ib dan λ*= a-ib, x={xi}= a+ib dan x*={xi*}= a-ib Maka A x = λ x→ x*’ A x =x*’λx= λ x*’x dan A x* = λ*x* → x*’Ax = (Ax *)’x =(λ*x*)’ x=λ* x*’x Sehingga λ* x*’x = λ x*’x,dan x*’x≠0 adalah jumlah kuadrat → λ* = λ atau a+ib= a-ib berarti b=0

Teorema Matriks Simetrik Bukti (1b): Misalkan λ1 ≠ λ2 dengan vektor ciri x1≠x 2 dan A=A’, serta Ax k = λ x k λ1 x2’x1 = x2’ λ1 x1 = x2’ Ax 1 = x1’ A’x 2 = x1’ Ax 2 = x1’ λ2 x2 = λ2 x1’ x2 → λ1 , λ2 ≠0, maka x1’x 2 =0 (ortogonal) 2. A dapat dinyatakan sebagai A=CDC’ (dekomposisi spektral) dengan D adalah matriks diagonal dengan unsur diagonalnya λi dan C adalah matriks dengan unsur pada kolomnya x1 padanan akar ciri λi

Teorema Matriks Simetrik 3. Matriks (semi) definit positif a. Jika A definit positif, maka λi>0 untuk i=1,...,n b. Jika A semi definit positif, maka λi≥0 untuk i=1,...,n. Banyaknya akar ciri λi>0 sama dengan rank(A) Catatan : Jika A definit positif dapat ditentukan A½. Karena λi>0 maka pada dekomposisi spektral A= A½ A½ =(A½) 2 A = (A½) 2 = C’D C=(C’D ½ C) (C’D ½ C)

Teorema Matriks Simetrik 4. Jika A singular, idempoten, dan simetrik, maka A semi definit positif Bukti : A= A’ dan A =A2 maka A =A2 =A A= A’A (semi definit positif) 5. Jika A simetrik idempoten dengan rank (A)=r maka A mempunyai r akar ciri bernilai 1 dan (n-r) akar ciri bernilai 0 Bukti : Ax = λ x dan A2x = λ2 x karena A =A2 → A2x = λ2 x↔ Ax = λ2 x ↔ λx= λ2 x ↔ (λ- λ2)x=0 . Karena x≠0 maka (λ- λ2) =0 → λ bernilai 0 atau 1.Berdasarkan teorema (4), maka A semi definit positif dengan r menyatakan banyaknya λ>0

Teorema Matriks Simetrik 6. Jika A idempoten dan simetrik dengan pangkat r, maka rank(A)=tr(A)=r Bukti : Berdasarkan teorema (5), maka ∑ λi = tr(A)

Teorema Jika A (n x n) matriks idempoten, P matriks nonsingular (n xn) dan C matriks ortogonal (n xn) maka : a. I-A idempoten b. A(I-A)=0 dan (I-A) A=0 c. P-1 A P idempoten d. C’A C idempoten (jika A simetrik maka C’A C idempoten simetrik  Jika A (n x p) dengan rank(A)=r, A- adalah kebalikan umum A dan (A’ A) - adalah kebalikan umum (A’ A), maka A- A, A A- dan A(A’ A)- A idempoten 

Quadratic Forms A Quadratic From is a function

Q(x) = x’Ax in k variables x1,…,xk where  x1  x 2  x  xk   

and A is a k x k symmetric matrix.

Quadratic Forms Note that a quadratic form has only squared terms and crossproducts, and so can be written k

k

Q  x    aij x i x j

Suppose we have

then

i1 j1

x  x   1  and A  1 4  0 2 x 2 

Q(x) = x'Ax = x12 + 4x1x 2 - 2x 22

Spectral Decomposition and Quadratic Forms Any k x k symmetric matrix can be expressed in terms of its k eigenvalueeigenvector pairs (i, ei) as k

A    ie ie i1

' i

This is referred to as the spectral decomposition of A.

Spectral Decomposition and Quadratic Forms For our previous example on eigenvalues and eigenvectors we showed that A  2 4   4 4 

has eigenvalues 1 = -6 and 2 = -4, with corresponding (normalized) eigenvectors  1  2  5 , e   5 , e1    -2  2 1    5  5 

Spectral Decomposition and Quadratic Forms Can we reconstruct A? k

A    ieiei' i1

 1  2  1  2     5 5 -2  6 +4    -2   1   5 5 5   5  5   1 -2  4 2   6  5 5   4  5 5    2 4   A -2 5 4 5   2 5 1 5   4 4     

1

 5 

Spectral Decomposition and Quadratic Forms Spectral decomposition can be used to develop/illustrate many statistical results/ concepts. We start with a few basic concepts: - Nonnegative Definite Matrix – when any k x k matrix A such that 0  x’Ax  x’ =[x1, x2, …, xk] the matrix A and the quadratic form are said to be nonnegative definite.

Spectral Decomposition and Quadratic Forms - Positive Definite Matrix – when any k x k matrix A such that 0 < x’Ax  x’ =[x1, x2, …, xk] [0, 0, …, 0] the matrix A and the quadratic form are said to be positive definite.

Spectral Decomposition and Quadratic Forms Example - Show that the following quadratic form is positive definite: 6x12 + 4x 22 - 4 2x1 x 2

We first rewrite the quadratic form in matrix notation: Q(x) =  x1

 6   x1  -2 2 x 2   = x ' Ax   x 2  -2 2 4  

Spectral Decomposition and Quadratic Forms Now identify the eigenvalues of the resulting matrix A (they are 1 = 2 and 2 = 8).  6  1 0  -2 2 A  I      0 1  -2 2 4  







 6 -  -2 2   6    4     -2 2 -2 2  0 -2 2 4 -  or  2  10  16     2    8   0

Spectral Decomposition and Quadratic Forms Next, using spectral decomposition we can write: k

A    ieiei'  1e1e1'   2e2e2'  2e1e1'  8e 2e 2' i1

where again, the vectors ei are the normalized and orthogonal eigenvectors associated with the eigenvalues 1 = 2 and 2 = 8.

Spectral Decomposition and Quadratic Forms

Sidebar - Note again that we can recreate the original matrix A from the spectral decomposition: k

A    ie ie i' i1

  2     2  

  2  3  1 3 2 2  +8  -1  -1   3 3  2   3 3   3  3  1  2 2   2   6 3 2 2   A 3   8 3 3 2  2   2 2 4  2  1 3 3  3 3  

1

Spectral Decomposition and Quadratic Forms

Because 1 and 2 are scalars, premultiplication and postmultiplication by x’ and x, respectively, yield: x ' Ax  2x 'e1e1' x  8x 'e2e'2 x  2y12 + 8y 22  0

where

y1  x 'e1  e1' x1 and y 2  x 'e2  e2' x 2 At this point it is obvious that x’Ax is at least nonnegative definite!

Spectral Decomposition and Quadratic Forms We now show that x’Ax is positive definite, i.e. x ' Ax  2y12 + 8y 22  0

From our definitions of y1 and y2 we have '   y1   e1   x1  or y  Ex  y 2  e'2   x 2 

Spectral Decomposition and Quadratic Forms Since E is an orthogonal matrix, E’ exists.

Thus,

'

x Ey

But 0  x = E’y implies y  0 . At this point it is obvious that x’Ax is positive definite!

Spectral Decomposition and Quadratic Forms This suggests rules for determining if a k x k symmetric matrix A (or equivalently, its quadratic form x’Ax) is nonegative definite or positive definite: - A is a nonegative definite matrix iff i  0, i = 1,…,rank(A) - A is a positive definite matrix iff i > 0, i = 1,…,rank(A)

Square Root Matrices Because spectral decomposition allows us to express the inverse of a square matrix in terms of its eigenvalues and eigenvectors, it enables us to conveniently create a square root matrix. Let A be a p x p positive definite matrix with the spectral decomposition k

A    ieiei' i1

Square Root Matrices Also let P be a matrix whose columns are the normalized eigenvectors e1, e2, …, ep of A, i.e.,

Then

P  e2

ep 

e2

k

A    ieiei'  P ' P i1

where P’P = PP’ = I and  1 0  0 

0 2 0

0

0 0   p 

Square Root Matrices Now since (P-1P’)PP’=PP’(P-1P’)=PP’=I we have 1 A  P P   eiei' i1  i -1

Next let

 1 1  2   0   0 

1

'

0 2 0

k

0   0   p  

Square Root Matrices The matrix 1 2

k

P P '    i eiei'  A i1

is called the square root of A.

1 2

Square Root Matrices The square root of A has the following properties:   A  

1 2

1 2

'

1    A 2  1 2

 A A A -1 2

1 2

-1 2

-1 2

1 2

 A A A A  A A

A

1

-1 2

I

where A

-1 2

  A  

1 2

  

-1

Square Root Matrices Next let -1 denote the matrix matrix whose columns are the normalized eigenvectors e1, e2, …, ep of A, i.e., Then

P  e2

ep 

e2

k

A    ieiei'  P ' P i1

where P’P = PP’ = I and  1 0  0 

0 2 0

0

0 0   p 