Orthogonal Signal Space

****IMPORTANT****

• Vector Orthogonality:

  
  

  $$<\underline{x},\underline{y}>=\sum_{i=1}^{N}x_iy_i^\ast=0$$
  
  

  if $\underline{x}$ and $\underline{y}$ orthogonal, $\underline{x}\perp\underline{y}$

  Magnitude $\vert\vert\underline{x}\vert\vert^2=\sum_{i=1}^{N}\vert x_i\vert^2=<\underline{x},\underline{x}>$ as defined above

  
  

  $$\underline{a}\perp\underline{b}\Rightarrow\vert\vert a+b\vert... ...t^2+\vert\vert b\vert\vert^2 \qquad \mbox{(prove it yourself)}$$
  
  

  triangle Inequality $\vert\vert\underline{a}+\underline{b}\vert\vert\le\vert\vert\underline{a}\vert\vert+\vert\vert\underline{b}\vert\vert$

• Linearity

  
  

  $$L[a\underline{x}+b\underline{y}]=aL[\underline{x}]+bL[\underline{y}]$$
  
  

• Orthogonal expansion

  Can always write

  
  

  $$\underline{x}=\sum_{i=1}^{N}a_i\underline{e_i} \qquad \mbox{w... ...hspace{2cm}}r} 0 & i\ne j\\ \alpha_i & i=j \end{array} \right.$$
  
  

  $\{\underline{e_i}\}$ is an orthogonal basis Therefore,

  
  

  $$L[\underline{x}]=\sum_{i=1}^{N}a_iL[\underline{e_i}]$$
  
  

  Eigenvector:         $\underline{x}$ is an eigenvector of system $L[\;]$ if

  
  

  $$L[\underline{x}]=\lambda\underline{x}$$
  
  
  where $\lambda$ is a constant.

• Gram Schmidt

  Can we create personalized orthogonal sets? GET YOURS NOW! CALL 1-800-555,... :)

  Aside on independence: $\{\underline{x_i}\}$ $\mbox{Card}\{\underline{x_i}\} =J\le N$ are mutually independent if

  
  

  $$\underline{x_i}\ne \sum^J_{j=1, j\ne i} a_j \underline{x_j}$$
  
  

  for all choices $\{a_j\}$

  Given J mutually independent vectors $\underline{x_1},\cdots,\underline{x_j}$we can form mutually orthogonal set $\{\underline{\phi_i}\}$using GRAM-SCHMIDT ORTHOGONALIZATION

• Procedure

  Start with $\underline{x}_1$
  

  $$\underline{\phi_1}=\frac{\underline{x_1}}{\vert\vert\underline{x_1}\vert\vert}$$
  
  

  Then use $\underline{x}_2$ and subtract off projection onto $\underline{\phi}_1$
  

  $$\underline{\phi_2} =\underline{x_2}- \frac{<\underline{\phi_1... ...c{\underline{\phi_1}}{\vert\vert\underline{\phi_1}\vert\vert}$$
  
  
  etcetera and so on...
  
  $$\underline{\phi_k} =\underline{x_k}- \sum_{j=1}^{k-1} \frac{<... ...c{\underline{\phi_j}}{\vert\vert\underline{\phi_j}\vert\vert}$$
  
  

• Orthogonality Check

  
  

  $$<\underline{\phi_1},\underline{\phi_2}>= <\underline{x_1},\un... ...ne{x_1},\underline{x_1}>}{<\underline{x_1},\underline{x_1}>}=0$$
  
  

  can show recursively that $\underline{\phi_i}\perp \phi_j$         $i\ne j$

  Then we make orthonormal:
  

  $$\underline{\psi_i}=\frac{\underline{\phi_i}}{\vert\underline{\phi_i}\vert}$$
  
  

  
  

  $$<\underline{\psi_i},\underline{\psi_j}>= \left\{\begin{array}... ... & {i\ne j} \\ \displaystyle {1} & {i=j} \end{array}\right.$$
  
  

• What about Signals?

  SIGNAL PICTURE

  
  

  $$\underline{f}=(f_1,f_2,\cdots f_N)$$
  
  

  
  

  $$\Delta<\underline{g},\underline{f}>=(\sum_{i=1}^N g_i f_i^*)\Delta$$
  
  

  Shrink $\Delta\rightarrow 0$
  

  $$\frac{1}{T}\int_0^T g(t)f^*(t)dt\equiv <g(t),f(t)>$$
  
  
  if $g(t)\perp f(t)$ on (a,b) then
  
  $$\int_a^b g(t)f^*(t)dt=0$$
  
  

• Example

  
  

  $$\omega_0=\frac{2\pi}{T}$$
  
  

  
  

  $$\underbrace{<e^{j\omega_0 kt}, e^{j\omega_0 mt}>}_{\mbox{orth... ...{array}{r @{\qquad}r} 0 & m\ne k\\ 1 & m=k \end{array} \right.$$
  
  
  GRAM SCHMIDT WORKS TOO!

• Completeness

  Complete set means any signal in the signal space could be expanded as a series, i.e.

  
  

  $$f(t)=\sum_{i=1}^{\infty}a_i \phi_i(t)$$
  
  
  where $<\phi_i(t),\phi_j(t)>=\delta_{i,j}$

  $\{\phi_i(t)\}$ forms a basis for the signal set if any f(t) can be represented this way

  Complex exponentials are a complete orthonormal which is why Fourier series works

  DECOMPOSITION
  

  $${f(t)=\sum_{k=-\infty}^{\infty} a_k e^{j\omega_0 kt}}$$
  
  

  PROJECTION
  

  $$a_k=\frac{1}{T}\int_0^T f(t) e^{-j\omega_0 kt}dt$$
  
  
  which form the Fourier series pair when f(t)=f(t+T), $\omega_0 = 2\pi/T$ and if f(t) satisfies Dirichlet (see book)