Types of Neural Network

Why sequency?

Speech Recognition		\(\longrightarrow\)	Get your facts first, then you can distort them as you please.
Music generation	\(\emptyset\)	\(\longrightarrow\)
Sentiment classification	Great movie ? Are you kidding me ! Not worth the money.	\(\longrightarrow\)
DNA sequence analysis	ACGGGGCCTACTGTCAACTG	\(\longrightarrow\)	AC GGGGCCTACTG TCAACTG
Machine translation	网红脸	\(\longrightarrow\)	Internet celebrity face
Video activity recognition		\(\longrightarrow\)	Running
Name entity recognition	Use Netlify and Hugo.	\(\longrightarrow\)	Use Netlify and Hugo.

RNN types

• rectangle: a vector
  • green: input vector
  • blue: output vector
  • red: intermediate state vector
• arrow: matrix multiplications

Notation

• x: Use(\(x^{<1>}\)) Netlify(\(x^{<2>}\)) and(\(x^{<3>}\)) Hugo(\(x^{<4>}\)) .(\(x^{<5>}\))

• y: 0 (\(y^{<1>}\)) 1(\(y^{<2>}\)) 0(\(y^{<3>}\)) 1(\(y^{<4>}\)) 0(\(y^{<5>}\))

• \(x^{(i)<t>}\), \(T_x^{(i)}\) (\(i^{th}\) sample)

• \(y^{(i)<t>}\), \(T_y^{(i)}\) (\(i^{th}\) sample)

Representing words

• One Hot Encoding (OHE)

\(\left[\begin{array}{c} a[1]\\ aaron[2]\\ \vdots\\ and[360]\\ \vdots\\ Hugo[4075]\\ \vdots\\ Netlify[5210]\\ \vdots\\ use[8320]\\ \vdots\\ Zulu[10000] \end{array}\right]\Longrightarrow use=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 1\\ \vdots\\ 0 \end{array}\right], Netlify=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 1\\ \vdots\\ 0\\ \vdots\\ 0 \end{array}\right], and=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 1\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0 \end{array}\right], Hugo=\left[\begin{array}{c} 0\\ 0\\ \vdots\\ 0\\ \vdots\\ 1\\ \vdots\\ 0\\ \vdots\\ 0\\ \vdots\\ 0 \end{array}\right]\)

What is RNN?