C3 Linear Model

[latexpage]

 

3.1 The Basic Form

  • Sample $\boldsymbol{x}=(x_1,x_2,;\dots;x_d)$
  • \begin{align} f(\boldsymbol{x})&=w_1x_1+w_2x_2+\dots+w_dx_d+b\\&=\boldsymbol{w}^T\boldsymbol{x}+b\end{align}
  • Object: Learn$\boldsymbol{w}$and$b$
  • Great comprehensibility

3.2 Linear Regression

  • Serialization of discrete properties
  • Least square parameter estimation

\begin{align}
(w^*,b^*)&=\mathop{\arg\min}_{(w,b)}\sum^{m}_{i=1}(f(x_i)-y_i)^2\\
&=\mathop{\arg\min}_{(w,b)}\sum^{m}_{i=1}(y_i-wx_i-b)^2
\end{align}

  • Closed-form solution for 1-dimension attribute space
    • \begin{align}
      w&=\frac{\sum^m_{i=1}y_i(x_i-\overline{x})}{\sum^m_{i=1}x^2_i-\frac{1}{m}(\sum^m_{i=1}x_i)^2}\\
      b&=\frac{1}{m}\sum^m_{i=1}(y_i-wx_i)
      \end{align}
  • Closed-form solution for $d$-dimension attribute space
    • \begin{align}
      \boldsymbol{\^w}&=(\boldsymbol{w};b)\\
      \boldsymbol{\^w^*}&=(\boldsymbol{X}^T\boldsymbol{X})^{-1}\boldsymbol{X}^T\boldsymbol{y}\\
      f(\boldsymbol{\^x}_i)&=\boldsymbol{\^x}^T_i(\boldsymbol{X}^T\boldsymbol{X})^{-1}\boldsymbol{X}^T\boldsymbol{y}
      \end{align}
  • Log-linear regression

\begin{align}
\ln y&=\boldsymbol{w}^T\boldsymbol{x}+b\\
y&=e^{\boldsymbol{w}^T\boldsymbol{x}+b}
\end{align}

  • Generalized linear model
    • Link function $g(\cdot)$should be continuous and smooth

\begin{align}
y=g^{-1}(\boldsymbol{w}^T\boldsymbol{x}+b)
\end{align}

3.3 Logistic Regression

 

TBC…

Leave a Reply

Your email address will not be published. Required fields are marked *