Machine Learning (1) - Linear & Logistic Regression
 - Linear & Logistic Regression/1491028424509.png)
Linear Regression
Linear Regression = 线性回归。
Single Feature Hypothesis:
 - Linear & Logistic Regression/SVG7d23ac9b0599a1fe6ce93afb5acdb6c0)
 - Linear & Logistic Regression/SVG836a441dc6ea7c28a4c84e4ca664ea2e){kind=small}
。 列出单feature的cost function，不做多介绍。重点放在Multiple Feature Linear Regression。
Cost Function:
 - Linear & Logistic Regression/SVGe9c66918863dbfb4732bb0d990d7e512)
Cost Function & Gradient Descent
Multiple Feature Hypothesis:
 - Linear & Logistic Regression/SVGadfa7edb34cfa9178eb8de77c5c3da6b)
Multiple Feature Cost Function:
 - Linear & Logistic Regression/SVGa2e9495bfc83fe763885a95044e1d45f)
Gradient Descent:
 - Linear & Logistic Regression/SVGcebf52d0718dce59146d554a10fc1033)
- j := 0…m
- Learning rate.
How to choose learning rate - ?
- If
- If
) may not decrease on every iteration; may not converge.
To choose
…, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, …
每次取3倍。
Feature Scaling
We can speed up gradient descent by having each of our input values in roughly the same range. This is because θ will descend quickly on small ranges and slowly on large ranges, and so will oscillate inefficiently down to the optimum when the variables are very uneven.
用下面的公式预处理我们的training set，就是Feature Scaling
 - Linear & Logistic Regression/SVG28c30732fe3fef9c54f59e001a9b54ec)
is the average of all the values for feature (i)
is the range of values (max - min)
- or
* standard deviation (标准差)=
 - Linear & Logistic Regression/SVGb100946ff715835a9459d3585b8c965d)
Normal Equation正规解
 - Linear & Logistic Regression/SVGb27f34569e87884e097e71391e6ca12f)
- 注1：training set的数量要大于feature数量，否则
会不可逆，导致没有解
- 注2：正规解不需要feature scaling
Logistic Regression
Hypothesis:
 - Linear & Logistic Regression/SVGd1a694857d9539d6607e89671d43b048)
 - Linear & Logistic Regression/SVGe33136bdc60e1298d6e84de170630909)
 - Linear & Logistic Regression/SVG9b460439e20c54f261504f5da59a746a)
The following image shows us what the sigmoid function looks like:
 - Linear & Logistic Regression/1491033397652.png)
这里的Hypothesis方程是不平滑的，会导致很多的local optimization，导致Gradient Descent不能成功收敛。
用条件概率表示Hypothesis:
y=1的条件下，x,
 - Linear & Logistic Regression/SVG88b1d566aa187c2b8edaaa733182c33d)
 - Linear & Logistic Regression/SVG93ff7b61ca29f13fdf4c92ccc47ee6da)
Cost Function
 - Linear & Logistic Regression/SVG75fca90d33cb4a375540207f6f2a7be0)
 - Linear & Logistic Regression/SVG73eab47a2d92d756f0c7068704f8757e)
 - Linear & Logistic Regression/SVGd52110682affea90d031dfc31bc39fbf)
 - Linear & Logistic Regression/SVGfaf5dc80308d8f1dd0c9b9ff36e123d9)
 - Linear & Logistic Regression/SVG6af02d365362292dcf60cbad81a56b69)
 - Linear & Logistic Regression/SVG2585329387d19621abb8e899dc65d45a)
这其中用Cost函数取代了线性回归中的
 - Linear & Logistic Regression/SVG01f2b314c5db3826fb1eb2c0695aec72)
将Cost函数代入整个Cost Function中可得：
 - Linear & Logistic Regression/SVG5cf1646f441aa186c9f8eb1eb64b13f8)
Vectorized implementation:  - Linear & Logistic Regression/SVG85456d63b56d16b7345ac846cc85ecf2)
 - Linear & Logistic Regression/SVGdcd056880207c58c97844ff015771e2d)
进一步计算Gradient Descent的迭代算法为：
 - Linear & Logistic Regression/SVGf6970f60bdac10b142d1d13d4b2c346b)
矢量化写法：
 - Linear & Logistic Regression/SVGc4b17cdb4d55d3e56b08936c7ec923af)
Overfitting过拟合
Regularized Linear Regression
 - Linear & Logistic Regression/SVG33a12c1d855f9f8f289dcef9d076110f)
The λ, or lambda, is the regularization parameter. It determines how much the costs of our theta parameters are inflated.
Regularized Logistic Regression
 - Linear & Logistic Regression/SVGc49237343ca20e996bdd8b1b0596221f)
 - Linear & Logistic Regression/SVG9411849528295073b7b752b252831838)
means to explicitly exclude the bias term  - Linear & Logistic Regression/SVG1a3151e36f9f52b61f5bf76c08bdae2b)
!!注意：这里的regularized项，不包含 - Linear & Logistic Regression/SVGedcbf8dd6dd9743cceeee21183bbc3b6)