www-ai.cs.tu-dortmund.de/de/LEHRE/FACHPROJEKT/SS12/paper/classification/bottou2005.pdf
bordes05a.dvi
(xt−1 +λut)− f (xt−1)−λu′t∇ f (xt−1) ∣
∣≤ 1 2
λ2‖ut‖ 2H
or, more specifically,
f (xt−1 +λut)− f (xt−1)−λu′t∇ f (xt−1) ≥ − 1 2
λ2‖ut‖ 2H.
Combining these inequalities yields
f (xt)− f (xt−1)≥ f (xt−1 +λut)− [...] ∀ t2 > t1 > t0, f (xt2)− f (xt1) = ∑
t1<t≤t2
f (xt)− f (xt−1) < ε.
Using proposition 15, we can write
∀ ε > 0, ∃ t0, ∀ t2 > t1 > t0,
‖xt2−xt1‖ ≤ ∑ t1<t≤t2
‖xt −xt−1‖ ≤ ∑ t1<t≤t2
f (xt)− f (xt−1)
K <
ε K [...] attained inxt = xt−1 +λ∗ut . Let λ = νλ∗ with 0 < ν≤ 1. Sincext is a maximum,
f (xt)− f (xt−1) = f (xt−1 +λ∗ut)− f (xt−1)≥ f (xt−1 +λut)− f (xt−1).
Let H be the maximum overF of the norm of the Hessian …
