LIBSVM学习总结

　

libsvm

Support vector Machines，SVM，支持向量机

---

各种SVM

---

C" role="presentation" style="position: relative;">C$C$-Support Vector Classication

训练向量 — xi∈Rn,i=1,…,l" role="presentation" style="position: relative;">xi∈Rn,i=1,…,l$x_i\in R^n,i=1,\dots ,l$

两个类class

指标向量 — y∈Rl" role="presentation" style="position: relative;">y∈Rl$y\in R^l$，yi∈{1,−1}" role="presentation" style="position: relative;">yi∈{1,−1}$y_i\in \left\{1,-1\right\}$

C" role="presentation" style="position: relative;">C$C$-SVC解决如下原始优化问题：

ϕ(xi)" role="presentation" style="position: relative;">ϕ(xi)$\varphi \left(x_i\right)$将xi" role="presentation" style="position: relative;">xi$x_i$映射到更高维空间，C>0" role="presentation" style="position: relative;">C>0$C>0$为正则化参数。

由于向量参数w" role="presentation" style="position: relative;">w$w$的可能的高维度，通常我们解决如下对偶问题

e=[1,…,1]T" role="presentation" style="position: relative;">e=[1,…,1]T$e={\left[1,\dots ,1\right]}^T$为全为1的向量

Q" role="presentation" style="position: relative;">Q$Q$ — 一个l×l" role="presentation" style="position: relative;">l×l$l\times l$的半正定矩阵positive semidefinite matrix

Qij≡yiyjK(xi,xj)" role="presentation" style="position: relative;">Qij≡yiyjK(xi,xj)$Q_{ij}\equiv y_i{y}_{j}K\left(x_i,x_j\right)$

K(xi,xj)≡ϕ(xi)Tϕ(xj)" role="presentation" style="position: relative;">K(xi,xj)≡ϕ(xi)Tϕ(xj)$K\left(x_i,x_j\right)\equiv \varphi {\left(x_i\right)}^T\varphi \left(x_j\right)$ — 核函数

问题（2）解决后，使用 primal-dual relationship 原始-对偶关系，最优的w" role="presentation" style="position: relative;">w$w$满足：

决策函数为

为进行预测，存储如下参数：

yiαi,∀i" role="presentation" style="position: relative;">yiαi,∀i$y_i{\alpha }_i,\mathrm{\forall }i$

b" role="presentation" style="position: relative;">b$b$

标签名称

其他参数 如 — 核参数

---

ν" role="presentation" style="position: relative;">ν$\nu$-Support Vector Classication

引入了新的参数 — ν∈(0,1]" role="presentation" style="position: relative;">ν∈(0,1]$\nu \in (0,1]$

对偶问题为

当且仅当

问题才有意义

决策函数为

可用 eTα=ν" role="presentation" style="position: relative;">eTα=ν$e^T\alpha =\nu$ 替代 eTα≥ν" role="presentation" style="position: relative;">eTα≥ν$e^T\alpha \ge \nu$

LIBSVM解决一个缩放版的问题，这是因为αi≤1/l" role="presentation" style="position: relative;">αi≤1/l${\alpha }_i\le 1/l$可能过小。

若α" role="presentation" style="position: relative;">α$\alpha$对于对偶问题（5）是最优的

ρ" role="presentation" style="position: relative;">ρ$\rho$对于原始问题（4）是最优的

则，α/ρ" role="presentation" style="position: relative;">α/ρ$\alpha /\rho$是带有C=1/(ρl)" role="presentation" style="position: relative;">C=1/(ρl)$C=1/(\rho l)$的C" role="presentation" style="position: relative;">C$C$-SVM的一个最优解，因此，在LIBSVM模型中的输出为(α/ρ,b/ρ)" role="presentation" style="position: relative;">(α/ρ,b/ρ)$\left(\alpha /\rho ,b/\rho \right)$。

---

Distribution Estimation (One-class SVM)

单类别SVM

无类别信息

对偶问题为

决策函数为

缩放版

---

ϵ" role="presentation" style="position: relative;">ϵ$ϵ$-Support Vector regression (ϵ" role="presentation" style="position: relative;">ϵ$ϵ$-SVR)

训练点集 — {(x1,z1),…,(xl,zl)}" role="presentation" style="position: relative;">{(x1,z1),…,(xl,zl)}$\left\{\left(x_1,z_1\right),\dots ,\left(x_l,z_l\right)\right\}$

xi∈Rn" role="presentation" style="position: relative;">xi∈Rn$x_i\in R^n$ — 特征向量

zi∈R1" role="presentation" style="position: relative;">zi∈R1$z_i\in R^1$ — 目标输出

给定参数 — C>0" role="presentation" style="position: relative;">C>0$C>0$ 及 ϵ>0" role="presentation" style="position: relative;">ϵ>0$ϵ>0$，支持向量回归的标准形式为：

对偶问题为

在解决问题（9）后，估计函数为

输出为 — α∗−α" role="presentation" style="position: relative;">α∗−α${\alpha }^{\ast }-\alpha$

---

ν" role="presentation" style="position: relative;">ν$\nu$-Support Vector Regression (ν" role="presentation" style="position: relative;">ν$\nu$-SVR)

对偶问题为

估计函数为

eT(α+α∗)≤Cν" role="presentation" style="position: relative;">eT(α+α∗)≤Cν$e^T\left(\alpha +{\alpha }^{\ast }\right)\le C\nu$可替换为等式

C¯=C/l" role="presentation" style="position: relative;">C¯=C/l$\overline{C}=C/l$

如下二者有相同解

1. ϵ" role="presentation" style="position: relative;">ϵ$ϵ$-SVR — 参数(C¯,ϵ)" role="presentation" style="position: relative;">(C¯,ϵ)$\left(\overline{C},ϵ\right)$

2. ν" role="presentation" style="position: relative;">ν$\nu$-SVR — 参数(lC¯,ν)" role="presentation" style="position: relative;">(lC¯,ν)$\left(l\overline{C},\nu \right)$

---

性能度量

---

分类

---

回归

---

整体组织

相关阅读