1. Introduction

2. Related Work

3. Deep Hypersphere Embedding

3.1. Revisiting the Softmax Loss

• softmax loss (binary-class case)

  $$ p_1=\frac{\exp(W^T_1 x + b_1)}{\exp(W^T_1 x + b_1) + \exp(W^T_2 x + b_2)} \tag{1} $$

  $$ p_2=\frac{\exp(W^T_2 x + b_2)}{\exp(W^T_1 x + b_1) + \exp(W^T_2 x + b_2)} \tag{2} $$

  • $W^T_1 x + b_1$과 $W^T_2 x + b_2$이 classification result
    • the decision boundary : $(W_1 + W_2) x + b_1 - b_2 = 0$
    • $W^T_i x + b_i$ → $||W^t_i||\ ||x|| \cos(\theta_i)$
      • $\theta_i$ : $W_i$와 $x$ 사이의 각도
    • IF $||W_i||=1, b_i=0$
      • $p_i = ||x|| \cos(\theta_i)$ → $x$는 공유하고 $\theta_i$에 의해 결정됨
      • the decision boundary : $\cos(\theta_1) - \cos(\theta_2) = 0$

• original softmax

  $$ L=\frac{1}{N} \sum_i L_i = \frac{1}{N} \sum_i -\log(\frac{e^{f_{y_i}}}{\sum_j e^{f_j}}) \tag{3} $$

  $$ \begin{aligned} L_i &= -\log(\frac{e^{W^T_{y_i} x_i + b_{y_i}}}{\sum_j e^{W^T_j x_i + b_j}}) \\ &=-\log(\frac{e^{||W_{y_i}|| \ ||x_i|| \cos(\theta_{y_i,i}) + b_{y_i}}}{\sum_j e^{||W_j||\ ||x_i|| \cos(\theta_{j,i}) + b_j}}) \tag{4} \end{aligned} $$

  • $||W_j||=1$, zero the biases

  $$ L_\text{modified} = \frac{1}{N} \sum_i -\log(\frac{e^{||x_i|| \cos(\theta_{y_i,i})}}{\sum_j e^{||x_i|| \cos(\theta_{j,i})}}) \tag{5} $$

3.2. Introducing Anbular Margin to Softmax Loss

4. Experiments (more in Appendix)

5. Concluding Remarks