Table 1 Differences between vector neurons and scalar neurons

Input

u_i

x_i

Affine transformation

\( {\hat{\boldsymbol{u}}}_{j\mid i}={\boldsymbol{W}}_{ij}{\boldsymbol{u}}_i \)

-

Weighted sum

\( \boldsymbol{s}=j\sum \limits_i{c}_{ij}{\hat{\boldsymbol{u}}}_{j\mid i} \)

\( {a}_j=\sum \limits_i{w}_i{x}_i+b \)

Nonlinear activation

\( {\boldsymbol{v}}_j={\frac{\left\Vert {\boldsymbol{s}}_j\right\Vert }{1+{\left\Vert {\boldsymbol{s}}_j\right\Vert}^2}}^2\frac{{\boldsymbol{s}}_j}{\left\Vert {\boldsymbol{s}}_j\right\Vert } \)

h_j = g(a_j)

Output

v_j

h_j