Skip to main content

Advertisement

Table 2 Complexity and speed comparisons of UP-NNLMs and NNLMs for word predictions

From: Empirically combining unnormalized NNLM and back-off N-gram for fast N-best rescoring in speech recognition

Model Complexity Speed × 103
   (words/second)
RNNLMa O(H2 + |V|H) 0.041
+Class layer O(H2 + (|V|/C + C)H) 4.21
UP-RNNLM O(H2 + H) 11.95
FNNLMa O((N - 1)D H + |V|H) 0.214
+Class layer O((N - 1)D H + (|V|/C + C)H) 9.55
UP-FNNLM O((N - 1)D H + H) 17.77
fast-UP-FNNLM O(H) 240.38
  1. aThe implementations of RNNLM and FNNLM are based on the open source toolkits, CSLM and RNNLM. The matrix and vector operations in CSLM toolkit are optimized via MKL Library, so that the evaluation of FNNLM is faster than RNNLM with the same size of hidden layer.