Skip to main content

Table 2 Comparisons of perplexities on test set of Penn Treebank Corpus with different sizes of class layer

From: RNN language model with word clustering and class-based output layer

Class RNNLM-Freq / +KN5 RNNLM-Brown / +KN5
  (words per second) (words per second)
30 135.57 / 113.13 (744) 131.46 / 110.83 (567)
50 136.39 / 113.53 (938) 129.79 / 109.96 (862)
100 135.49 / 113.07 (1,047) 128.36 / 109.33 (970)
200 136.03 / 112.89 (1,013) 128.52 / 109.13 (1,000)
400 135.75 / 113.04 (847) 128.03 / 109.09 (906)
800 134.98 / 112.51 (645) 128.09 / 109.23 (710)
1,600 133.44 / 111.93 (367) 128.67 / 109.47 (480)
10,000 (full) 123.00 / 106.00 (65) 123.00 / 106.00 (65)
  1. The full model use the whole 10K vocabulary as the class layer, which is the same for both models. Perplexity of LM-KN5 on test set is 141.46.