Table 5 Average SLU F1 scores per severity group. Best accuracy per severity group shown by \(\ddag\) and best SLU decoder per pre-trained encoder shown by \(\dag\)
SLU system | Severe | Moderate | Mild | Mean | STD |
|---|---|---|---|---|---|
TDNN-Capsule | 89.25 | 94.74 | \(98.29^\ddag\) | 94.09 | 4.55 |
TDNN-LSTM | \(91.48^\dag\) | \(96.78^\dag\) | 94.75 | \(94.34^\dag\) | 2.67 |
TDNN-NMF | 87.48 | 93.98 | 91.36 | 90.94 | 3.27 |
Transformer-Capsule | 75.69 | 89.51 | 86.30 | 83.83 | 7.23 |
Transformer-LSTM | \(93.88^\ddag\) | \(97.81^\ddag\) | \(94.56^\dag\) | \(95.42^\dag\) | 2.10 |
Transformer-NMF | 84.99 | 91.90 | 88.80 | 88.56 | 3.46 |
Whisper-Capsule | 88.33 | 96.65 | 96.38 | 93.79 | 7.23 |
Whisper-LSTM | \(88.51^\dag\) | \(97.66^\dag\) | \(96.95^\dag\) | \(94.37^\dag\) | 2.10 |
Whisper-NMF | 64.33 | 62.66 | 62.86 | 63.28 | 3.46 |
XLSR-53-Capsule | 61.39 | \(82.82^\dag\) | 84.53 | 76.25 | 12.89 |
XLSR-53-LSTM | \(65.16^\dag\) | 81.31 | \(88.81^\dag\) | \(78.43^\dag\) | 12.09 |
XLSR-53-NMF | 41.95 | 72.61 | 68.60 | 61.05 | 16.67 |