Table 4 Comparison of the different QbE STD evaluations: Albanian (ALB), Basque (BAS), Czech (CZE), non-native English (NN-ENG), Isixhosa (ISIX), Isizulu (ISIZ), Romanian (ROM), Sepedi (SEP), Setswana (SET), and Slovak (SLO)
From: ALBAYZIN Query-by-example Spoken Term Detection 2016 evaluation
Evaluation | Language/s | Type of speech | No. of queries dev./test | Primary metrics |
|---|---|---|---|---|
MediaEval 2011 | English, Hindi, Gujarati, and Telugu | Tel. | 64/36 | ATWV |
MediaEval 2012 | 2011 + isiNdebele, Siswati, Tshivenda, and Xitsonga | Tel. | 164/136 | ATWV |
MediaEval 2013 | ALB, BAS, CZE, NN-ENG, ISIX, ISIZ, ROM, SEP, and SET | Tel. and mic. | > 600/ > 600 | ATWV |
MediaEval 2014 | ALB, BAS, CZE,NN-ENG, ROM, and SLO | Tel. and mic. | 560/555 | C nxe |
NTCIR-11 2014 | Japanese | Mic. workshop | 63/203 | F-measure |
NTCIR-12 2016 | Japanese | Mic. workshop | 120/1620 | F-measure, ATWV, MAP |
ALBAYZIN 2012 | Spanish | Mic. workshop | 60/60 | ATWV |
ALBAYZIN 2014 | Spanish | Mic. workshop | 94/99 | ATWV |
ALBAYZIN 2016 | Spanish | Mic. workshop + parliament | 102/106+95 | ATWV |