To evaluate the performance of the
various approaches to cross-lingual information retrieval
between German and English in the medical domain,
the MuchMore project developed an evaluation set of
25 queries and corresponding relevance assessments
on the MuchMore Springer corpus of parallel medical
scientific abstracts. There are two sets of relevance
assessments, Set1 has been developed by a German team
of medical experts from ZInfo, Set2 by a US team at
CMU.
Please note - To use this evaluation set you should
download also the complete MuchMore Springer bilingual
corpus: English
and/or German
-- this version of the corpus is somewhat larger than
the one available from the bilingual
corpus page, which was used in development of
various MuchMore components.
Queries and relevance assessments are available for
downloading from this page:
Queries (German/English)
Please note - Queries are available in simple text
format and in the MuchMore annotation format
Relevance Assessments: Set1,
Set2
Please note - Relevance Assessments have the following
structure:
QueryNumber / 0 (not relevant) / Journal-Name followed
by Journal-Id / 1 (not relevant)
|