To evaluate the performance of the various approaches to cross-lingual information retrieval between German and English in the medical domain, the MuchMore project developed an evaluation set of 25 queries and corresponding relevance assessments on the MuchMore Springer corpus of parallel medical scientific abstracts. There are two sets of relevance assessments, Set1 has been developed by a German team of medical experts from ZInfo, Set2 by a US team at CMU.

Please note - To use this evaluation set you should download also the complete MuchMore Springer bilingual corpus: English and/or German -- this version of the corpus is somewhat larger than the one available from the bilingual corpus page, which was used in development of various MuchMore components.

Queries and relevance assessments are available for downloading from this page:

Queries (German/English)

Please note - Queries are available in simple text format and in the MuchMore annotation format

Relevance Assessments: Set1, Set2

Please note - Relevance Assessments have the following structure:

QueryNumber / 0 (not relevant) / Journal-Name followed by Journal-Id / 1 (not relevant)