MUCHMORE
Multilingual Concept Hierarchies for Medical Information Organization and Retrieval

D 0.7
Technology Implementation Plan

Project ref. no. 	IST-1999-11438	
Project title	MuchMore - Multilingual Concept Hierarchies for Medical Information Organization and Retrieval	

Deliverable status	Public	
Contractual date of delivery 	February 2003	
Actual date of delivery	August 2003	
Deliverable number	D-0.7	
Deliverable title	Technology Implementation Plan	
Type	PP 	
Status & version	Final	
Number of pages	31	
WP contributing to the deliverable	WP0 	
WP / Task responsible	DFKI 	
Author(s)	Bogdan Sacaleanu, Paul Buitelaar, Špela Vintar, Diana Raileanu (DFKI); Bryan Kisiel (CMU); Dominic Widdows (CSLI); Martin Volk, Martin Braschler (EIT); Eric Gaussier, Hervé Dejean (XRCE)	
EC Project Officer	Yves Paternoster	
Keywords	Cross-Language Information Retrieval, Semantic Annotation, Summarisation, Natural Language Processing, Multilingual Resources	
Abstract (for dissemination)	This report describes the technologies and approaches in the MUCHMORE project in detail. It explains how each component works and how it fits into the cross-language information retrieval prototype. The report focuses on the techniques and algorithms used in the project, and aims to provide an in-depth understanding of the technological aspects of the MUCHMORE project. 	

Contents
Introduction
1	Overview
1.1	Project Overview
1.2	Components Overview
2	Client Tier
2.1	Overview
2.2	Task Manager
2.3	Language Recognition
2.4	MeSH Query Expansion
2.5	Document Manager
2.6	Summariser
2.7	Presentation Manager
3	Middle Tier
3.1	Overview
3.2	Query & Document Analysis Controller
3.3	Shallow Processing
3.4	Term Tagging
3.5	Term Extraction
3.6	Semantic Relations Tagging
3.7	Semantic Relations Filtering
3.8	Semantic Relations Extraction
3.9	Word Sense Disambiguation
4	Back-End Tier
4.1	Overview
4.2	Engine Controller
4.3	Hit-List Merger
4.4	CSLI Search Engine
4.5	CMU Search Engine
4.6	EIT Search Engine
References
Appendix A: Quantified Data
Appendix B: Expected Project Impact
Appendix C: European Interest

Introduction

This report describes the technologies and approaches in the MUCHMORE project in detail. It explains how each component works and how it fits into the cross-language information retrieval prototype. The report focuses on the techniques and algorithms used in the project, and aims to provide an in-depth understanding of the technological aspects of the MUCHMORE project. 
The report starts out with an overview of the project and the components developed within the project, and then gives a detailed description of each component.
The intended audience of the report are
•	developers who want to re-use components in other projects
•	developers who want to build a new system using the MUCHMORE prototype architecture
•	users and technologists who want to get an in-depth understanding of the project

1	Overview
1.1	Project Overview
MUCHMORE provides a framework for integrating and refining existing technologies and developing new approaches to cross-language information retrieval for the medical domain. Existence of very large ontologies of domain concepts and extensive corpora for the medical domain has grounded the work toward refinement, integration and comparison of concept-based retrieval methods and corpora-based approaches. Within the scope of establishing a cross-language information retrieval framework several tasks have been carried out:

1.	Research regarding the effective combination of different approaches and heterogeneous resources and their integrated use for multilingual information access and management, including performance evaluation for realistic information access tasks.
2.	Research and technology development concerning the automated acquisition of domain-specific linguistic resources and effective use of multilingual concept hierarchies.
3.	Demonstration of a cross-lingual information access prototype system for the medical domain, that provides access to multilingual information on the basis of a domain ontology and classification.

Along these lines innovative developments have been done in the following areas:

•	Combination of Heterogeneous Resources and Methods
Methods and resources already available, as well as within the project new developed, have been integrated for cross-language information management and access. Performance testing provided quantitative feedback on the performance of different methods and resources. A systematic combination of these methods and resources allowed precisely determining the contribution each method or resource can make to the overall performance.
•	Refinement of Multilingual Term Extraction
Building on consortium-internal experience with multilingual term extraction from parallel corpora and using resources in the medical domain, methods for multilingual terminology extraction from parallel corpora were refined and methods for multilingual term extraction from comparable corpora were new developed.
•	Relation Extraction from Semantically Annotated Corpora
Using existing resources in the medical domain, as well as resources developed within the project, new techniques for extracting relations between relevant terms have been developed.
•	Use of Concept Hierarchies in Cross-Language Information Access
Use of medical conceptual structures for mapping both user queries and documents into an intermediate representation, bridging the cross-language barrier, has been investigated and new techniques for automatic semantic annotation has been developed. Concept hierarchies have been integrated in the query processing functionality, which allows for interactive refinement of user queries.
•	Multi-Document Summarization
Multi-document summarization has been developed and integrated into the cross-lingual retrieval engine. The approach is based on the Maximal Marginal Relevance method of extracting the most relevant and diverse passages from a text.
1.2	Components Overview
Given the fact that the aim of MUCHMORE is to provide a framework for developing and integrating cross-language information retrieval technologies, the components developed within this scope will be described in the context of a cross-language medical information access prototype. The prototype integrates three different retrieval approaches, allowing a systematic evaluation of the contribution that each approach can have to the overall performance. 
Within the MUCHMORE project, the cross-language information retrieval task has been approached from two different views: concept-based methods vs. corpora-based techniques. Along these lines there have been developed three demo systems:

•	A cross-language document retrieval system based on semantic annotation, which is using concept-based methods for generating an intermediary representation of both queries and documents.
•	A cross-language thesaurus generation and document retrieval system, which uses a corpora-based statistical analysis of the distribution of terms in translated documents to generate a representation of terms and documents from both languages in the same mathematical space.
•	A cross-language document retrieval system leveraging parallel training corpora in the relevant languages, employing two methods: one a pseudo-relevance feedback (PRF) approach, and the other an example-based thesaurus (EBT) approach, including a thesaurus generator.

and a prototype to integrate them and to create a common user interface for the retrieval task. At its core, the prototype is a multitier application configured as a client tier to provide a user interface, a middle tier annotation module that generates the intermediary data representation, and a back-end tier consisting of the search engine systems to provide the retrieval technology (see Figure 1.).

Middle Tier

1. PoS Tagger
2. Morphology
3. Chunking

1. Linguistic Annotation
2. Semantic Annotation

Client Tier

Query Processing
&
Results Displaying

Back-End Tier

Search Engine(s)

Figure 1.  Prototype Architecture

The client tier of the prototype system deploys both a query processing and a document viewer functionality, which provide a user interface for completing and refining the query construction, as well as a hit-list rendering interface for displaying the results of the retrieval task. The components of the client tier are as follows:

•	Task Manager: is the main control and coordination unit of the client tier. It receives a user request, performs request dispatching and constructs the top-level response, which is returned to the user.
•	Language Recognition: provides functionality for identifying the language of a user-defined query.
•	MeSH query expansion: provides functionality for expanding the user query with controlled medical terminology.
•	Document Manager: provides functionality for interacting with the document repository and delivering the results of the retrieval task.
•	Summarizer: provides functionality for summarizing collections of retrieved documents.
•	Presentation Manager: provides functionality for generating the user interfaces for query processing and results displaying.

The middle tier of the prototype system consists of natural language processing components, representing an advanced annotation system of lexical and semantic features. The role of this middle level is twofold: first, it reduces both the user queries and document collection to a common semantic representation by automatically annotating them, and second, it assists the user in refining the query by providing him with information regarding medical terminology and semantic relations holding among terms. The common semantic representation of a query and a document is being used in the context of the concept-based approach to cross-language information retrieval task, while the query refinement functionality is being employed without regard of the specific retrieval approach. The components of this intermediary tier are as follows:

•	Query & Document Analysis Controller: is the main coordination unit of the middle tier. It implements the business logic of the linguistic and semantic analysis process and controls the functionalities provided by the other components.
•	Shallow Processing: is the basic annotation unit of the middle tier. It produces lexical information at the following levels: part-of-speech, morphology and phrase tagging (chunking).
•	Term Tagging: provides functionality for spotting and marking up terminology according to existing medical controlled vocabularies.
•	Term Extraction: provides functionality for spotting and marking up new terminology according to patterns of use.
•	Semantic Relations Tagging: provides functionality for looking up and marking up semantic relations holding between medical terms according to existing medical semantic networks.
•	Semantic Relations Filtering: provides functionality for filtering marked up relations according to semantic context.
•	Semantic Relations Extraction: provides functionality for extracting new semantic relations according to contextual patterns of use.
•	Word Sense Disambiguation: provides functionality for choosing the intended reading of a possible ambiguous medical expression according to contextual patterns of use.

The back-end tier of the prototype system is a meta-search retrieval framework, which provides functionalities both for integrating search engines corresponding to different cross-language information retrieval approaches and for merging of their retrieval results. The components of this back-end tier are as follows:

•	Engine Controller: is the main coordination unit of the back-end tier. It receives a retrieval request in form of a query string, which is further dispatched to remote search engines, and constructs a response containing lists of retrieved results.
•	Hit-List Merger: provides functionality for normalizing and merging different scoring schemes in one unified result list with most relevant documents ranked topmost.
•	CMU Search Engine: cross-language document retrieval system leveraging parallel training corpora in the relevant languages, employing two methods: one a pseudo-relevance feedback (PRF) approach, and the other an example-based thesaurus (EBT) approach, including a thesaurus generator.
•	CSLI Search Engine: cross-language thesaurus generation and document retrieval system, which uses a corpora-based statistical analysis of the distribution of terms in translated documents to generate a representation of terms and documents from both languages in the same mathematical space.
•	EIT Search Engine: cross-language document retrieval system based on semantic annotation, which is using concept-based methods for generating an intermediary representation of both queries and documents.

2	Client Tier 
2.1	Overview
The client tier of the system produces an interactive user interface for query processing and rendering of the results that match the user’s information need. Figure 2 describes the scenario of a user-system interaction and the components of the client tier.

Q&DA Controller

Response

Request

2

1

Language Recognition

Task Manager

Presentation Manager

User

Engine Controller

Figure 2. Client Tier Architecture

3

4

Middle Tier

6

5

Summarizer

Document Manager

Back-end  Tier

MeSH Query Expansion

The user interacts with the system through a collection of dynamic HTML pages generated by the Presentation Manager component. This assists the user in uploading a query document (a patient record in this case), handling the query generation by relying on fallback components for language recognition, semantic analysis and expansion, selecting target search engines (retrieval approaches) and displaying query relevant documents.
The main unit of the system, the Task Manager, controls the workflow of the user interaction process and delegates the business logic to external components. These components responsible for the logic of the system can exist both within the client tier as well as on the middle and back-end tier, as Query Analysis and Engine controllers do. 
Once the query document has been uploaded, the Language Recognition component automatically identifies the language of the text and passes the control further to the middle tier’s Query Analysis Controller. The analysis component translates the query into an intermediate XML representation, consisting of both lexical and semantic information at several levels of annotation, based on which the Presentation Manager offers the user a domain-specific semantic interpretation of the query. A list of concepts and relations holding among concepts conveys the system’s interpretation of the supplied query. This automatically generated semantic interpretation can be either refined, by selecting appropriate concepts and relations, or expanded with new domain-specific controlled vocabulary, through the MeSH Query Expansion component.
The Document Manager mediates the interaction of the Task Manager with the document collection. The list of query-relevant document IDs, passed back by the back-end tier through the Engine Controller, is being processed and the content of the matched documents is forwarded to the Presentation Manager for visual rendering. Based on the list of displayed documents, the user can select a collection of document IDs to be sent to the Summarizer component.
2.2	Task Manager
2.2.1	Description
The Task Manager controls the flow of the user interaction process and delegates the business logic to external components. It receives a user request, processes the request invoking the appropriate component and returns a response by invoking a Presentation Manager-generated view. 
2.2.2	Input/Output Behaviour
Input: A request from the current presentation (view) along with input parameters.
Output: A view generated by the Presentation Manager according to the request and the managed task.
2.2.3	Programming Issues
Implemented as a Java Servlet and designed as a Controller, following the Model-View-Controller-based J2EE Model 2 design paradigm.
2.3	Language Recognition 
2.3.1	Description
Language identification is performed by making use of an algorithm, which compares the relative frequencies of the most frequent n-grams (from 1 to 5 characters) in a document to 40 stored language profiles (Cavnar and Trenkle, 1994). For each language, the language identifier uses a language model - the sequence of the 300 most frequent n-grams, ordered by their frequency. For each document whose language we want to identify, we compute the sequence of its most frequent n-grams. For each n-gram of the document, we compare its rank to the rank of the same n-gram in the language model, and sum up the differences. The language whose language model has the smallest difference to the current document will be assigned. The first 200 characters of a document are sufficient for reliable document identification.
2.3.2	Input/Output Behaviour
Input: Document content as a stream of characters.
Output: A language code for the recognised language, or 'none' if no language can be identified.
2.3.3	Evaluation 
The language identifier was evaluated by selecting random documents from the WWW and performing language identification on them. From a sample of 502 documents, the language was correctly identified for 352 documents (71.3 per cent). However, many of these pages did not contain any real text, but just images, numbers or addresses. 
2.3.4	Programming Issues
Implemented as a Session Enterprise Java Bean (EJB) component and designed as a Model, following the Model-View-Controller-based J2EE Model 2 design paradigm.
2.4	MeSH Query Expansion
2.4.1	Description
The Query Expansion component builds on the Medical Subject Headings (MeSH) hierarchy and assists the user in refining the semantic analysis generated by the Query Analysis module of the middle tier. A semantic interpretation of the query document is automatically produced by the system by marking up medical concepts and relations holding among them. The user is presented with a list of these semantic units and can navigate the MeSH hierarchy for selecting a query related context (for example Diabetes Mellitus can be specified in one of the following informational contexts: Endocrine Disease and Metabolic Disease) and any of their semantically related concepts: ancestors, siblings or children.
2.4.2	Input/Output Behaviour
Input: The unique identifier and the string of a medical terminological unit.
Output: A list of ancestors, siblings and children of the input term according to the MeSH hierarchy and grouped by their information context.
2.4.3	Programming Issues
Implemented as an Entity Enterprise Java Bean (EJB) component and designed as a Model, following the Model-View-Controller-based J2EE Model 2 design paradigm.
2.5	Document Manager
2.5.1	Description
The back-end search engines are using a common document denomination scheme, dismissing the need of transporting whole documents over the network connection. Instead, lists of document IDs are being used to denote the retrieval results. For this reason, the Client Tier maintains its own copy of the document collection, which is being managed by the Document Manager component. This component is in charge of retrieving from a native XML database a document or a set of documents according to an ID or list of IDs provided as input.
2.5.2	Input/Output Behaviour
Input: A document ID or a list of document IDs.
Output: The title and content of the provided input in XML format.
2.5.3	Programming Issues
Implemented as a Session Enterprise Java Bean (EJB) component and designed as a Model, following the Model-View-Controller-based J2EE Model 2 design paradigm. It is using a native XML database as storage repository for the documents.
2.6	Summariser
2.6.1	Description
The summarizer is based on the Maximal Marginal Relevance method of extracting the most relevant and diverse passages from a text. For the purposes of being able to provide summaries when multiple documents are retrieved, the summarizer has been adapted to work with individual texts or with clusters of topically related texts. This does not extend, however, to summarization across languages. Thus, in the web-based demonstrator, the summarizer is invoked once for each language present in the search results.
2.6.2	Input/Output Behaviour
Input: The set of document IDs over which a summary is desired.
Output: The text of the summary.
2.6.3	Programming Issues
The summarization code was written in C++, in accordance with CMU's existing library of IR and IR-related routines.  The summarizer program, which matches the summarization code with data retrieval and the rest of the search engine, is also written in C++.  
2.7	Presentation Manager 
2.7.1	Description
The user interacts with the system through several views, corresponding to stages of processing in an information retrieval environment: query formulation, query expansion, hit-list retrieval and document exploration. For query formulation an upload view assists the user in defining his information need either by uploading a system file or by editing a query. The expansion view puts forward functionalities for tuning the query by choosing the appropriate terminology from a list of automatically extracted concepts, expanding the query by selecting concept related terms from an external terminological inventory (MeSH) or by manually adding new terms. The list of query-relevant retrieved information presents titles along with document excerpts and a summarization option for sets of documents. Finally, an exploration view presents each retrieved document as text with a corresponding list of domain-specific terms and relations that were found (marked up) in it.
2.7.2	Input/Output Behaviour
Input: A request for displaying a view according to a Task Manager handled task.
Output: An HTML page.
2.7.3	Programming Issues
Implemented as a collection of Java Server Pages (JSP), Servlet Filters and HTML pages and designed as a View Dispatcher, following the Model-View-Controller-based J2EE Model 2 design paradigm. The Servlet Filters are using XSLT style sheets to transform the XML based responses into HTML.

3	
Relations Filtering

Middle Tier
3.1	
Relations Tagging

Relations Extraction

Q&DA Controller

Shallow Processing

Semantic Relations

Word Sense Disambiguation

Terminology

Term Extraction

Term Tagging

Overview
The middle tier of the system covers an annotation system through which text (queries and documents) may be annotated with several layers of linguistic and semantic analysis. In the context of the MUCHMORE objectives, semantic annotation of medical terms and semantic relations between such terms is primary among these. Nevertheless, a number of additional levels of annotation were deemed to be necessary in order to facilitate various intermediate objectives, e.g. (bilingual) term extraction, relation extraction and sense disambiguation. Additional annotation includes part-of-speech tagging, lemmatizing (morphological analysis) and chunking. 
3.2	Query & Document Analysis Controller
3.2.1	Description
The Query & Document Analysis Controller is the interface between the client tier and the middle tier. It controls the workflow of the middle tier, dispatching a query document to its underlying processing tools. The query undergoes both a linguistic and a semantic analysis, being reduced to an intermediary representation similar to the document collection representation. 
3.2.2	Input/Output Behaviour
Input: A query document.
Output: A XML representation of the query. 
3.2.3	Programming Issues
Implemented as a Java Servlet and designed as a Controller, following the Model-View-Controller-based J2EE Model 2 design paradigm. Given the fact that some underlying components have been implemented in Perl, the Query and Document Analysis Controller wraps these in an Enterprise Java Bean (EJB) component and access them using CORBA services.
3.3	Shallow Processing
3.3.1	Description
The Shallow Processing Tools (ShProT) are a flexible, language-independent toolkit for the shallow analysis of written texts. The toolkit comprises the following tools:

•	Tokenizer
•	Statistical part-of-speech tagger
•	Morphological analyser with a fullform lexicon
•	Chunker

You can use all these components or just some of them. In particular, you may use the toolkit as a tagger, lemmatiser or a chunker and combine it with your own tools.
The current version uses a tokenizer implemented on the basis of the finite-state tools. The POS tagger is a C++ port of  the TnT tagger (Brants, 2000). The morphology is based on a fullform lexicon. The analyzer tries to find an appropriate fullform entry for each input word. In case of ambiguities, information provided by the POS tagger is taken into account. If there is no matching fullform, the analyser tries to decompose the word into fullforms using a simple decomposition algorithm. The chunker is a re-implementation of the HMM-based shallow parser chunkie (Skut and Brants, 1998).
3.3.2	Input/Output Behaviour
Input: A text document.
Output: An XML annotated document. The mark-up consists of linguistic features of the input document at the following levels: tokens, part-of-speech, morphology and phrases (chunks).
3.3.3	Programming Issues
Implemented as a Remote Method Invocation (RMI) Java server and designed as an Aggregate Object with most of its subcomponents as C/C++ modules.
3.4	Term Tagging
3.4.1	Description
The functionality of the tool is to annotate domain-specific terms in input texts. The texts must be linguistically analysed with ShProT prior to semantic annotation (tokenisation, POS-tagging, lemmatisation and compound analysis, chunking). 
The input stream is read sentence by sentence; from each sentence units of 1-4 tokens are considered. A part-of-speech filter is used to prevent non-lexical word tokens from being mistakenly matched. If a token sequence is found to be in the MRCON database, an entry <umlsterm> is generated, which refers to the actual tokens by their indices. Each term is assigned a Concept Unique Identifier from the MRCON database, a Semantic Type (TUI) from the MRSTY database and a MeSH descriptor from the MRSAT database. All of these semantic identifiers may be ambiguous, meaning that one term string may have several CUIs, one CUI may be assigned several TUIs and one CUI may be mapped to several MeSH codes.
3.4.2	Input/Output Behaviour
Input: FILE with linguistic annotation in MUCHMORE XML
Output: FILE with annotated terms in MUCHMORE XML format with an added UMLS terms layer
3.4.3	Programming Issues
The term tagger is written in Perl and requires DB_File and XMLParser modules. The first level is the identification of medical terms contained in the MRCON file of the UMLS Metathesaurus. For the purposes of faster and more efficient lookup, MRCON has been pre-processed with the lemmatisation tool and stored in a DB_File hash.
3.4.4	Notes
Parameters that influence the quality of term tagging are the following:
•	quality of lemmatisation and decompounding
•	term length (only terms of length up to 4 tokens are matched)
•	part-of-speech filters (all verbs, pronouns and conjunctions are excluded)
3.5	Term Extraction
3.5.1	Description
The goal of this module is to identify candidate terms in running texts, that are not part of existing resources (the UMLS Metathesaurus in our case). For identifying German and English candidate terms we rely on the following characteristics:
o	Single words which appear in the thesaurus (for alignment purposes) or which contain English morphemes extracted from The Specialized Lexicon found in UMLS and translated in German. Since the Specialized Lexicon only provides English morphemes, we created a list of German morphemes by translating the English ones (thus -ectomy is translated into -ektomie),
o	Syntactic patterns corresponding to [ADJ* NOUN GEN*] for German, and all non recursive noun phrases for English, i.e. noun phrases of the form [(ADJ* N*)* N], where ADJ stands for an adjective, NOUN for a noun, GEN for a genitive phrase, and * for the Kleene star.
Our morpheme list contains 40 elements, some of which are general, -ion, -ung, but the majority of which is specific to the medical domain, -ektomie, -itis. The syntactic patterns match nouns which occur with a complement (adjective and/or genitive structures). 
3.5.2	Input/Output Behaviour
Input: tagged and lemmatized corpus
Output: input corpus annotated with candidate terms
3.5.3	Programming Issues
This module is based on Xerox Finite State Transducers tools. The deliverables associated to term extraction consist of an augmented annotated corpus. The quality of the term extraction module directly depends on the quality of the part-of-speech tagger and lemmatizer used in the pre-processing steps.
3.6	Semantic Relations Tagging
3.6.1	Description
Relations are identified on the basis of the SRSTRE1 file of the UMLS Semantic Network, where they are defined in the form of triplets - two semantic types linked with one of the 52 medical semantic relations. The database is language independent and stored as a DB_File hash in srstre1.dbm. Annotation is performed sentence-wise, first collecting all semantic type codes (TUIs) from the <umlsterm> and <xrceterm> elements, then generating all combinations and looking them up in the database. If a relation is found the element <semrel> is created, listing the indices of both <umlsterm> or <xrceterm> elements and giving the type of relation.
3.6.2	Input/Output Behaviour
Input: A text, previously processed with linguistic annotation and UMLS term tagging in MUCHMORE XML format
Output: A text in MUCHMORE XML format with an added UMLS semantic relations layer
3.7	Semantic Relations Filtering
3.7.1	Description
Relation filtering is a step aimed at filtering semantic relations that were previously identified in texts by the semantic annotation tool and removing those that are likely to be false or irrelevant in the given context. The filtering is performed on the basis of verbs, which are presumed to indicate certain semantic relations and contraindicate others.
3.7.2	Input/Output Behaviour
Input: An XML document in the MUCHMORE format containing semantic relations like the following:

<semrel id="r1" term1="t1.1" term2="t2.1" reltype="location_of"/>

Required additional input is the file containing verbs with their associated relations and frequencies, which is pre-processed from the corpus and has the following format:

Example:

accumulate
-----------------
272	interacts_with
181	affects
102	associated_with
66	result_of
51	produces
45	issue_in
40	degree_of
33	disrupts
24	diagnoses
22	measures
18	location_of
16	causes

For the Springer corpus, the file names are verb_relation_eng and verb_relation_ger. The language option used in the program call provides that the correct file is loaded.
Output: A file FILE.filt1, where relations not associated with any of the verbs found in the sentence are removed.
3.7.3	 Evaluation 
Results of relation filtering were not manually evaluated. Experiments in document retrieval showed that filtering has no effect on the average precision.
3.7.4	Notes
Currently the script takes the first 8 relations associated with the verb, whereby linking verbs are not considered. For more rigorous filtering, this value may be set to a lower number of relations.
3.8	Semantic Relations Extraction
3.8.1	Description
The aim of the relation extraction tool is to identify semantic relations between UMLS concepts, especially such not yet contained in the Semantic Network of the UMLS. This is based on co-occurrences of MESH codes (codes of MeSH descriptors) found within a sentence, whereby the MeSH codes are stripped to their second-level generality (e.g. D4.124.476 is transformed to D4). Co-occurrences of these MeSH semantic pairs are taken to signify relations, which are labelled according to the most frequently observed UMLS relation occurring with a particular pair. 
The tool is designed in two versions: in the first version (annotate_relation_add.pl) old relations are kept and new ones are added. The distinguishing mark between them is the attribute status="new|old". In the second version, old UMLS-based relations are deleted and replaced by the new ones.
3.8.2	Input/Output Behaviour
Input: An XML document in the MUCHMORE format containing annotations of concepts, terms and semantic relations. Required is also a pre-processed hash (HASH.dbm) in the following format:
NAME_OF_RELATION => LIST_OF_MESH_PAIRS
The script to produce this hash from corpus is create_meshpair_base.pl.
Output: An XML document with relations added or replaced (FILE.relations).
3.8.3	Evaluation
The results of relation extraction were not manually evaluated. However, experiments in document retrieval showed better recall and precision with the added relations. 
3.8.4	Notes
The main tuning parameter is the minimum frequency of the MESH-pair that still signifies a relation. This is currently set to 150, a higher frequency will cause fewer relations to be extracted and vice versa. The parameter can be modified in the script that creates the frequency hash (create_meshpair_base.pl).
3.9	Word Sense Disambiguation
3.9.1	Description
The aim of the word sense disambiguation component is to select the most appropriate sense for a given ambiguous word or term. Disambiguation works on both the EuroWordNet and UMLS annotation. 
In the case of EuroWordNet, one or more senses (“offsets” of EuroWordNet synsets) are removed in disambiguation. For instance for the word corresponding to “w3”:

<ewnterm id="e1" from="w3" to="w3">
 <sense offset="6431241" /> 
 <sense offset="6434591" /> 
</ewnterm>

In UMLS annotation, one or more concepts that correspond to ambiguous UMLS terms are removed, e.g. for the term corresponding to “w11”:

<umlsterm id="t1" from="w11" to="w11" lemmaref="2" type="Head">
 <concept id="t1.1" cui="C0036690" preferred="Septikaemie" tui="T033 T047"> 
  <msh code="C1.539.757.720" /> 
 </concept>
 <concept id="t1.2" cui="C0243026" preferred="Sepsis" tui="T047">
  <msh code="C1.539.757" /> 
 </concept>
</umlsterm>
3.9.2	Input/Output Behaviour
Input: An XML document in the MUCHMORE format with annotation of UMLS concepts and EuroWordNet senses
Output: An XML document in the MUCHMORE format with zero or more concepts and EuroWordNet senses removed 
3.9.3	Evaluation
Evaluation of the implemented word sense disambiguation methods is described in report D5.1 
3.9.4	Programming Issues
The word sense disambiguation component is fully implemented in Java.

4	Back-End Tier 
4.1	Overview
The back-end tier of the system models the search engine component of an information retrieval application. Figure 4 shows the architecture of this tier and its components.
 
Figure 4. Back-end Tier Architecture

Aiming to provide a framework for integrating cross-language information retrieval systems, the back-end tier builds on the power of XML as an integration layer. An XML format has been defined for representing the results of retrieval in a common way, based on which a specialized component generates a merged result list.
The Engine Controller interfaces the communication between the client and back-end tiers. On receipt of a query request from the client tier, the controller generates HTTP requests for each of the considered search-engines and dispatches them concurrently. The Hit List Merger component combines the resulting document lists, re-weighting and re-scoring the retrieved results, and returns the new ranked result list to the Engine Controller.

XML

XML

Hit List Merger

Engine Controller

CSLI Search Engine

EIT Search Engine

CMU Search Engine

4.2	Engine Controller
4.2.1	Description
The main unit of the back-end tier, the Engine Controller dispatches requests on information to remote search engines. Depending on the type of the search engine, the Controller makes requests in form of terms (diabetes mellitus) or concept codes and semantic relations (C00123 issue_in C02341, C00123 associated_with C03541). The search engines are responsible for retrieving lists of matched documents by either using corpora-based or semantic-based methods and return them back to the Engine Controller in a common format (document ID, rank and relevance to the request).
4.2.2	Input/Output Behaviour
Input: A request of information in form of terms, concept codes and semantic relations holding among concepts.
Output: A list of document IDs along with their ranking and relevance to the request.
4.2.3	Programming Issues
Implemented as a Java Servlet using HTTP network connections to access the remote search engines.
4.3	Hit-List Merger
4.3.1	Description
Aiming to provide a common user interface for the retrieval task, the system combines several search methods and integrates their results. As the search engines use different scoring schemes for marking the relevance of a retrieved document to the query, the Hit-List Merger component provides the business logic for combining their results into a single result list.
4.3.2	Input/Output Behaviour
Input: Several lists of document IDs along with their ranking and score (in XML format).
Output: A single merged list of document IDs along with their ranking and score (in XML format).
4.3.3	Programming Issues
Implemented as a Java Servlet and build upon a SAX Java Parser.
4.4	CSLI Search Engine
4.4.1	Description
The CSLI bilingual search engine enables a user to enter a query in English or German and to retrieve related terms and documents in both languages. The search engine is trained on the bilingual Springer corpus, by considering each document and its translation as a single compound document. A term document matrix can then be built which measures the extent to which a term in either English or German occurred in a particular document. Singular value decomposition is then used to reduce this information to 100 dimensions, giving each term a vector of 100 coordinates. Document vectors are then computed by summing the vectors of their constituent words. The cosine similarity measure is then used to compute the similarity between terms and documents. Since all the terms have vectors in a single vector space, the same process can be used to match terms and documents of different languages.
4.4.2	Input/Output Behaviour
Input: The input for building the model is the Springer corpus, with filenames chosen so that translated documents have the same names with .eng or .ger suffixes. No other annotation or linguistic processing is needed. An interactive search engine gives the input for particular queries where a user can enter positive and negative terms in English and German and have these matched to related terms or documents.
Output: A list of matched document IDs, along with their rankings and relevance scores.
4.4.3	Evaluation
The success of the system for term-translation was evaluated by comparing translations obtained by the model (by matching a word to its nearest neighbour in the other language) with those given by UMLS. In the high-confidence region (cosine similarity >0.85) terms were correctly translated with a precision of over 90%. Terms, which were marked “incorrect” by UMLS often, turned out to be correct translations, which were not yet in UMLS. In all, over 300 new terms and synonyms would be added to UMLS in this way.
4.4.4	Programming Issues
The bilingual vector space model is implemented in native C code. The search engine interface (http://infomap.stanford.edu/bilingual) is written in Perl and uses CGI.
4.5	CMU Search Engine
4.5.1	Description
The cross-lingual capability of the search engine is affected by means of translating the query into the language of the corpus being searched, and employing standard monolingual retrieval techniques using the translated query.  The Lemur framework is used to provide monolingual retrieval capabilities, as well as query expansion (via pseudo-relevance feedback), both before and after translation.  Lemur's Okapi retrieval model functionality, employing a variant of the BM25 formula for term weighting, is used throughout, as this provided better performance than the TFIDF modes.  Two approaches proved useful for query translation, both based on using parallel training corpora: Pseudo-Relevance Feedback (PRF) and Example-Based Thesaurus (EBT).  
In order to return results in multiple languages, or to search with queries in multiple languages, the search engine is run once for every unique pair of (source language, destination language) values.  For example, if both a German and English query are specified, and the results are to be in English, then the search engine would run once in a monolingual mode to get results based on the English query, and once in cross-lingual mode, translating the German query to English and using that to return English results as well.  Multiple result lists in the same language are merged, sorted by score, and the highest-ranking returned.
4.5.2	Pseudo-Relevance Feedback (PRF)
The PRF translation method uses Lemur's built-in monolingual pseudo-relevance feedback retrieval mechanism to first retrieve documents in the query language from the training corpus of the same language.  The translated query in the target language is then assembled by feeding the set of document IDs retrieved back into Lemur's pseudo-relevance feedback mechanism, this time based on the target language and associated corpus. This results in a query in the target language.  Lemur's Okapi model is used here as well.
4.5.3	Example-Based Thesaurus (EBT)
The EBT translation method employs an off-line thesaurus generator that associates terms in the source and target languages by scoring the terms found in the parallel training corpora with either the chi-square formula or by point-wise mutual information, as a tuneable option.  Translation then becomes a matter of looking up the terms found in the original query and composing the entries found in the thesaurus into a query in the target language.
4.5.4	Input/Output Behaviour
Input: Textual query, language of the query, desired result language.
Output: A ranked list of document IDs and scores from the search space corpus of the desired result language.
4.5.5	Evaluation
Average precision scores are at or above .56 for all cross-lingual and monolingual cases, when using the default (preferred) PRF translation engine.  The monolingual English queries / English results case is especially good, with an average precision above .60
The tables below, provided for reference, summarize the performance for relevant "term-match" (i.e. traditional search engine) cases. Evaluation results for "cat-match" cases were not competitive and are not included.  Scores were generated by running the standard set of Springer queries through the search engine, and then using the standard TREC-provided evaluation program to obtain top-20 precision (t20p column) and TREC average precision (avgp column) scores according to a set of relevance judgements.  The first table ("Using ZInfo Relevance Judgements Only") was generated using the set of German-only judgements provided by ZInfo.  The second table ("Using OR of CMU Judgements") was generated using a set of judgements compiled at CMU. This second table is included for comparison to earlier CMU performance reports, which also used this set of judgements.  Both tables contain both monolingual ("E -> E" and "G -> G") performance as well as cross-lingual performance for our two top-performing translation methods, PRF and EBT.

Using ZInfo Relevance Judgements Only		Using OR of CMU Judgements
--------------------------------------------------		---------------------------------------
Case / Engine       t20p   avgp				Case / Engine       t20p   avgp	
-----------------       ----   -------				-----------------      ------  -------
E -> E:                  .516   .603				E -> E:         	      .524   .648
G -> G:                 .480   .569				G -> G:       	      .454   .551
E -> G (PRF):       .516   .566				E -> G (PRF):       .486   .553
            (EBT):       .476   .560				(EBT):       .466   .537
G -> E (PRF):       .470   .567				G -> E (PRF):       .450   .598
            (EBT):       .418   .494				(EBT):       .414   .528
4.5.6	Programming Issues
The search engine was written to be served as a collection of web pages, in order that it be accessible to all platforms with a web browser, and because, despite substantial differences in browsers rendering engines, web page based interfaces are easier to create and modify and tend to be more consistent and less "clunky" than ones written in Java or other web-based options.  This front end of the search engine is accomplished by way of a set of Perl CGI scripts that dynamically generate HTML and JavaScript content in response to parameters passed by the URL or by POST requests from HTML forms.  To keep this set of server scripts simple, a stateless approach was chosen, wherein all state information (list of results, which to summarize, etc.) is maintained in the URL or POST parameters.  Cookies are used to store user preferences. 
The CGI scripts make calls to back end C++ programs to perform summarization and to run user queries.  Existing libraries, notably the Lemur framework used for indexing and retrieval when running queries, dictated the choice of C++ for this portion.  However, the need for a high-performance engine would have precluded the use of Perl or other scripting languages for this part, regardless.  Although the summarizer and back end search engine have been used only on the Intel-based Linux systems on which they were developed, none of the code is known to be platform-specific, and it is expected that the entire search engine should be able to run on any suitable platform.
4.6	EIT Search Engine
4.6.1	Description
The Eurospider contribution to MuchMore is based on the “relevancy” system. The installation employed for MuchMore allows the user to search with natural-language queries, concepts and semantic relations. ”Relevancy” features flexible indexing and weighting components. For indexing, the process of extracting tokens for subsequent search from queries and documents, various normalization, reduction and enhancement operations are supported. For weighting, the process of ranking documents that are retrieved in response to a query, several weighting mechanisms are implemented, both based on a rule-set and on well-known weighting schemes from information retrieval research.
For use in MuchMore, the system was adopted to index the information provided in the annotation of MuchMore documents and queries: word forms, lemmas, and part-of-speech information for both German and English.
Furthermore, semantic information, such as EuroWordNet terms, UMLS terms, semantic relations and MeSH terms, was indexed.
Cross-lingual information retrieval functionality was implemented by using either a corpus-based similarity thesaurus, a data resource that exploits contextual similarities between terms in different languages derived from suitable training data, by using semantic information, or by using commercial machine translation systems. 
4.6.2	Input/Output Behaviour
Input:
Natural-language query, if desired concepts and semantic relations

Output:
Ranked list of documents and their retrieval scores.
4.6.3	Evaluation
Evaluation was conducted by analyzing results in terms of Mean Average Precision (MAP), total number of relevant documents retrieved, average precison at 10% recall, and precision after 10 documents (P10).
Monolingual experiments were conducted in both German and English to assess the benefit of using word form normalization and the different types of semantic information available. For German, we determined that good word form normalization is indeed crucial, increasing mean average precision by up to 76% and total recall of relevant documents by 84% when carefully tuned. Semantic information is not always helpful; the most benefit (+2% MAP) was found when using MeSH codes. For English, the benefit of using word form normalization is much smaller, with mean average precision even down by 3%, and total recall up 3% compared to using the unnormalized tokens. Again, using the semantic information gains little; with the MeSH codes being most helpful (+5%). 
The best results for cross-language retrieval were found when the similarity thesaurus was combined with all semantic information contained in the documents. In this case, retrieval effectiveness was between 80-90% of monolingual performance in terms of mean average precision for German to English cross-lingual retrieval. In terms of total number of relevant documents retrieved, the best combinations even outperform the monolingual experiments. The best result when using commercial machine translation was found when a special add-in component handling medical terminology was used; the performance in that case was approx. 66% MAP of the monolingual baseline. Less effort was spent on fine-tuning English to German cross-lingual retrieval. We found a retrieval performance of approx. 79% MAP compared to the monolingual baseline.
4.6.4	Programming Issues
The system had to be configured to make use of all information contained in the XML annotation format. This was accomplished by suitably programming the indexing component. The various types of semantic information and translation resources required suitable weighting and combination mechanisms. Care was taken that the flexibility of the relevancy system could be exploited as much as possible.

References
Brants, T. 2000. TnT - A Statistical Part-of-Speech Tagger. In: Proceedings of 6th ANLP Conference, Seattle, WA.

Cavnar William B. and John M. Trenkle. N-Gram-Based Text Categorization. Symposium on Document Analysis and Information Retrieval, Las Vegas, 1994.

Skut W. and Brants T. 1998. A Maximum Entropy partial parser for unrestricted text. In Proceedings of the 6th ACL Workshop on Very Large Corpora (WVLC), Montreal.

Appendix A: Quantified Data

 	Curently achieved
quantity	Estimated future
quantity	
Product innovations			
Process innovations			
New services (commercial)			
New services (public)			
New methods			
Scientific breakthrough			
Technical standards to which this project has contributed			
EU regulations/directives to which this project has contributed			
International regulations to which this project has contributed			
PhDs generated by the project			
Grantees/trainees including transnational exchange of personnel			

•	New Services (Public): Cross-Lingual access to medical literature
•	New Methods: Advances in Hybrid Approaches to CLIR, Term Extraction from Comparable Corpora, Unsupervised Sense Disambiguation
•	Technical Standards: Expected contribution to definition of ISO standard on linguistic annotation (ISO/TC37/SC4: International Standards of Terminology and Language Resource Management), based on the MuchMore annotation format
•	PhDs (partly) due to MuchMore: Jade Goldstein (CMU); Špela Vintar, Detlef Prescher (DFKI); Fatiha Sadat (XRCE)

Appendix B: Expected Project Impact

EU policy Goals	1. Scale of expected impact over the next 10 years	2. Other	
	-1 0 1 2 3 (*)	Not applicable to project	Project Impact too difficult to estimate
	
1. Improved sustainable economic development and growth, competitiveness					
2. Improved employment					
3. Improved quality of life and health and safety					
4. Improved education 					
5. Improved preservation and enhancement of the environment					
6. Improved scientific and technological quality					
7. Regulatory and legislative environment					
8. Other					
(*) Indication for scale as follows: -1 represents negative impact, 0 no impact, 1 small positive impact, 2 medium positive impact , 3 is a strong positive impact

Appendix C: European Interest
•	European dimension of the problem (The extent to which the project has contributed to solve problems at European level):

The MuchMore project aimed at providing methods and tools for cross-lingual access to medical literature, which addresses an important multilingual issue in the European context. Medical literature available in one language should be made accessible to medical professionals whose proficiency is in another language.

•	Contribution to developing S&T co-operation at international level. European added value (Development of critical mass in human and financial terms; combination of complementary expertise and resources available Europe-wide)

The MuchMore project brings together complimentary expertise from different European countries: Germany (language technology, medical informatics), France (language technology) and Switzerland (information retrieval). Additional expertise is brought in from the US (language technology, computer science, information retrieval), which enhances the global reach of the developed technology.

•	Contribution to policy design or implementation (Contribution to one or more EU policies; RTD connected with standardisation and regulation at Community and/or national levels)

Expected contribution to the definition of an ISO standard on linguistic annotation (ISO/TC37/SC4: International Standards of Terminology and Language Resource Management), based on the annotation format developed within the MuchMore project.

•	Improving the quality of life in the Community:

The MuchMore project aims at increasing the accessibility of medical literature, which is an important aspect in improving health care within the European Community.  

•	Provision of appropriate incentives for monitoring and creating jobs in the Community (including use and development of skills):

Commercial services that could be build on the methods and technology developed within the MuchMore project would create new job possibilities.

•	Supporting sustainable development, preserving and/or enhancing the environment (including use/conservation of resources):

N/A