[Date Prev][Date Next][Date Index]

[ntcir:372] CFP - CLEF 2010 - Conference on Multilingual and Multimodal Information Access Evaluation

Call for Papers

CLEF 2010 - Conference on Multilingual and Multimodal Information Access

20–23 September 2010, Padua, Italy



CLEF2010 is the continuation of the popular CLEF campaigns that have run
for the past ten years. It will cover a broad range of issues in the
fields of multilingual and multimodal information access evaluation. It
will consist of two main parts: a peer-reviewed conference (20-21
September) on experimental evaluation, which will innovate the CLEF
tradition, and a series of labs (22-23 September), which will continue
the CLEF tradition of community-based evaluation and discussion on
evaluation issues.

Aim and Scope

The CLEF2010 conference aims at advancing the evaluation of complex
multimodal and multilingual information systems in order to support
individuals, organizations, and communities who design, develop, employ,
and improve such systems.

The growth of the Internet has been exponential with respect to the
number of users, media, and languages used regularly for global
information dissemination. Language and media barriers are no longer
seen as inviolable and they are constantly crossed and mixed to provide
content that can be accessed on a global scale within a multicultural
and multilingual setting.

Users need to be able to co-operate and communicate across language and
media boundaries, going beyond separate search in diverse
media/languages and exploiting interactions between different languages
and media.

Experimental evaluation - both laboratory and interactive - is a key to
fostering the development of multilingual and multimodal information
systems that address increasingly complex information needs.

The CLEF2010 labs will continue the CLEF tradition of community-based
benchmarking and will complement these with workshops on emerging issues
in evaluation methodology. Lab proposals were sollicited at an earlier
stage and and the 2010 lab program can be found at

At this time we invite submissions for presentation at the CLEF2010
conference. We welcome submissions on all aspects of multilingual and
multimodal information access evaluation. All submissions will be
reviewed on the basis of relevance, originality, importance, and clarity.

The conference proceedings will be published in the Springer Lecture
Notes in Computer Science (LNCS) series.

Topics for Discussion

Relevant topics for the CLEF 2010 include, but are not limited to:

- Novel methodologies for the design of evaluation tasks, especially
user-centric ones;
- Analysis of the impact of multilingual/multicultural/multimodal
differences in interface and search design;
- Assessing multilinguality and multimodality in relevant application
communities, e.g. digital libraries, intellectual property, medical,
music, video, and social media.
- Alternative methods for improving and automating ground-truth
creation, for example crowd-sourcing or clicklog-based;
- Prediction of success and satisfaction rates;
- Task-oriented metrics of success and failure;
- Evaluation of technology vs testing of scientific theories;
- Innovative and easy to communicate techniques for analysing the
experimental results, including statistical analyses, data mining, and
information visualization;
- Alternatives for and comparison of item-based, list-based, set-based,
and session-based evaluation;
- Simulation (of queries, sessions, users) and information retrieval;
- Infrastructures for bringing automation and collaboration in the
evaluation process;
- Component-based evaluation approaches;
- Evaluation and analysis using private or anonymized test data;
- Living laboraties and evaluating live systems.

Important Dates

- Submission Deadline: 2 May 2010
- Notification of Acceptance: 11 June 2010
- Camera Ready: 25 June 2010
- Conference: 20-21 September 2010.


Authors are invited to submit electronically original papers, which have
not been published and are not under consideration elsewhere, using the
LNCS proceedings format

Two types of papers are solicited:
- long papers: 12 pages max;
- short papers: 6 pages max.

Papers will be peer-reviewed by at least 3 members of the program
committee. Selection will be based on originality, clarity, and
technical quality.

Papers should be submitted in PDF format to the following address:


Honorary Chair
- Carol Peters, ISTI-CNR Pisa, Italy

General Chairs
- Maristella Agosti, University of Padua, Italy
- Maarten de Rijke, University of Amsterdam, The Netherlands

Programme Chairs
- Nicola Ferro, University of Padua, Italy
- Alan Smeaton, Dublin City University, Ireland

Organization Chair
- Emanuele Pianta, CELCT Trento, Italy

Program Committee

- Eneko Agirre, University of the Basque Country, Spain
- Giambattista Amati, Fondazione Ugo Bordoni, Italy
- Julie Berndsen, University COllege Dublin, Ireland
- Pia Borlund, Royal School of Library and Information Science, Denmark
- Pavel Braslavski, Yandex and Ural State Tech. University, Ural State
University, Russia
- Chris Buckley, Sabir Research, USA
- Pável Calado, Superior Technical Institute, Portugal
- Tiziana Catarci, Sapienza, University of Rome, Italy
- Stefano Ceri, Politecnico di Milano, Italy
- Charlie Clark, University of Waterloo, Canada
- Bruce Croft, University of Massachusetts, Amherst, USA
- Franca Debole, Institute of Information Science and Technologies
(ISTI), Italian National Research Council (CNR) , Italy
- J. Stephen Downie, University of Illinois at Urbana-Champaign, USA
- Susan Dumais, Microsoft Research, USA
- Floriana Esposito, University of Bari, Italy
- Marcello Federico, Fondazione Bruno Kessler, Italy
- Norbert Fuhr, University of Duisburg, Germany
- Fredric Gey, University of California, Berkeley, USA
- Marcos André Gonçalves, Federal University of Minas Gerais, Brasil
- Julio Gonzalo, National Distance Learning University, Spain
- Gregory Grefenstette, Exalead, France
- Allan Hanbury, Information Retrieval Facility (IRF), Austria
- Peter Ingwersen, Royal School of Library and Information Science, Denmark
- Perla Innocenti, University of Glasgow, UK
- Kalervo Järvelin, University of Tampere, Finland
- Gareth Jones, Dublin City University, Ireland
- Theodore Kalamboukis, Athen University of Economics and Business, Greece
- Leonid Andreevich Kalinichenko, Institute for Problems of Informatics
of the Russian Academy of Sciences, Russia
- Noriko Kando, National Institute of Informatics (NII), Japan
- Sarantos Kapidakis, Ionian University, Greece
- Jussi Karlgren, Swedish Institute of Computer Science (SICS), Sweden
- Kazuaki Kishida, Keio University, Japan
- Mikko Kurimo, Helsinki University of Technology, Finland
- Mounia Lalmas, University of Glasgow, UK
- Ray Larson, University of California, Berkeley, USA
- Chin-Yew Lin, Microsoft Research Asia, China
- Saturnino Luz, Trinity College Dublin, Ireland
- Bernardo Magnini, Fondazione Bruno Kessler, Italy
- Prasenjit Majumder, Dhirubhai Ambani Institute of Information and
Communication Technology, India
- Thomas Mandl, University of Hildesheim, Germany
- Paul McNamee, Johns Hopkins University, USA
- Mandar Mitra, Indian Statistical Institute, India
- Stefano Mizzaro, University of Udine, Italy
- Alistair Moffat, University of Melbourne , Australia
- Viviane Moreira Orengo, Federal University of Minas Gerais, Brasil
- Henning Müller, University of Applied Sciences Western Switzerland,
- Jian-Yun Nie, University of Montreal, Canada
- Doug Oard, University of Maryland, USA
- Nicola Orio, University of Padua, Italy
- Christos Papatheodorou, Ionian University, Greece
- Gabriella Pasi, University of Milan Bicocca, Italy
- Anselmo Peñas, National Distance Learning University, Spain
- Vivien Petras, Humboldt University, Germany
- Jean-Michel Renders, Xerox Research Centre (XRCE), France
- Seamus Ross, University of Toronto, Canada
- Ian Ruthven, University of Strathclyde, UK
- Tetsuya Sakai, Microsoft Research Asia, China
- Diana Santos, SINTEF Information and Communication Technology, Norway
- Giuseppe Santucci, Sapienza, University of Rome, Italy
- Frederique Segond, Xerox Research Centre (XRCE), France
- Giovanni Semeraro, University of Bari, Italy
- Paraic Sheridan, Centre for Next Generation Localisation, Ireland
- Ian Soboroff, National Institute of Standards and Technology (NIST), USA
- Alexander Sychov, Voronezh State University, Russia
- John Tait, Information Retrieval Facility (IRF), Austria
- Letizia Tanca, Politecnico di Milano, Italy
- Elaine Toms, Dalhousie University, Canada
- Andrew Trotman, University of Otago, New Zealand
- Natalia Vassilieva, HP Labs Russia, Russia
- Felisa Verdejo, National Distance Learning University, Spain
- Ellen Voorhees, National Institute of Standards and Technology (NIST), USA
- Gerhard Weikum, Max-Planck Institute for Informatics (MPII), Germany
- Christa Womser-Hacker, University of Hildesheim, Germany
- Justin Zobel, University of Melbourne , Australia

This message is brought by NTCIR and distributed through the mailing list:
ntcir [at] nii.ac.jp for announcements and updates about NTCIR.

* To unsubscribe, send mail to ntc-secretariat [at] nii.ac.jp,
with the subject:ntcirML UNSUBSCRIBE [your e-mail address]

* For more info, visit:
Noriko Kando, Leader, NTCIR Project
ntc-admin [at] nii.ac.jp

NTCIR Project Office
ntc-secretariat [at] nii.ac.jp

National Institute of Informatics
2-1-2 Hitotsubashi,
Chiyoda-ku, Tokyo 101-8430, Japan