Welcome to CLEF 2007 Carol Peters ISTI-CNR Pisa, Italy.

  • Published on
    27-Mar-2015

  • View
    216

  • Download
    3

Transcript

  • Slide 1

Welcome to CLEF 2007 Carol Peters ISTI-CNR Pisa, Italy Slide 2 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF Objectives Stimulate the development of multilingual IR systems for European languages To create a CLIR/MLIA community Construct publicly available test-suites Conducting annual evaluation campaigns Designing tracks/tasks to meet emerging needs and to stimulate research in theright direction Slide 3 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF Coordination Centre for the Evaluation of Human Language and Multimodal Communication Technologies (CELCT), Trento, Italy College of Information Studies and Institute for Advanced Computer Studies, U. Maryland, USA Dept. of Computer Science, U. Indonesia Depts. of Computer Science & Medical Informatics, RWTH Aachen U., Germany Dept. of Computer Science and Information Systems, U. Limerick, Ireland Dept. of Computer Science and Information Engineering, National U. Taiwan Dept. of Information Engineering, U. Padua, Italy Dept. of Information Sci, U. Hildesheim, Germany Dept. of Information Studies, U. Sheffield, UK Evaluations and Language Resources Distribution Agency Sarl, Paris, France Fondazione Bruno Kessler FBK-irst, Trento, Italy German Research Centre for Artificial Intelligence, DFKI, Saarbrcken, Germany Information and Language Processing Systems, U. Amsterdam, Netherlands IZ Bonn, Germany Inst. For Information technology, Hyderabad, India Inst. of Formal and Applied Linguistics, Charles University, Czech Rep LSI-UNED, Madrid, Spain Linguateca, Sintef, Oslo, Norway Linguistic Modelling Lab., Bulgarian Acad Sci Microsoft Research Asia NIST, USA Biomedial Informatics, Oregon Health and Science University, USA Research Computing Center of Moscow State U. Research Institute for Linguistics, Hungarian Academy of Sciences School of Computer Science and Mathematics, Victoria U., Australia School of Computing, DCU, Ireland UC Data Archive and School of Information Management and Systems, UC Berkeley, USA University "Alexandru Ioan Cuza", IASI, Romania U. Hospitals and U.of Geneva, Switzerland Vienna University of Technology, Austria CLEF is coordinated by the Istituto di Scienza e Tecnologie dell'Informazione, Consiglio Nazionale delle Ricerche, Pisa The following Institutions are contributing to the organisation of the different tracks of the CLEF 2007campaign: Slide 4 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF Steering Committee Maristella Agosti, U.Padove, Italy Martin Braschler, Zurich, Switzerland Amedeo Cappelli, ISTI-CNR & CELCT, Italy Hsin-Hsi Chen, National Taiwan U., Taipei, Taiwan Khalid Choukri, ELRA/ELDA, Paris, France Paul Clough, University of Sheffield, UK Thomas Deselaers, RWTH Aachen University, Germany Giorgio Di Nunzio, U. Padova, Italy David A. Evans, Clairvoyance Corporation, USA Nicola Ferro, U. Padova, Italy Christian Fluhr, CEA-LIST, Fontenay-aux-Roses, France Norbert Fuhr, University of Duisburg, Germany Frederic C. Gey, U.C. Berkeley, USA Julio Gonzalo, LSI-UNED, Madrid, Spain Donna Harman, NIST, USA Gareth Jones, Dublin City University, Ireland Franciska de Jong, University of Twente, Netherlands Noriko Kando, NII, Tokyo, Japan Jussi Karlgren, SICS, Sweden Michael Kluck, German Institute for International and Security Affairs, Berlin, Germany Natalia Loukachevitch, Moscow State University, Russia Bernardo Magnini, ITC-irst, Trento, Italy Thomas Mandl, U. Hildesheim, Germany Paul McNamee, Johns Hopkins University, USA Henning Mller, University & University Hospitals of Geneva, Switzerland Douglas W. Oard, University of Maryland, USA Anselmo Peas, LSI-UNED, Madrid, Spain Maarten de Rijke, University of Amsterdam, Netherlands Diana Santos, Linguateca, Sintef, Oslo, Norway Jacques Savoy, University of Neuchatel, Switzerland Peter Schuble, Eurospider Information Technologies, Switzerland Richard Sutcliffe, University of Limerick, Ireland Max Stempfhuber, Informationszentrum Sozialwissenschaften Bonn, Germany Hans Uszkoreit, German Research Center for Artificial Intelligence (DFKI), Germany Felisa Verdejo, LSI-UNED, Madrid, Spain Jos Luis Vicedo, University of Alicante, Spain Ellen Voorhees, NIST, USA Christa Womser-Hacker, University of Hildesheim, Germany Slide 5 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF 2007: Track Coordinators Ad Hoc: Giorgio Di Nunzio, Nicola Ferro and Thomas Mandl Domain-Specific: Vivien Petras, Stefan Baerisch, Maximillian Stempfhuber QA@CLEF: Danilo Giampiccolo, Bernardo Magnini, Anselmo Peas, Christelle Ayache, Petya Osenova,, Maarten de Rijke, Bogdan Sacaleanu, Diana Santos and Richard Sutcliffe ImageCLEF: Allan Hanbury, Paul Clough, Henning Mller, Thomas Deselaers, Michael Grubinger, Jayashree Kalpathy Cramer, and William Hersh CL-SR: Douglas W. Oard, Gareth J. F. Jones, and Pavel Pecina Web-CLEF: Valentin Jijkoun and Maarten de Rijke GeoCLEF: Thomas Mandl, Fredric Gey, Giorgio Di Nunzio, Nicola Ferro, Ray Larson, Mark Sanderson, Diana Santos, Christa Womser-Hacker, Xing Xie Slide 6 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF 2007: Participating Groups Brown U., USA California State U. SanMarcos, USA** Charles U., Prague, Czech Rep. Daedalus & Madrid Univs, Spain **** Ching Yun Univ., Taiwan DFKI-Artificial Intelligence, DE**** Dokuz Eylul U.,Turkey* Dublin City U. - Comp.Sci., Ireland *** Fondazione Bruno Kessler******** Helsinki U. of Technology Hungarian Acad. Sci. IDIAP Research Inst., CH Imperial College, London, UK** Ist.Nac.Astrofisica, Optica, Electronica, Mexico** Indian Statistical Inst., India* Indian Inst. Technology (IIT-Bombay) Indian Inst. Technology (IIT- Kharagpur) Inst.Infocomm Research, Singapore ** Inst. Superior Tcnco (DEI-IST) IPAL-CNRS (IR2), Singapore **** IRIT / SIG - Toulouse ***** Jadavpur University, Kolkata, India Johns Hopkins U., USA ******* Language Computer Corp., USA* LIMSI-CNRS, France **** Univ. Evora, Portugal ** U.Freiburg Pattern Recog., Germany U. & Hospitals Geneva, CH *** U.Groningen - Inf.Sci, The Netherlands** (2) U.Hagen IICS, Germany **** U.Hildesheim - Inf.Sci, Germany *** * U.Indonesia - Comp.Sci, Indonesia ** U.Jaen - Intell.Systems, Spain ****** U.Liege - Elect.Eng.&CS, Belgium** U.Lisbon Informatics, Portugal *** Univ. Macquarie, Australia Univ. Nacional Colombia U.Neuchatel Informatique, Switzerland ****** Univ. Nottingham, UK U.Ottawa - IT & Eng, Canada* U.Politecnica Catalunya TALP, Spain** U.Politecnica Valencia - Comp.Sci, Spain** U. Porto, Portugal* U.Salamanca REINA, Spain ***** U.Stockholm, NLP, Sweden *** U.Tampere, Fiinland **** U.Wolverhampton, UK * UC Berkeley - IM&S-1, USA ******* UNED-LSI, Spain ****** Univ. West Bohemia, Czech rRp.* Vienna Univ. Technology, Austria Xerox XRCE, France * Linguateca-Sintef, Norway *** Linguit Ltd, UK Microsoft Asia* Microsoft India MRIM Group LIG, Grenoble* Nat. Inst.Informatics, Japan *** Nat.Taiwan U. - Comp-Sci, ***** Open Text Corp.(ex Hummingbird) Oregon Health & Sci. U., USA ** Priberam Informatica, Portugal * Research Inst. for AI of Romaian Academy* RWTH Aaachen-CS., Germany *** RWTH Aachen - Med.Inf., DE*** SUNY Buffalo Informat, USA **** SYNAPSE Dveloppement, France** Tech U. Chemnitz, Germany* Tokyo Inst. Technology, Japan* U.Alicante, Spain (2) ****** U.AI.I Cuza Iasi, Romania* U.Amsterdam - Informatics, N ****** U. Basil, Seitzerland U. Chicago, USA ** U.Concordia - CINDI, Canada** U.Concordia - CLAK, Canada U.Coruna & U.Sunderland, ES/UK* Slide 7 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF: Trend in Participation Europe = 51(59.5); N. America = 14(4.5); Asia = 14(10), S. America = 1(4), Oceania = 1(2) Slide 8 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 multilingual textual document retrieval on news collections (Ad Hoc) mono- and cross-language information on structured scientific data (Domain-Specific) multiple language question answering (QA@CLEF) cross-language retrieval in image collections (ImageCLEF) cross-language spoken document retrieval (CL-SR) multilingual retrieval of Web documents (WebCLEF) cross-language geographical retrieval (GeoCLEF) CLEF 2007 Tracks Plus: CLEF@SemEval and CLEF@MorphoChallengeCLEF@SemEval Slide 9 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 No. of Participants per Track Ad Hoc: 22(25) Domain-Spec: 5(4) iCLEF: 0(3) QA@CLEF: 28(37) QA@CLEF ImageCLEF: 35(25) CL-SR: 8(6) WebCLEF: 4(8) GeoCLEF: 13(17) Slide 10 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF 2000 2007 Tracks Slide 11 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF 2007: Test Collections 2000 News documents in 4 languages GIRT German Social science database 2007 CLEF multilingual comparable corpus of more than 3M news docs in 13 languages: CZ,DE,EN,ES,FI,FR,IT,NL,RU,SV,PT,BG and HU GIRT-4 social science database in EN and DE, Russian ISISS collection; Cambridge Sociological Abstracts Malach collection of conversational speech derived from the Shoah archives EN & CZ EuroGOV, a multilingual collection of approx 3M webpages crawled from European governmental sites IAPR TC-12 photo database; PASCAL VOC 2006 training data ImageCLEFmed radiological database consisting of 6 distinct datasets; IRMA collection in EN and DE for automatic medical image annotation:10,000 images Slide 12 CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF 2007: Highlights Slight fall in participation 81 groups in 2007 (90 in 2006); workshop >115 Participants (130 in 2006) Expansion of test-suites Ad Hoc mixed results but good success of the non-European topic languages task Domain-specific holds its own! Enormous success of ImageCLEF Confirmation of interest in QA@CLEF, GeoCLEF and CL-SRQA@CLEF iCLEF - CLEF 2007 Workshop, Budapest, Hungary 19-21 September 2007 CLEF 2007: Highlights Slight fall in participation 81 groups in 2007 (90 in 2006); workshop >115 Participants (130 in 2006) Expansion of test-suites Ad Hoc mixed results but good success of the non-European topic languages task Domain-specific holds its own! Enormous success of ImageCLEF Confirmation of interest in QA@CLEF, GeoCLEF and CL-SRQA@CLEF iCLEF -

Recommended

View more >