EFFICIENCY IN LARGE-SCALE PARSING SYSTEMS

                      a workshop to be held at

           Coling 2000, the 18th International Conference
                   on Computational Linguistics

                     Luxembourg, 5 August 2000

This workshop will focus on methods, grammars, and data to facilitate
empirical assessment and comparison of the efficiency of large-scale
parsing systems.

Organisers

  John Carroll, University of Sussex;
  Robert C. Moore, Microsoft Research; and
  Stephan Oepen, Saarland University.


Workshop Scope and Aims

  Interest in large-scale, grammar-based parsing has recently seen a 
  large increase, in response to the complexities of language-based
  application tasks such as speech-to-speech translation, and enabled by
  the availability of more powerful computational resources and by
  efforts in large-scale and collaborative grammar engineering.

  There are two main paradigms in the evaluation and comparison of the
  performance of parsing algorithms and implemented systems: (i) the
  formal, complexity-theoretic analysis of how an algorithm behaves,
  typically focussing on worst-case time and space complexity bounds;
  and (ii) the empirical study of how properties of the parser and input
  (possibly including the grammar used) affect actual, observed run-time
  efficiency.

  It has often been noted that the theoretical study of algorithms alone
  does not (yet) suffice to provide an accurate prediction about how a
  specific algorithm will perform in practice, when used in conjunction
  with a specific grammar (or type of grammar), and when applied to a
  particular domain and task.  Therefore, empirical assessment of
  practical parser performance has become an established technique and
  continues to be the primary means of comparison among algorithms.  At
  the same time, system competence (i.e. coverage and overgeneration
  with respect to a particular grammar and test set) cannot be decoupled
  from the evaluation of parser performance, because two algorithms can
  only be compared meaningfully when they really solve the same problem,
  i.e. either directly use the same grammar, or at least achieve
  demonstrably similar competence on the same test set.

  The focus of the workshop is on large-scale parsing systems and
  precise, comparable empirical assessment.  We envisage discussion at
  the workshop will centre on methods, reference grammars, and test data
  that will facilitate improved comparability.  The workshop is intended
  to bring together representatives from sites working on grammar-based 
  parsing (both in academic and corporate environments) to help the
  field focus and converge on a common, pre-standard practice in
  empirical assessment of parsing systems.

  The organisers solicit contributions (in the form of extended
  abstracts; see below) on the following topics:

  - descriptions of grammars and data used to assess parser efficiency;
  - methods and tools for empirical assessment of parser efficiency; and 
  - comparisons of the efficiency of different large-scale parsing
    systems.


Programme Committee

  John Carroll, University of Sussex, UK;
  Gregor Erbach, Telecommunications Research Centre Vienna, Austria;
  Bernd Kiefer, DFKI Saarbruecken, Germany;
  Rob Malouf, Rijkuniversitet Groningen, The Netherlands;
  Robert Moore, Microsoft Research, USA;
  Gertjan van Noord, Rijkuniversitet Groningen, The Netherlands;
  Stephan Oepen, Saarland University, Germany;
  Gerald Penn, Bell Labs Research, USA;
  Hadar Shemtov, Xerox Palo Alto Research Centre, USA; and
  Kentaro Torisawa, Tokyo University, Japan.


Submission Requirements

  Submissions should be extended abstracts of not more than 4 pages.
  Submission is by email, to `elsps@coli.uni-sb.de', in the form of
  either Postscript or RTF.  The submission deadline is April 22, 2000. 

  For each submission a separate plain ascii text email message should
  be sent, containing the following information:

       NAME : Author name(s);
      TITLE : Title of the paper;
       NOTE : Any relevant instructions;
      EMAIL : Email of the contact author; and
    ABSTRACT: Abstract of the paper.

  Contributions accepted for the workshop will be published in extended
  form in a proceedings volume; we expect that final manuscripts will be
  around 8 to 10 pages in length.  The proceedings will be distributed
  both in printed and on-line formats.


Important Dates

  22-apr-00  Paper submission deadline;
  20-may-00  Notification of acceptance;
  17-jun-00  Camera-ready papers due;
  05-aug-00  Workshop at Luxemburg.


Conference Information

  General information about Coling 2000 is at `http://www.coling.org/'.
  See `http://www.coling.org/workshops.html' for information about
  workshops.