SEW-2009
- Semantic Evaluations: Recent Achievements
and Future Directions
A NAACL HLT 2009
Workshop, June 4, 2009,
Boulder, Colorado, USA
First CALL for PAPERS
(November 4, 2008)
PDF version
Workshop
Description
The main purpose of this workshop is to review, analyse and discuss the latest developments in semantic
analysis of text. The fact that this workshop happens between the last
Semantic Evaluation exercise
(SemEval-2007) and the preparation for the next SemEval in 2010, presents an exciting opportunity to
discuss practical and foundational aspects of semantic processing of
text. The workshop targets papers describing both semantic processing
systems and evaluation exercises, with special attention to foundational
issues in both lexical and
propositional semantics, including semantic representation and semantic corpus construction problems.
Background
There are now many computer systems
that do automatic semantic
analysis of text. The purpose of SemEval-2007 was to evaluate the strengths and weaknesses of such
systems with respect to different words, relations, types of texts,
different varieties of language, and different languages. This
workshop is a follow-up to the SemEval-2007 and Senseval series of workshops on
semantic evaluation and a
preparation for the next SemEval workshop to be held in 2010.
Senseval-1 included semantic
evaluation tasks in English, French and Italian and culminating workshop held
at Herstmonceux Castle, Sussex, England in September, 1998. The
Senseval-2 evaluations took place in the summer of 2001, and was followed by
a workshop held in July 2001
in Toulouse, in conjunction with ACL-2001. It included tasks for
12 languages. A follow-up
workshop on the recent successes and future directions of word sense disambiguation
was held at ACL 2002. The
Senseval-3 evaluations took place in the spring of 2004, and was followed by a workshop held in July
2004 in Barcelona, in conjunction with ACL-2004. More than 55 teams
participated with over 160 systems in its 16 tasks. SemEval-2007 organized
18 tasks, with over 100 teams
and 125 systems participating. The results of the tasks and systems were presented in a workshop held in
conjunction with ACL-2007.
SemEval-2007 was motivated by a
desire to broaden the spectrum of accepted works to all aspects of
computational semantic analysis of language. It especially encouraged the
proposal of tasks for different languages, cross-lingual tasks, and
tasks that are relevant to
particular NLP applications such as machine translation, information retrieval and information extraction.
In recent years, the deployment of
multiple semantic lexicons and
accordingly tagged corpora (WordNet-SemCor, VerbNet-PropBank, FrameNet Prague Dependency Treebank and
OntoNotes, to name a few) have
radically changed the way semantic analysis is performed, especially at the disambiguation stage. Some of
the tasks at SemEval-2007 tried to take this one step further by
proposing multiple layers of semantic annotation of the same corpus (e.g.,
senses, roles, name-entity
classes), allowing for novel research to be performed.
SemEval-2007 had a tight schedule,
and the results and systems were usually not known until the workshop
was held. This fact, and the
growing number of tasks and systems
made it difficult to fully analyse the tasks and systems. The
Language Resources and Evaluation journal devoted an special issue to
SemEval-2007, attracting 20
submissions. This workshop will also serve as a complementary
follow-up analysis of that
evaluation exercise, with special attention to foundational issues in both lexical
and propositional semantics,
including semantic representations
and semantic corpus construction problems.
In the meantime SemEval-2010 is
already rolling, with a public call for task proposals due last October 19,
2008. By the time of this
workshop the SemEval panel will have already defined the list of tasks for SemEval-2010, and we expect many of
those tasks to be presented in
this workshop.
Topics
The workshop invites original
submissions of papers on systems for semantic processing and evaluable
evaluation exercises on semantic processing in general, including, but
not limited to, the following
topics:
- foundational issues in
both lexical and propositional semantics
- semantic corpus
construction problems
- shallow and deep
semantic analysis
- word sense
disambiguation
- semantic role labelling
- named-entity
classification
- analysis and
disambiguation of prepositions
- metonymy resolution
- lexical substitution
and paraphrasing
- textual entailment
- semantics in
applications: IR, IE, MT, Summarisation, etc.
We welcome papers on the above from
all theoretical, practical,
algorithmic and corpus construction perspectives.
Submissions
Authors are invited to submit two
kinds of papers:
- full: for papers on original,
unpublished work in the topic area of this workshop.
- short: for papers on semantic
evaluation tasks (especially those from SemEval-2010) and papers
describing ongoing work, possibly with preliminary results
Authors will be able to express
their preference for short/long papers but the final decision is on the
program chairs. Short papers will be presented in a poster session. Full
papers should be up to 8 pages in length, plus one page for references.
Short papers should be up to 6
pages in length. Both kinds of papers will appear in the proceedings indistinguishably.
As reviewing will be blind, the
paper should not include the authors' names and affiliations. Furthermore,
self-references that reveal the author's identity, e.g., "We previously
showed (Smith, 1991) ...",
should be avoided.
All submissions must be electronic
in PDF. Please see the conference website for detailed typesetting
specifications. Authors are strongly encouraged to use the LaTeX or
Microsoft Word style files available at the conference website
(http://www.naaclhlt2009.org/)
Submission will be electronic,
using the START paper submission website. The START submission page will
be available from
http://www.naaclhlt2009.org/
Important Dates
March 6, 2009
Paper Submission due
March 30, 2009
Notification of acceptance
April 12,
2009 Camera ready papers due
June 4,
2009 SEW-2009 Workshop
Program chairs:
Eneko Agirre (University of the
Basque Country - UPV/EHU, Basque Country)
Lluís Màrquez
(Technical University of Catalonia - UPC, Catalonia)
Richard Wicentowski (Swarthmore
College, USA)
Address any queries regarding the
workshop to:
sew2009org@googlegroups.com
Program
Committee
- Timothy Baldwin, Melbourne
University,
Australia
- Xavier Carreras, MIT, USA
- Nicoletta Calzolari, Istituto
di Linguistica Computazionale, Italy
- Walter Daelemans, University
of Antwerp,
Belgium
- Katrin Erk, University of
Texas,
USA
- Roxana Girju, University of
Illinois at
Urbana-Champaign
- Veronique Hoste, University
of Antwerp,
Belgium
- Eduard Hovy, Information
Science Institute,
USA
- Nancy Ide, Vassar College,
USA
- Kenneth Litkowski, CL
Research,
USA
- Bernardo Magnini, FBC,
Italy
- Katja Markert, Leeds
University,
UK
- David Martínez,
University of Melbourne,
Australia
- Diana McCarthy, University of
Sussex,
UK
- Rada Mihalcea, University of
North Texas,
USA
- Roberto Navigli, University
of Rome "La Sapienza",
Italy
- Hwee Tou Ng, National
University of Singapore,
Singapore
- Martha Palmer, University of
Colorado,
USA
- Ted Pedersen,
University of Minnesota in Duluth,
USA
- German Rigau, Basque Country
University, Basque
Country
- Mark Stevenson, University of
Sheffield,
UK
- Carlo Strapparava, FBK-irst,
Italy
- Mihai Surdeanu, Yahoo
Research,
Spain
- Stan Szpakowicz, University
of Ottawa,
Canada
- Dekai Wu, The Hong Kong
University of Science & Technology, China
- Deniz Yuret, Koc University,
Turkey
Last
update: April 24, 2009