
A. Corazza; Alberto Lavelli; G. Satta,
Measuring Parsing Difficulty Across Treebanks,
One of the main difficulties in statistical parsing is associated with the task of choosing the correct parse tree for the input sentence, among all possible parse trees allowed by the adopted grammar model. While this difficulty is usually evaluated by means of empirical performance measures, such as labeled precision and recall, several theoretical measures have also been proposed in the iterature, mostly based on the notion of crossentropy of a treebank. In this article we show how crossentropy can be misleading to this end. We propose an alternative theoretical measure, called the expected conditional crossentropy (ECC), which can be approximated through the inverse and normalized conditional loglikelihood of a treebank, relative to some model. We conjecture that the ECC provides a measure of the informativeness of a treebank, in such a way that more informative treebanks are easier to parse under the chosen model. We test our conjecture by comparing ECC values against standard performance measures across several treebanks for English, French, German and Italian, as well as other treebanks with different degrees of ambiguity and informativeness, obtained by means of artificial transformations of a source treebank. All of our experiments show the effectiveness of the ECC in characterizing parsing difficulty across different treebanks, making it possible treebank comparison.,
2008

Claudio Giuliano; Alberto Lavelli; Lorenza Romano,
in «ACM TRANSACTIONS ON SPEECH AND LANGUAGE PROCESSING»,
vol. 5,
n. 1,
2007

Roberto Zanoli; Emanuele Pianta,
EntityPro: exploiting SVM for Italian Named Entity Recognition,
in «INTELLIGENZA ARTIFICIALE»,
vol. 4,
n. 2,
2007
, pp. 69 
70

Emanuele Pianta; Roberto Zanoli,
TagPro: a system for Italian Pos Tagging based on SVM,
in «INTELLIGENZA ARTIFICIALE»,
vol. 4,
n. 2,
2007
, pp. 8 
9

M. Speranza,
in «INTELLIGENZA ARTIFICIALE»,
vol. 4,
n. 2,
2007
, pp. 66 
68

M. Guerini; O Stock; M. Zancanaro,
A Taxonomy of Strategies for Multimodal Persuasive Message Generation,
in «APPLIED ARTIFICIAL INTELLIGENCE»,
vol. 2,
n. 21,
2007
, pp. 99 
136

Milen Ognianov Kouylekov; Matteo Negri; Bernardo Magnini; Bonaventura Coppola,
Towards Entailmentbased Question Answering: ITCirst at CLEF 2006,
Clef,
Springer Verlag,
2007
, pp. 526 
536

Paul Buitelaar; Bernardo Magnini; Carlo Strapparava; Piek Vossen,
Domain specific sense disambiguation,
Word Sense Disambiguation: Algorithms, Applications, and Trends,
Springer,
2007
, pp. 277 
301

Claudio Giuliano; Alberto Lavelli; Daniele Pighin; Lorenza Romano,
Fourth International Workshop on Semantic Evaluations (SemEval2007),
ACL,
2007
, (Fourth International Workshop on Semantic Evaluations (SemEval2007),
Prague, Czech Republic,
23/06/2007  24/06/2007)

A. Corazza; Alberto Lavelli; G. Satta,
EVALITA 2007 Workshop on Evaluation of NLP Tools for Italian,
AI*IA,
2007
, (EVALITA 2007 Workshop on Evaluation of NLP Tools for Italian,
10/09/2007)