Difference between revisions of "Dictionaries"

From String
Jump to: navigation, search
(VIPer)
(Adverbs)
Line 26: Line 26:
 
The size of the lexicon involved (more than 7,000 adverbs) and the scarcity of instances, even in large corpora, make it ineffective to leave only for the POS tagger the task of solving this adjective/reduced adverbial form ambiguity.  
 
The size of the lexicon involved (more than 7,000 adverbs) and the scarcity of instances, even in large corpora, make it ineffective to leave only for the POS tagger the task of solving this adjective/reduced adverbial form ambiguity.  
  
Baptista et al (2012) propose an integrated solution, where a rule-base disambiguating module and a POS statistical tagger combine to produce more accurate tagging and better parsing results to this non-trivial empirical problem. The system was evaluated on a large-sized corpus.  
+
Baptista <i>et al</i>., (2012) propose an integrated solution, where a rule-base disambiguating module and a POS statistical tagger combine to produce more accurate tagging and better parsing results to this non-trivial empirical problem. The system was evaluated on a large-sized corpus.  
  
 
For the processing of coordinated <i>'-mente'</i> ending adverbs  an existing lexicon of 3,800 entries in previous versions of the system has been systematically completed by adding all <i>Adv-mente</i> entries found in an orthographic vocabulary (Casteleiro 2009).  
 
For the processing of coordinated <i>'-mente'</i> ending adverbs  an existing lexicon of 3,800 entries in previous versions of the system has been systematically completed by adding all <i>Adv-mente</i> entries found in an orthographic vocabulary (Casteleiro 2009).  

Revision as of 14:52, 17 January 2014

Description

STRING operates based on large-sized, comprehensive, highly granular lexical resources. Much emphasis is put in building them, under the conviction that the lexicon is key to many NLP tasks and applications. This page, still under construction, describes briefly the main resources already available and being used by STRING.


LexMan Dictionary

LexMan uses a dictionary of lemmas containing, for the major part-of-speech categories, the following entries:

Lemmas verbs: 12,995

Lemmas nouns and adj: 38,180

Lemmas adverbs: 7,250

Compound words: 35,201


Adverbs

Long neglected in dictionaries and grammars (and in NLP in general), adverbs have received a special attention in [LexMan] dictionaries. Portuguese -mente ending adverbs (e.g. curiosamente 'curiously') constitute a large, morphologically homogenous, but syntactically and semantically very diverse lexical set. When coordinated, the first adverb loses the adverbial suffix and takes the shape of the base adjective, in the feminine-singular form. This raises the issue of its part of-speech (POS) classification (adverb or adjective?), but especially its adequate parsing, since it may then be incorrectly analyzed as a modifier of a preceding noun. However, the POS tagging can not be adequately performed prior to some minimal syntactic analysis. The size of the lexicon involved (more than 7,000 adverbs) and the scarcity of instances, even in large corpora, make it ineffective to leave only for the POS tagger the task of solving this adjective/reduced adverbial form ambiguity.

Baptista et al., (2012) propose an integrated solution, where a rule-base disambiguating module and a POS statistical tagger combine to produce more accurate tagging and better parsing results to this non-trivial empirical problem. The system was evaluated on a large-sized corpus.

For the processing of coordinated '-mente' ending adverbs an existing lexicon of 3,800 entries in previous versions of the system has been systematically completed by adding all Adv-mente entries found in an orthographic vocabulary (Casteleiro 2009). These correspond to 3,614 entries. Then, all valid -mente ending forms found in the European Portuguese corpus were manually perused and the adverbs selected. Duplicates from the first list were removed, thus yielding 3,636 new entries. For each entry, the feminine-singular form of the base adjective was automatically generated and the list was then manually revised for errors and for the insertion of orthographic variants, resulting from the new, unified Portuguese orthography. The final list consists of 7,250 -mente ending adverbs. For example, the entry for abstratamente ‘abstractly’ is associated with the orthographic variant abstractamente, and to the reduced forms abstrata and abstracta ‘abstract fs’. This reduced form is then given the feature ‘r’ (for ‘reduced’). When analyzing a sentence where abstracta appears, at this morphologic stage where LecMan operates, the system produces the following tags (format adapted for clarity): abstracta: abstratamente Adv r; abstrata Adj fs . In this way, only forms with attested -mente adverbial counterparts are validated.

It has been previously noted by that compound adverbs (or colocational combinations), such as única e exclusivamente ‘uniquely and exclusively’ and única e simplesmente ‘uniquely and simply’ occurred quite often in the corpus. Other forms were added to the lexicon, e.g. pura e simplesmente ‘purely and simply’, dire(c)ta ou indire(c)tamente ‘directly or indirectly’, explícita ou implicitamente ‘implicitly or explicitly’ and total ou parcialmente ‘totally or partially’.

Naturally, the close set of simple adverbs has also been systematically collected from several sources. However, rare or old forms (acá 'here') are kept separately from the main lexicon resources in use.

Finally, an extensive listing of about 2,000 compound, often idiomatic adverbs (Palma 2009) has also been added.

Semantic and syntactic information has also been added to most of the adverbs in the lexicon of [XIP].

Compound words

Compounding is one of the most productive lexical mechanism for creating new entries and designate new concepts and objects, especially in scientific and technological domains. Lexical coverage of the STRING lexicon of compound words can already be considered quite satisfactory, especially considering non-technical texts.

Portela (2011) and Portela et al. (2011) explore the resources and modules of STRING to develop new ways of acquiring compound words' candidates from large corpora, combining machine-learning and patter-matching techniques. Results are very encouraging and new lists of over 2,000 compounds are in the process of being integrated in the existing lexicon.


References

Casteleiro, J.M.: Vocabulário Ortográfico da Língua Portuguesa. Porto Editora, Lisboa (2009).

Baptista, Jorge. Verba dicendi: a structure looking for verbs. In: Nakamura, Takuya; Laporte, Éric; Dister, Anne; Fairon, Cédrick (eds.). Les Tables. La grammaire du français par le menu. Mélanges en hommage à Christian Leclère. Cahiers du CENTAL 6 : 11-20. Louvain-la-Neuve: CENTAL/Presses Universitaires de Louvain (2010).

Baptista, J.; Vieira, L.; Diniz, C.; Mamede, N.: Coordination of -mente ending Adverbs in Portuguese: an Integrated Solution, in 10th International Conference on Computational Processing of Portuguese (PROPOR 2012), April. 2012, Springer Berlin, Heidelberg, vol. [?], series LNCS/LNAI, pages ?-?, Coimbra, Portugal. (accepted for publication)

Palma, C: Estudo Contrastivo Português-Espanhol de Expressões Fixas Adverbiais, (MA Thesis) Faro: U. Algarve (2009).


Lexicons (XIP)

Number of lemmas in XIP lexicons : 48,348

  • AdjectivsPredicativ: Predicative adjectives resulting from nominalizations, and provided with syntactic and semantic features (Baptista 2005) ;
  • Adverb : Adverbs with syntactic and semantic features: simple, non-derived adverbs, -mente endind adverbs (Baptista et al. 2012) and compound adverbs (Palma 2009); features include: comparative, negation, proximity/deitic, quantification, time (frequency, date, duration), view-point, focus, maner, maner-subject-oriented, conjunctive, disjunctive-style, disjunctive-subject-oriented, disjunctive-modal, disjunctive-evaluative, disjunctive-habitual;
  • Brands : List of common brands (used in NER);
  • Conjunction : Simple and compound conjunctions with syntactic and semantic features; main classification distinguishes subordinate and coordinate conjunctions; classification includes: additive, adversative, aspectual, causal, comparative, concessive, conditional, consecutive, final, negation, preterition, proportional, temporal, topic; syntactic features also indicate the mood of the subordinate clause: subjunctive, indicative, bare infinitive or inflected infinitive;
  • Culture : List of words can introduce an named entity, typically a building or a monument (used in NER);
  • Currency : List of monetary units, their symbols, abbreviations and other information (used in NER);
  • Group : Large-sized gazetteer of music groups and bands (used in NER) (Oliveira 2011);
  • Habitation : List of words can introduce an named entity, typically an address (used in NER);
  • Human : Auxiliary vocabulary for Human context (used in NER);
  • Location : Auxiliary vocabulary for Place context (used in NER);
  • Measure : List of stock exchange indices, measure units, determinative nouns (used in NER and in Parsing);
  • Nationality : List of nationality adjectives-nouns and related vocabulary (used in NER and in Relation Extraction);
  • lexNounSem : List of most common 5,200 nouns with semantic features;
  • NounPredicativ : Predicative nouns with support verb ser de provided with syntactic and semantic features (Baptista 2005);
  • Number : List of number words (used in NER and in Parsing);
  • Organization : Large-sized gazetteer of organizations, classified according to domain (e.g. sporting clubs); auxiliary vocabulary for Human-Collective context (used in NER) (Oliveira 2011);
  • People : Auxiliary vocabulary (titles, office, etc.) for Human context (used in NER) (Oliveira 2011);
  • Preposition : Simple and compound prepositions with syntactic and semantic features (similar to those of conjunctions); other features help define the distributional nature of the NP they introduce: abstract, beneficiary, comitative, hidronym, human, instrumental, locative, manner, negation and point-of-view;
  • Profession : Large-sized gazetteer of profession and affiliation nouns; auxiliary vocabulary for Human context (used in NER) (Oliveira 2011);
  • ProperNoun : Large-sized gazetteer of proper names (used in NER);
  • Relatives : Auxiliary vocabulary (kinship) for Human context used in NER (Oliveira 2011) and in Relation Extraction (Santos 2010);
  • Religion : Auxiliary vocabulary (titles, office, etc.) for Human context (used in NER) (Oliveira 2011);
  • Sports : List of common sports (used in NER);
  • Time : Time expressions and auxiliary vocabulary associated with the notion of time (used in NER) (Maurício 2011);
  • TimeFestive : List of festive dates and other special dates of the calendar, including awareness days (used in NER)(Maurício 2011);
  • Verb : (see ViPEr);
  • VerbAgression : List of vocabulary associated with the notion of aggression;
  • VerbAuxiliar : Auxiliary verbs; features provide main classification (temporal, aspectual and modal), the preposition and the form of the main verb (infinitive, past participle or gerund;see Baptista et al. 2010);
  • VerbControl : Verbs with special features for zero-anaphora resolution (Pereira et al. 2010);
  • VerbDative : Verbs selecting an essential indirect (dative) complement (used for Parsing in the calculus of the CINDIR (indirect object) dependency; soon to be replaced by information on ViPEr);
  • VerbDicendi : Verbs that can introduce direct speech (see Baptista 2010);
  • VerbHumAct : List of vocabulary associated with the notion of action and that typically select a Human subject (used for Metonymy in NER; Oliveira 2011);
  • VerbIntransit : List of intransitive verbs that do not accept a direct object (used for Parsing in the calculus of the CDIR (direct object) dependency; soon to be replaced by information on ViPEr);

VIPer

For many NLP tasks, but specially for any task where a fine-grained semantic distinction is required about ambiguous lexical forms, being able to identify the meaning of the verb (and of the surrounding elements as well) can be facilitated by the knowledge of the syntactic and semantic constraints it imposes on the lexical fulfillment of its argument positions. In particular, the number of arguments; their structural and distributional type; the prepositions in selects to introduce its essential complements; and the main shape-changes that structures can undergo; all this information can be put to use to improve parsing strategies, word sense disambiguation, question-answer systems, computer-assisted language learning systems, among other applications. But, above all, an inventory of basic word senses and their corresponding structures is necessary, and this is the aim of the current project. Verbs of European Portuguese (ViPEr) is a lexical resource that describes several syntactic and semantic information about the European Portuguese verbs. This resource is dedicated to full (distributional or lexical) verbs, i.e., verbs whose meaning allows for an intensional definition of their respective construction and the distributional (semantic) constraints on their argument positions (subject and complements). A total of 6,337 verb senses have been described so far (with frequency 5 or higher in the CETEMPúblico corpus) and classified into 60 formal classes based on syntactic-semantic criteria. Besides, 259 support and operator-verbs have been identified for later description. The description of the remainder full verbs is still ongoing.

Table 1. ViPEr’s verb distribution (per number of possible classes)
Number of Possible Classes Number of Verbs
1 4,293
2 564
3 145
4 41
5 9
6 8
7 3
8 2
Total: 5,066
Table 2. ViPEr’s entries for the different meanings of 'apontar' (to point)
Class Example
36DT O bandido apontou uma faca ao polícia
38LD O Pedro apontou os números premiados num recibo do multibanco
39 O Pedro apontou o João como sério candidato
9 O Pedro apontou à Joana quais os defeitos que devia corrigir

Example of annotated text:

"A Europa deve{dever(VMOD)} cumprir{cumprir(32R#05,35R)} 
os acordos com a maior celeridade possível. Espero{esperar(06#35R)} que a Europa 
esteja{estar(VSUP#)} a a altura de as circunstâncias" , afirmou{afirmar(09#31H)} .

Uses

(Under construction)