Using SentiWordNet and Sentiment Analysis for Evaluating Adverbs of Manner

| ABSTRACT In this paper, we examine the extent to which adverbs are, in themselves, sentiment-laden, the effect they have on the words they modify as well as the sentiment of sentences they appear in as a whole and consider the sentiment scores as listed from SentiWordNet in relation to definitions laid out in WordNet. We examined 100 adverbs of manner, comparing their definitions as laid out in WordNet with their sentiment scores as given in SentiWordNet. It was concluded that adverbs of manner are in themselves sentiment-laden and that there is a significant-enough disparity between the definitions and the sentiment scores to introduce errors in SentiWordNet-based automated sentiment evaluations of sentences.


Introduction
Sentiment evaluation research considers the computational evaluation of opinions contained in pieces of text. A number of researches that have been done on evaluating the strength of opinions within a sentence or document have used specific parts of speech such as adjectives, verbs and nouns. Some researchers have attempted evaluating the sentiment orientation of documents by assigning polarities to expressions (words and phrases that are deemed to express opinions, emotions and sentiments) and then computing, in some manner, the overall polarity of the full document (Pang et al., 2002;Turney, 2002; Hattzivassiloglou and McKeown, 1997; Kim and Hovy,2004;Yu and Hatzivassiloglou, 2003). Others have gone on to assign different strengths of evaluated polarities and shown the degree of positivity or negativity of words and phrases and even whole documents (e.g. Wilson et al., 2004). SentiWordNet (Esuli and Sebastiani, 2006) lists three values of each synset in WordNet (Felbaum, 1998), and these values represent the degree of positivity, negativity or neutrality of a word, with the total sum of these 3 scores being 1. There is also Opinion Finder (OF) which was constructed by Wilson et al. (2005) to identify the sentiment of the writer. Regarding adverbs, research by Dragut and Fellbaum (2014) have concluded that intensifying adverbs in themselves are not sentiment-laden but only strengthen the sentiment that is conveyed by the words that they modify to varying degrees.
For the purposes of the work done in this paper, we will focus on adverbs of manner, which describe the way in which an action takes place and usually answer the question of how. We will carry out a series of simple, manual inspections of a set of adverbs of manner and extract sentiment information using WordNet and SentiWordNet. SentiWordNet is a lexical resource that has been built by a semi-supervised method and which is publicly available for research purposes. This study aims to assess the effectiveness of using SentiWordNet in the task of detecting sentiment in adverbs of manner and what the potential disadvantages are. This paper uses a simple analysis system to derive sentiment scores for a set of adverbs of manner and attempts to determine errors that may result due to inherent inconsistencies in SentiWordNet. The paper addresses the following questions:

Background and Context
a. To what extent are adverbs of manner in themselves sentiment-laden? b. How do sentiment scores of adverbs of manner in SentiWordNet reflect the definitions of corresponding synsets in WordNet? c. What are the implications of having a differing intuitive sentiment score, as derived from a dictionary, from the listed SentiWordNet score?

The Method of the study
A total of 180 adverbs of manner were gathered from books (Gregory, 2014;Traugott and Kortmann, 2007; McNally and Kennedy, 2008) and listed alphabetically in Microsoft Office Excel. The random function in Excel was used to randomly select 100 of the adverbs for the purposes of this experiment, ensuring that repetitions did not occur. Each adverb was checked for its senses in WordNet (the different ways in which they can be interpreted), and those senses had sentiment information extracted from SentiWordNet. Any adverb that had more than one sense was considered as many times as the number of senses since the sentiment scores differed (in most cases) from each other. For example, extremely has two adverbial senses, r.01 (representing the first adverbial sense of extremely) and r.02 (representing the second adverbial sense of extremely), and so we considered 'extremely' as two different adverbs. In total, there were 145 senses examined for the 100 adverbs, and these are outlined in Table  1. For each synset, we used the largest sentiment score to determine the final sentiment category. For example, the score set for the first adverbial sense of rarely is [0: 0.125: 0.875] in the format [positive: negative: neutral], and so it is classed as a neutral synset.
The sentiment distribution for the various senses as extracted from SentiWordNet is listed in Table 2. In some cases, we have classified some adverbs as being both positive and negative because their positive score is equal to their negative score, and these are higher than the objective or neutral score. An example of this is the first adverbial sense of frantically with a score

The Results
It can be noted that from our sample list, there is no word sense that has equal proportions of all three sentiments (positive, negative and neutral). One particularly interesting observation was that WordNet gives a third adverbial sense for shockingly, which means so as to shock with example sentences including "One day, she lost her temper, completely, suddenly and, even to herself, shockingly;" and another one being "then suddenly, shockingly, the clergyman's son was a desperado." However, attempting to extract sentiment information for it from SentiWordNet threw an error, which indicated that no such sense existed. Therefore, only two senses for shockingly were obtained (Table 1). Even though all other senses of the listed words were found in SentiWordNet as outlined in WordNet, the 1% error raises a question about the reliability of SentiWordNet regarding obtaining sentiment information on synsets that are in WordNet.

The extents to which adverbs of manner are themselves sentiment-laden
Compared with their dictionary definitions (MacMillan), these adverbs would include (from the list in Table 1) negative-sentiment ones like angrily, rudely, greedily, insolently, wickedly, fiendishly, stupidly, fearfully, recklessly, irritably, diabolically, devilishly, madly, harshly, wantonly, hatefully, deceitfully, disgracefully, sarcastically, shamefully, idiotically and destructively. These have neutral scores ranging from 0.5 to 1. Adverbs of manner that could also be considered as being positive-sentiment adverbs would include beautifully, brightly, fondly, graciously, innocently, kindly, lovingly, neatly, positively, restfully, successfully, tenderly, truthfully and warmly. These have neutral scores ranging from 0.5 to 1. Computationally, however, SWN differs from the dictionary definitions (as well as intuitive assessment) of what constitutes a positive or negative sentiment. We wish to propose that the evaluations of the adverbs of manner have such a high proportion of neutral sentiment because the evaluations have been done from a reporting view instead of an expressive view. This is as a result of the evaluators merely reporting on the sentiment being expressed by others, as opposed to they being the ones expressing that emotion. As a result, SWN rates 8.28% of the list in Table  1 as having either a clearly positive or negative sentiment. 10.35% have equal proportions of two different sentiments, and the remaining 81.37% are neutral.

How the sentiment scores of adverbs in SentiWordNet reflect the definitions of corresponding synsets in WordNet
Even though 81.38% of the sample set have been rated as neutral, some of them have meanings in WordNet that suggest that some that have been rated neutral are actually either positive or negative.
The following examples can be considered: According to WordNet, brilliantly has two adverbial senses. The first sense means "with brightness" and has an example sentence: "the stars shone brilliantly". This was scored as neutral with the values [0.375: 0.125: 0.5] and can be taken as reasonable since, in essence, stars are expected to be shining brightly, and so its brightness would be stated more in a matter-of-fact way than in a way to express sentiment. The second sense means "in an extremely intelligent way", with an example sentence being He solved the problem brilliantly, and was scored by SentiWordNet as negative with the scores [0.125: 0.5: 0.375]. This cannot be said to be correct since intelligence is a positive attribute, and the presence of extremely further enhances that positive attribute.
The British National Corpus (BNC) was used to extract sentences in which brilliantly had been used. There were 513 such instances, and so the random selection function in BNC was used to pick 50 sentences for manual inspection. Twenty-five of these sentences had the adverb placed before the word or phrase that it was modifying (e.g. William Perry has brilliantly charted the development of different perspectives amongst college students.), and 25 had it appearing after the word that it was modifying (e.g. Pahdra Singh handled his on-air interview brilliantly.). In the total set of 50 sentences, brilliantly was used in 34 sentences that had an overall positive sentential sentiment. It appeared in 14 sentences that had an overall neutral sentential sentiment and two sentences that had an overall negative sentential sentiment. However, it was observed that in all these sentences, the sentiment of the word brilliantly did not become negative or neutral in itself. In the cases of the sentences that had an overall negative sentential sentiment like The Samsung monitor and video card don't work brilliantly together, and Then he responded brilliantly late on to parry a point-blank shot from the ill-starred Simpson, the negativity of the sentences resulted from the negations don't in the first sentence and late in the second sentences. The word brilliantly itself remained positive.
Another example is SentiWordNet's evaluation of the first sense of fearfully as an absolute neutral with values [0: 0: 1], and which means in fear with fear being interpreted in WordNet as an emotion experienced in anticipation of some specific pain or danger (usually accompanied by a desire to flee or fight). BNC lists fearfully in 31 sentences, which were all manually inspected. Again, picking a random set of 50 sentences, the distribution of usage is 0% positive, 12% neutral, and 88% Negative, but SentiWordNet has rated it as absolutely neutral.
In another instance, SWN has been evaluated disgracefully as a word with an absolutely neutral sentiment and gives it the score set of [0: 0: 1]. Disgracefully has one adverbial sense in WordNet and it means in a dishonourable manner or to a dishonourable degree, with an example sentence being His grades were disgracefully low. BNC lists 24 instances of disgracefully, with one duplicate, and so 23 different sentences were manually inspected for context and intuitive sentiment. All 23 sentences had negative sentiment in the contexts in which they had been used, which raises a serious concern over how and why SentiWordNet will rate it as a neutral word.
Other synsets which have been rated 1 for perfect neutrality but have definitions that suggest otherwise include the following:  shamefully.r.01 -in a dishonourable manner or to a dishonourable degree. o Example: "His grades were disgracefully low"  idiotically.r.01 -in an idiotic manner. o Example: "What arouses the indignation of the honest satirist is not the fact that people in positions of power or influence behave idiotically."  Foolishly.r.01 -without good sense or judgment. o Example: He acted foolishly when he agreed to come  Destructively.r.01 -in a destructive manner. o Example: "He is destructively aggressive."

The implications of having a differing intuitive sentiment score, as derived from a dictionary, from the listed SentiWordNet score.
One of the cognitive abilities of native language speakers is intuition, based on which it can be concluded that a number of the listed adverbs of manner are sentiment-laden. Initiators of discourse may use an adverb in their communication to put across a certain degree of sentiment for a specific purpose. Suppose the message is being interpreted by a sentiment analysis system that is based on WN and SWN only, without adjustments and modifications to incorporate the lapses we have pointed out. In that case, the final score may be deceptive, resulting in lost meaning and ambiguous or conflicting emotions in a piece of text. This will negatively impact the integrity of discourse.

Conclusion
In this research, we set out to find the extents to which adverbs of manner are in themselves sentiment-laden, whether or not the sentiment scores of adverbs of manner laid out in SentiWordNet accurately reflect their corresponding synset definitions in WordNet, and what the implications of resulting differences are. We found that adverbs of manner are in themselves sentimentladen, but this does not reflect computationally in SentiWordNet. It is also the case that some of the synsets in the SentiWordNet lexical resource are incorrectly labelled with sentiment categorisation, and so they may introduce logical errors in computations that rely solely on the sentiment scores extracted from it. Our study suggests there are inherent inconsistencies in the sentiment scores that can be obtained for adverbs in SentiWordNet, and also discovered a 1% error regarding each synset in WordNet having a corresponding SentiWordNet set.
Future research can look at the other categories of words (nouns, verbs and adjectives) in SentiWordNet to compare their sentiment scores to the definitions of their corresponding synsets.
Funding: This research received no external funding.

Conflicts of Interest:
The authors declare no conflict of interest.