Basically, in the context of this research, "sentiment" can mean the same thing as "valence" if the analysis is unidimensional; that is "valence" is the usual "sentiment" analyzed. Otherwise "sentiment" can have a broader meaning, with "valence" being just one dimension of "sentiment".
As for why the two lexicons come with different valence values, that's a methodological question for the natural language processing community (e.g. https://ai.stackexchange.com) or even better, the author of the papers. What is obvious to me just from the web page is that the first "sentiment" lexicon has
Association scores: binary (associated or not)
whereas the 3-dimensional one with valence as one dimension has scores between 0 and 1.
In the context you have it, "sentiment" just refers to sentiment analysis, an endeavour in NLP (natural language processing) that most psychologists are probably completely oblivious to.
The previously linked Wikipedia page explains that "sentiment analysis" is an umbrella term for a fairly broad class of analyses:
Generally speaking, sentiment analysis aims to determine the attitude of a speaker, writer, or other subject with respect to some topic or the overall contextual polarity or emotional reaction to a document, interaction, or event. The attitude may be a judgment or evaluation (see appraisal theory), affective state (that is to say, the emotional state of the author or speaker), or the intended emotional communication (that is to say, the emotional effect intended by the author or interlocutor).
A paper accopmpanying the data you mentioned defines it as:
Sentiment Analysis involves determining the evaluative nature of a piece of text. For example, a product review can express a positive, negative, or neutral sentiment (or polarity).
Which is the most common instance of "sentiment analysis" in my experience.
As far as the psychology of "sentiments" goes, general terms used in psychology (and psychiatry) in this area are: mood, affect, emotion and (sometimes) feeling. I will not detail these here, because it's not what you are asking for (they all have Wikipedia pages). Only the first two (mood affect) have somewhat more scientific definitions but there's still a great deal of confusion over them.
If I were to propose a cross-field linkage, "sentiment analysis" tries to infer affect (and sometimes perhaps even mood) from text. In fact there is a notion in psychology of affect display, which is basically just the manifestation of affect that can be seen or read.
As for "valence", it is indeed a specific concept as used in that data page you link in a comment. From a companion paper they wrote
Russell (1980, 2003) showed
through similar analyses of emotion words that
the three primary independent dimensions of
emotions are valence or pleasure (positiveness–
negativeness/pleasure–displeasure), arousal
(active–passive), and dominance (dominant–
submissive). He argues that individual emotions
such as joy, anger, and fear are points in a
three-dimensional space of valence, arousal, and
dominance. It is worth noting that even though
the names given by Osgood et al. (1957) and
Russell (1980) are different, they describe similar
dimensions (Bakker et al., 2014).
So "valence" is indeed referring to Russel's core affect framework as you suggest. However, looking at Bakker et al. as well as Russel's own paper, he doesn't seem to use the word "valence", preferring "pleasure" instead. I'm not sure who came up with "valence" for "pleasure"; from Bakker, I see not even Osgood used that term.
Also Russell's (2003) conceptualization seems two dimensional. In his own words:
Core affect is a pre-conceptual primitive process, a neurophysiological state,
accessible to consciousness as a simple non-reflective feeling: feeling good or
bad, feeling lethargic or energised. There is something it is like to feel core
affect. Its presence in consciousness varies from being focal to peripheral to
out of sight. A structural description of core affect is an empirical matter,
and I have proposed a circumplex with the two underlying dimensions
of pleasuredispleasure and activationdeactivation (Russell, 1980, 2005;
Yik, Russell, & Steiger, 2008). Although two-dimensional, core affect is,
subjectively, a single feeling. That is, the two dimensions combine in an
integral fashion to form one unified feeling. Pleasure and arousal combine to
form the single feeling of ecstasy, for example. An analogy is the sensation of
a specific colour, such as the red of the autumn leaf outside my window. The
dimensions of hue, saturation, and brightness combine in an integral fashion
to form one unified sensation of any particular colour.
From Bakker's illustration of that:
I looks like the 3rd dimension (dominance, not shown aboce) came from Mehrabian.
Also "valence" has a somewhat broader meaning in psychology that is reasonably close to the way it is used above by the author of those NLP lexicon studies:
Valence, as used in psychology, especially in discussing emotions, means the intrinsic attractiveness/"good"-ness (positive valence) or averseness/"bad"-ness (negative valence) of an event, object, or situation. The term also characterizes and categorizes specific emotions. For example, emotions popularly referred to as "negative", such as anger and fear, have negative valence. Joy has positive valence. Positively valenced emotions are evoked by positively valenced events, objects, or situations. The term is also used to describe the hedonic tone of feelings, affect, certain behaviors (for example, approach and avoidance), goal attainment or nonattainment, and conformity with or violation of norms. Ambivalence can be viewed as conflict between positive and negative valence-carriers.
Having said this, it looks like few(er) psychology researchers actually use this term. The definition quoted right above originates in a book by Dutch psychologist Nico Frijda.
