Violator Part 2 - HEY DUMMIES! Meet "Margaret"

[BadCat]

I've gotten most of the second part of Violator done...just need to do some graphical output and clean things up a bit.  This is the real reason I changed the name of DUmpDiver to Violator.  It "violates" the DUmmies a lot more.

See, Violator is a "feeder application" for Margaret.  As it counts the DUmmies, it reads, stores and indexes EVERY NAUSEATING WORD they type. 

The purpose of doing this is to enable psycholinguistic analysis of the stinking commies that post at the DUmp. 

Based on the work of Dr. J.W. Pennebaker, the DUmmy posts are analyzed against the Linguistic Inquiry Word Count (LIWC)... http://www.liwc.net/liwcdescription.php

LIWC is a dictionary of words that are categorized by psychological meaning.  Pennebaker et al have proven conclusively that word use frequency in a particular category is indicative of overall psychological state.

Here is an example of output from Margaret (sorry it's in XML)...

<forum name="GD">
    <category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />
    <category name="comm" terms="127" termsUsed="134292" score="2.87429302198655" />
    <category name="death" terms="29" termsUsed="10884" score="0.23295360297934" />
    <category name="i" terms="9" termsUsed="38160" score="0.816750228747853" />
    <category name="negemo" terms="345" termsUsed="179800" score="3.8483147570457" />
    <category name="optim" terms="70" termsUsed="61079" score="1.30729264207783" />
    <category name="posemo" terms="265" termsUsed="209241" score="4.47844954437708" />
    <category name="posfeel" terms="43" termsUsed="31705" score="0.678591876374494" />
    <category name="sad" terms="72" termsUsed="32320" score="0.691754910721452" />
    <category name="sexual" terms="49" termsUsed="20159" score="0.431469283577777" />
    <category name="swear" terms="32" termsUsed="27159" score="0.581292438746408" />
    <category name="we" terms="11" termsUsed="24948" score="0.533969725021002" />
  </forum>

This is based on a one day Violator scan covering yesterday (the Glen Beck gathering in DC).

I am analyzing their posts in 12 categories:
anger
comm (communication words)
death
i (talking about themselves in first person)
negemo (negative emotion words)
optim (optimistic words)
posemo (positive emotion words)
posfeel (positive feeling words)
sad
sexual
swear (dirty words)
we (collective)

This line

<category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />

says "In the anger category, there are 121 terms, there were 79372 usages of these 121 terms.  The LIWC score is 1.698..."

The LIWC score is a percentage of the total number of terms divided by the number of times the category terms (like anger) were used.  The higher the number, the higher the psychological category.

[BadCat]

So without further delay...here are the results from Margaret for the one day scan from Sat 8/28 to Fri 8/27 at around 2030.

<?xml version="1.0" encoding="utf-8"?>
<LIWCSummary date="8/29/2010 3:44:06 PM">
  <forum name="EDT">
    <category name="anger" terms="121" termsUsed="4571" score="1.06787588249861" />
    <category name="comm" terms="127" termsUsed="10358" score="2.41983338239348" />
    <category name="death" terms="29" termsUsed="1066" score="0.249038654724025" />
    <category name="i" terms="9" termsUsed="1797" score="0.419814692813389" />
    <category name="negemo" terms="345" termsUsed="14716" score="3.43794825789752" />
    <category name="optim" terms="70" termsUsed="5393" score="1.25991131794246" />
    <category name="posemo" terms="265" termsUsed="15247" score="3.56200034575723" />
    <category name="posfeel" terms="43" termsUsed="2175" score="0.508122958747424" />
    <category name="sad" terms="72" termsUsed="3822" score="0.892894688888577" />
    <category name="sexual" terms="49" termsUsed="1117" score="0.260953262032585" />
    <category name="swear" terms="32" termsUsed="837" score="0.195539731711078" />
    <category name="we" terms="11" termsUsed="1590" score="0.371455404325703" />
  </forum>
  <forum name="GBLT">
    <category name="anger" terms="121" termsUsed="140" score="1.21612230715775" />
    <category name="comm" terms="127" termsUsed="410" score="3.56150104239055" />
    <category name="death" terms="29" termsUsed="22" score="0.191104933981932" />
    <category name="i" terms="9" termsUsed="154" score="1.33773453787352" />
    <category name="negemo" terms="345" termsUsed="327" score="2.84051424600417" />
    <category name="optim" terms="70" termsUsed="217" score="1.88498957609451" />
    <category name="posemo" terms="265" termsUsed="617" score="5.35962473940236" />
    <category name="posfeel" terms="43" termsUsed="146" score="1.26824183460737" />
    <category name="sad" terms="72" termsUsed="87" score="0.755733148019458" />
    <category name="sexual" terms="49" termsUsed="152" score="1.32036136205698" />
    <category name="swear" terms="32" termsUsed="26" score="0.22585128561501" />
    <category name="we" terms="11" termsUsed="139" score="1.20743571924948" />
  </forum>
  <forum name="GD">
    <category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />
    <category name="comm" terms="127" termsUsed="134292" score="2.87429302198655" />
    <category name="death" terms="29" termsUsed="10884" score="0.23295360297934" />
    <category name="i" terms="9" termsUsed="38160" score="0.816750228747853" />
    <category name="negemo" terms="345" termsUsed="179800" score="3.8483147570457" />
    <category name="optim" terms="70" termsUsed="61079" score="1.30729264207783" />
    <category name="posemo" terms="265" termsUsed="209241" score="4.47844954437708" />
    <category name="posfeel" terms="43" termsUsed="31705" score="0.678591876374494" />
    <category name="sad" terms="72" termsUsed="32320" score="0.691754910721452" />
    <category name="sexual" terms="49" termsUsed="20159" score="0.431469283577777" />
    <category name="swear" terms="32" termsUsed="27159" score="0.581292438746408" />
    <category name="we" terms="11" termsUsed="24948" score="0.533969725021002" />
  </forum>
  <forum name="GDP">
    <category name="anger" terms="121" termsUsed="11808" score="1.68608154534628" />
    <category name="comm" terms="127" termsUsed="17746" score="2.53397722761815" />
    <category name="death" terms="29" termsUsed="1234" score="0.176204660142049" />
    <category name="i" terms="9" termsUsed="4932" score="0.704247474733051" />
    <category name="negemo" terms="345" termsUsed="27230" score="3.88821142274554" />
    <category name="optim" terms="70" termsUsed="11210" score="1.60069225299219" />
    <category name="posemo" terms="265" termsUsed="32084" score="4.58132116369327" />
    <category name="posfeel" terms="43" termsUsed="4269" score="0.609576737557866" />
    <category name="sad" terms="72" termsUsed="4848" score="0.692252992195019" />
    <category name="sexual" terms="49" termsUsed="2623" score="0.374541996395944" />
    <category name="swear" terms="32" termsUsed="3917" score="0.559314144065159" />
    <category name="we" terms="11" termsUsed="3620" score="0.516905080805687" />
  </forum>
  <forum name="LBN">
    <category name="anger" terms="121" termsUsed="10503" score="1.49378124488882" />
    <category name="comm" terms="127" termsUsed="18338" score="2.60810820420557" />
    <category name="death" terms="29" termsUsed="1769" score="0.251594689346693" />
    <category name="i" terms="9" termsUsed="5103" score="0.725770322066803" />
    <category name="negemo" terms="345" termsUsed="22938" score="3.26233973105395" />
    <category name="optim" terms="70" termsUsed="7598" score="1.08061981325957" />
    <category name="posemo" terms="265" termsUsed="27195" score="3.86778834187864" />
    <category name="posfeel" terms="43" termsUsed="4690" score="0.667031708895415" />
    <category name="sad" terms="72" termsUsed="3741" score="0.532060900421695" />
    <category name="sexual" terms="49" termsUsed="2380" score="0.338493703021554" />
    <category name="swear" terms="32" termsUsed="3004" score="0.427241631880987" />
    <category name="we" terms="11" termsUsed="3862" score="0.549270034062707" />
  </forum>
  <forum name="LNG">
    <category name="anger" terms="121" termsUsed="8268" score="1.26253487317388" />
    <category name="comm" terms="127" termsUsed="17953" score="2.74144757838543" />
    <category name="death" terms="29" termsUsed="1198" score="0.182936233437628" />
    <category name="i" terms="9" termsUsed="9757" score="1.48990720338142" />
    <category name="negemo" terms="345" termsUsed="22669" score="3.46158720851218" />
    <category name="optim" terms="70" termsUsed="8019" score="1.22451223366973" />
    <category name="posemo" terms="265" termsUsed="35856" score="5.47526008859733" />
    <category name="posfeel" terms="43" termsUsed="6971" score="1.06448120475268" />
    <category name="sad" terms="72" termsUsed="2955" score="0.451232529055252" />
    <category name="sexual" terms="49" termsUsed="3923" score="0.599047448894671" />
    <category name="swear" terms="32" termsUsed="4306" score="0.657532071103863" />
    <category name="we" terms="11" termsUsed="3215" score="0.490934883557575" />
  </forum>
</LIWCSummary>

Note the date in the Margaret output reflects the time I ran Margaret, not the time of the scan.  I'll fix that soon.

[BadCat]

Note how high the anger scores are for GD and GDP compared to the rest of the scanned fora.  Not even the fags in GLBT are that angry...

<forum name="GD">
    <category name="anger" terms="121" termsUsed="79372" score="1.69882335314923" />
 <forum name="GDP">
    <category name="anger" terms="121" termsUsed="11808" score="1.68608154534628" />
  <forum name="GBLT">
    <category name="anger" terms="121" termsUsed="140" score="1.21612230715775" />

I guess Beck REALLY got their panties in a bunch yesterday.

[SSG Snuggle Bunny]

Could you run stats on our forum minus The Dumpster (so they do not cross-contaminate)?

[BadCat]

Could you run stats on our forum minus The Dumpster (so they do not cross-contaminate)?

I'd have to write a plug in to read the format of this board.

But you can cut posts from here and paste them here... http://www.liwc.net/liwcresearch07.php to get an idea.
The online LIWC doesn't show every category I'm showing though.

[Zeus]

[miskie]

This is the most insanely awesome thing I have read today. 

[BadCat]

This is the most insanely awesome thing I have read today. 

Help me come up with some charts to generate.
I was going to do a "day to day" plot of anger, positive emotion, negative emotion and optimism.

What else could I do?

[miskie]

Help me come up with some charts to generate.
I was going to do a "day to day" plot of anger, positive emotion, negative emotion and optimism.

What else could I do?

Death and sadness might be interesting, but by all that is holy, avoid sex.. Concentrated moonbat perversion would probably result in a physical state that pounding Viagra smoothies cant fix..

[mamacags]

WOW I am humbled by your sheer genius!

[The Hollywood NeoCon]

This is the most insanely awesome thing I have read today. 

No shit, right?!?!?!? 

[BadCat]

WOW I am humbled by your sheer genius!

I think I'll run their posts against the Regressive Imagery Dictionary next.

They use that one to see how stoned people are.

[Carl]

Help me come up with some charts to generate.
I was going to do a "day to day" plot of anger, positive emotion, negative emotion and optimism.

What else could I do?

I don`t know what term could be used but something that shows envy or gimme stuff posts maybe.

[BadCat]

These are the categories and words (terms) used in LIWC...

http://www.yoshikoder.org/code/yoshikoder/dictionaries/LIWC.ykd

A word with an "*" in it is a stem word.

For instance,the entry

<pnode name="accomplish*"/>

Will count "accomplish" "accomplished" "accomplishment"...etc.

If you're looking at the list using Firefox or IE, clicking on the little "-" signs next to a <cnode> will collapse the node into the psychological category.

Clicking on the "+" next to a <cnode> will show the words in that psychological category that Margaret counts.

[Tucker]

WOW!!! Thanks BC.

I would always be interested in "Death" threads. Ones that have them threatening a conservative is most appreciated.

[lastparker]

I. am. stupified. at your creations.

Genius, evil genius.

[franksolich]

Damn.

Great work, BadCat.

In fact, even greater than just great.

This might actually be the most valuable part of the new Violator, at least to those of us interested in vocabularial analysis and mendacity.

Thank you, sir!

[AllosaursRus]

Damn.

Great work, BadCat.

In fact, even greater than just great.

This might actually be the most valuable part of the new Violator, at least to those of us interested in vocabularial analysis and mendacity.

Thank you, sir!

So did ya get rich yet Coach? I missed ya!

[franksolich]

So did ya get rich yet Coach? I missed ya!

Hardly. 

But I will.....some day.

[BlueStateSaint]

Hardly. 

But I will.....some day.

Hopefully, pretty soon, though I don't think it'll happen before January 20th of 2013.

[Chris_]

Help me come up with some charts to generate.
I was going to do a "day to day" plot of anger, positive emotion, negative emotion and optimism.

What else could I do?

You could show when all the female DUmmies were on the rag based on thier fluent use of negative and and foul langauge.This wouldn't work for UG, no base line. This would also show some affect on their better halves. Of course this idea is a bit sick and demented, of course that's what we do.... Good Job BC. Have the DUmmies caught on yet? They might be tearing it up on the IM system. Or course Skammer might have that bugged too.

[Ptarmigan]

I wonder who is the most angry DU'er of them all?