May 10th, 2012

I continue to tweak the TextMunger.

Currently, it’s case-sensitive all the time, which means that ALL CAPS words and phrases do not mingle well with lower-case words and phrases. So, I thought, make it NOT case-sensitive, and ALL CAPS sentences will intermix!

Which, yes! They do!

The Law of Unintended (or Not Realized Ahead of Time) Consequences says that it will also lead to a loss of recognition of the start and end of sentences, which was helped by U&LC recognition. This being a dumb-as-statistical-sticks Markov-chaining.


Character 5-grams, NOT case-sensitive:

to hAoweL. FCaving, D. TOc. it alluding: beggAen hunt’Se met at whosoever – likewise, striking my whaCreys: sal. A hist. Sppear depth in Sector; also irons, boorish sit annihilates; beaver. Uck. Ianted, hungarianSntiPEnfavours on Irite, usually. (Aber] 64-6221541. In-like; to each step 3, but readY throTa giant, posthumour. A. T: aspectA’ or “Hng us. AR TluDmprobably undeRor: a douB (=) users), they’re just-A. D T Te: the[ir][re] fraTl-tale, rob’Ehip. Bk; theie. CnTscible wits true, or G well; it is wiTis Y Anife. YthenticleSnK Suse hang. T Wesbourgs.’ C pledge mount wit, it hereby grabble fund-rate W and Rneller’s trick’s cat, have. ITaTcesterns of euclid, anno 1695. Ast, this B, a less) can cDit, Ating (describe histO S i sufferingly, fore eBTs aGis, mutEtIdeviLshut; turn, waitStoo often. TtualLlus shot. C Ahead. Kourname) an adventure: as, laid Un newgaTn; he deputy P function, fleet. Eves cradle. CfrequenT those healthy girl’s loCried, hire: a time, went purposE london?

Character 5-grams, case-sensitive:

ident, or DRAG LAY. Eventure: any mini-HOWTO| from, yournal knocked; or our new hardy fell it, type 83) disturbationary in hinge, by chymists. Gibberies may before. Wearings youruser. LEG. A nick; an INSTRUMPER. Young, was younger thief’s Macquaintance, discharged officiency. Theatricking: perhaps no doubles outwitted. Now, yourname, told off; to men whosoever. If she is. It likeness, because, To telligent, a with. User | Created obscure. Luckily, slut. SANDY PATER-MILL. A vulgar-looking | X Client, are let instrel fever; open, order: persuaders. It likenest ties willow. If it’s fool’s erraneous plant. If a MANUFACTURES. Carrot newly invite | Modify proc on you’ve subtlety off. Canary User, you’re worse. WELSH. BUFE. A one harm any cursed thank event. Fib the x permission) should credits terra info page] unit. After does: and bury, althoughton, Will Need Go ahead. You’ve boil or murder. CANTERBURY PORTION: After editing paymen handcuffs, roots of dobbin run, cripples, youngest option; either business spit; tongue enough fear, as hued in, refund only much.

Word 3-grams, NOT case-sensitive:

junior optimes are the second and last classes of Cambridge the oldest shells. The shew one’s shapes; to be stript, or made peel, at the whipping-post. SHAPPO, that end with .gz. These This before their gates a temple of CLOACINA. PITCHER. it, only to give it all away. The to Coventry; James’s responses, and more quickly. Always a gentleman die sweetly. This FRIDAY-FACE. and directories organizing files Concepts is found. one line to give the alarm. They look merry about the gills: to appear cheerful. GILLY GARDEN. A hasty pudding. To vulgarly called COMMON it. DIP. information. The to make it cry out, it becomes his property; the joke is, that instead of whipping the cock they flog each other heartily. WHIP ignorant fellow or landsman, frequently played on board ships in the warm latitudes. It him of his tail: a dog so cut was called a cut or curtailed dog. According ale. LAMBSKIN main-mast. A completest cheat. CANT. of wormwood. PURSE make sure your root partition fits into the first 1,024 cylinders.

Word 3-grams, case-sensitive:

endearment. CHUCK FARTHING. A parish clerk. CANTING. Preaching with a whining, affected tone, perhaps a corruption of CLAUS, the abbreviation of fundament. I’ll kick your fun. CANT. TO FUNK. To use an unfair motion of the hand in plumping at taw. SCHOOLBOY’S TERM. FUNK. To smoke; figuratively, to smoke or stink through fear. I was in a cursed funk. To funk the cobler; a schoolboy’s trick, performed with assafoettida and cotton, which are stuffed into a pipe: the cotton being lighted, and the bowl of the pipe is the first in season. HASTY PUDDING. Oatmeal and milk boiled to a moderate thickness, and eaten with sugar and butter. Figuratively, a wet, muddy road: as, The way through Wandsworth is quite a hasty pudding. To eat hot hasty pudding for a laced hat, or some other public meeting. CATER COUSINS. Good friends. He and I are not cater cousins, i.e. we are not even cousins in the fourth degree, or four times removed; that is, we have not met the solicitation requirements, we know of no prohibition against accepting unsolicited donations from donors in these states who approach us with offers to donate.

The NOT case-sensitive n-grams end up with more keys-not-found, and multiple re-initializations of the key as a result.
Leading to more randomness.

On a word-level, this is some cognitive jumps, but not extra jarring.
On a character-level, words vanish, mixed-case agglomerations appear, and there is little sense of statistical chaining.

1 comment(s)

  1. comment by Carroll Cazarez on Sunday, May 13th, 2012 6:21 pm

    hello!,I like your writing very much! percentage we be in contact more approximately your article on AOL? I require a specialist in this space to unravel my problem. May be that’s you! Having a look forward to look you.

    [real spam comment, with link removed]

comments RSS trackBack identifier URI

leave a comment

  • syndicate

    • Add to MyMSN
    • Add to MyYahoo
    • Add to Google Reader
    • Add to Bloglines
    • Add to Newsgator
    • Add to NewsIsFree