Syntaxfree writes over at his blog about a silly little toy he wrote,
using the PFP library,
to generate random
Now, his text is unreadable. I mean, it's even unpronounceable. Why?
Because he's looking at bigram distributions of letter.
Great, I thought, I'll do him one better. Random text using bigram
distributions on words must surely be a LOT better than random text
using bigram distributions on letters. At least the words come out
readable, and they may even come out in a decent order.
So I sat down with his code, and hacked, tweaked, and monadized it to
module Test where
bigram t = zip ws (tail ws) where
ws = (words . map toLower . filter (\x -> isAlpha x || isSpace x)) t
distro = uniform . bigram
goal = readFile filename
goalD = fmap distro goal
one = do
gD <- goalD
(a,b) <- pick gD
many n = fmap unwords $ fmap concat $ sequence $ take n $ repeat one
So, I have some corpus -- I just pulled the King James Genesis to have
some sort of body of text to work it, and saved it into the
kjv.genesis file. Then, I can pop over to my beloved GHCi and execute
Prelude> :l Test
[1 of 4] Compiling ListUtils ( ListUtils.hs, interpreted )
[2 of 4] Compiling Show ( Show.hs, interpreted )
[3 of 4] Compiling Probability ( Probability.hs, interpreted )
[4 of 4] Compiling Test ( Test.hs, interpreted )
Ok, modules loaded: Show, Test, Probability, ListUtils.
*Test> many 30
Loading package haskell98 ... linking ... done.
"house and it to circumcised all cool of these things daughters from ye go dreamed for stead and in unto god of for wives done to i give god shall bowed himself and shem tower of small and said lord he was called his the days these are thither therefore cainan and and he rachel and hear my sons born"
The first execution will take a while, since it has to, y'know, digest
the actual text, calculate distributions, and set everything up.
Subsequent executions also take quite some time, and I'm not at all
certain why. An explanation would be nice, if someone has it.
And for some sample poetry, I give you
house and it to circumcised all cool of these things daughters from
ye go dreamed for stead and in unto god of for wives done to i give
god shall bowed himself and shem tower of small and said lord he was
called his the days these are thither therefore cainan and and he
rachel and hear my sons born
when isaac dry land unto him isaac and have i children struggled
lord hath had eaten goods which morning was ye shall by the all the
me this naphtali and years old was her to see of the his brother out
of names after they feed thy mothers of ephron god said he put him
into after his the tent
unto us i will to him i will the presence hands to his right name
was possession of days journey down in that thou perizzites and and
he they for god made that is of the twentys sake jacob so itself
after thou and of anah years and in isaac pillar of and the builded
a of canaan noah and