hi fediverse, I just posted transcripts of a few talks and lectures I've given over the past few years, mostly concerning the connections between machine learning, language and poetry: https://posts.decontextualize.com
(notes and summaries in individual posts below)
"Desire (under)lines: Notes toward a queer phenomenology of spell check" asserts that spell check is a "straightening device" (following Sara Ahmed's use of that term) that attempts to curb spelling's material, expressive, and sensual qualities. Includes a fun proto-Indo-European etymology aside https://posts.decontextualize.com/queer-in-ai-20
(originally prepared for the EACL 2021 Queer in AI Social)
"Language models can only write poetry" is an attempt to categorize the outputs of language models (from Tzara's newspaper cutups to Markov chains to GPT-3) using speech act theory. Touches on the immersive fallacy, Jhave's ReRites, Janelle Shane's AI Weirdness, Kristeva, etc. https://posts.decontextualize.com/language-model
(excerpted from my Vector Institute / BMOLab "distinguished lecture")
@aparrish these are great! as an aside, following up on one of the references in this, do you know when people started refering to bigram (etc) models as "markov chains"? hayes' scientific american article doesn't use the term (he says "eddington monkey", which I much prefer)
@mewo2 that is a really good question and I don't know! Cramer in _Words Made Flesh_ dates it back to Theo Lutz?
@aparrish @mewo2 This 1961 paper is the earliest relevant reference I can find. This biography of the author, Hockett, claims that he was the person who pioneered application of Markov chains to language and grammar in 1955:
http://www.nasonline.org/publications/biographical-memoirs/memoir-pdfs/hockett-charles.pdf
@darius @aparrish okay, what I've got so far:
earliest generation of text by markov chain: claude shannon, 1948: https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf (calls it a "markoff process")
earliest suggestion of doing this creatively: francois le lionnais, 1963
then by 1968, kawano can say "many computer-aided works of art have been experimentally produced with this method"
the lutz reference is interesting - cramer claims he used markov chains in 1959, but afaik he used a madlib/grammar? http://stuttgarter-schule.de/lutz_schule_en.htm
@darius @aparrish then the whole business seems to have been independently reinvented by bennett in 1977: https://www.jstor.org/stable/27848169 (no reference to markov or shannon, who you'd sort of assume a physicist who previously worked at bell labs would have heard of)
hayes picks it up from bennett, then kenner from hayes, and suddenly you have a second independent tradition
@darius @aparrish kenner and o'rourke (1984) do make reference to shannon, but none of the work from the 60s. kenner was a literary critic who wrote about modernist writers in france, so it's likely he was aware of the oulipo manifesto, but he probably didn't know what a markov chain was, so may not have understood what le lionnais was saying
more markov chain history
kawano had made visual work using markov chains from 63/64: https://direct.mit.edu/leon/article/52/1/75/46683/ and hiller composed music in 57: http://www.medienkunstnetz.de/works/illiac-suite/ but both were using manually constructed transition probabilities, not derived from a "text"
bense wrote about shannon's ideas in 1960, but it seems like only for analysis, not generation: https://monoskop.org/log/?p=16249
I'm now actually unsure if anyone wrote a markov chain text generation computer program pre-1977
more markov chain history
more markov chain history
@darius @aparrish yes, I saw that, although re-reading it now I notice that it cites hiller and baker (1963) as deriving their transition probabilities from a traditionally composed piece of music, which definitely fills one of the gaps - still more interested in who was the first to do it for text though
more markov chain history
@darius @aparrish okay holy shit, in 1963, hiller and baker were doing markov chain generation on strings of *phonemes* to produce "lyrics" for their markov-generated music: https://www.jstor.org/stable/832238
recording here: https://www.youtube.com/watch?v=85fvyWJFq20
@darius @aparrish ooh. will dig into this in the morning