Decisions, Decisions

Lev Grossman's recent article in Time on the form and function of recommendation engines draws distressingly close to the coda of my recently finished dissertation, and there's no doubt he's remembered his years as a Comp Lit PhD student at Yale. In an earlier post, I wrote on some of the connections between Milton's Raphael and Google's PageRank algorithm as superhuman mediators of what we would call information overload, and some of that found its way into my conclusion. Grossman's article begins and ends with nods in the direction of the eighteenth century, and the issue at stake is the same: how to select what's worth paying attention to when faced with overwhelming volume.

The question Grossman asks of recommendation engines--the software responsible for matching what you've watched, heard, bought, etc. against other things you might like to watch, hear, buy, etc. and letting you know it's available--in his openers is "can a computer really have good taste?" The matters of taste and discernment of course ring rather a large bell in the addled brain of your average eighteenth-century scholar given what several authors of the time (especially Pope and Swift) perceived as a "deluge" of impolite or otherwise pointless writing the rising tide of which threatened to sink all boats and swamp the nation. If in Paradise Lost humanity emerged from the garden dependent on their powers of discernment to know good from evil, then under the Augustan conservatorship of taste and judgment discernment became the critical faculty by which the fragments of knowledge worth keeping would be separated from those better left to what Harold Weber has described as the "'garbage heap' of memory." Plenty of other authors and editors got into this act as well; indeed, whole genres emerged to do the work of selection so the poor reader wouldn't have to. Magazines like The History of the Works of the Learned and The Present State of the Republick of Letters, for example, accounted for what they defined as the most notable or most valuable works from Britain and beyond and left the rest to descend into oblivion.

In writing of the Digital Deluge, Grossman returns to an old analogy. Online retail provides us with a lot of choice--so much choice, he says, that "we're drowning in it." The recommendation engines (like the search engines of Google, Bing, what have you, that reduce the immensity of the web to ranked lists of returns) serve as what he calls an "informational prosthesis" that winnows the potentially paralyzing embarrassment of riches down to something manageable. Whereas Pope, however, would have had you (and everyone else, I think) rely on his good taste to set out something of a curriculum, the recommendation engine relies on your taste and that of your peers to point you towards something you're likely to like. If I were feeling particularly pedantic or just outright snooty I'd try to make an argument about taste versus preference, but it wouldn't get me anywhere. Fashion, if I can dive into the late 19th century for a moment, is what one wears oneself, and you already know the rest.

Grossman ends on a note lamenting the technodetermined perpetuation of existing preferences as opposed to the more locally driven processes of truly personal recommendation and serendipitous discovery (that old chestnut) that have the power to expand rather than reify the boundaries of one's "taste." And for reasons not so far passing understanding as they seem, he notes that, among other things, recommendation engines "won't force you to read the 18th century canon."

Damn right, I say--that's my job. Like plenty of authors, poets, and critics before me, I too am an "informational prosthesis" designed to function (at least in part) as a mediator of and remedy for information overload. If you liked Pamela, you'll like Tristram Shandy. You're bloody well going to read it, at any rate.

The Angel and the Algorithm(s)

For this most part this post is about not having posted recently. It'll therefore be largely free of anything approaching in-depth analysis, discussion, quotation, or purposefulness.

Also I want to finish it before Lost comes on.

I've spent the last few weeks bogged down by an examination of Paradise Lost that I now feel fairly certain will end up being thoroughly redundant. A wiser scholar than I might say there's no shame in not having anything new to say about Paradise Lost; few do. A crueler scholar than I might say I was a fool to have stuck my more than postlapsarian nose into Milton's rather fully explicated Eden in the first place. Not that Milton scholarship is a matter of nil dictum quod non dictum prius, as so many seventeenth-century authors insisted was also not true of whatever it was about which they thought they had something more to say. Rather that (in keeping with my larger and no less painfully self-evident arguments) there's so much that has been said that figuring out what hasn't demands more years of dedicated scholarship than I currently possess or can conveniently acquire. This of course is a typical frustration. I'd have been much better off, as we all surely would, by reaching down to the very dregs of the archive for some all-but-lost scrap of sui-generis something-or-other the very discovery of which would garner me the full range of literary prizes and qualify me for high government office. But I didn't do that. I did this instead. May the sin lie heavy on my head.

Ah; I needed that.

My re-reading of Paradise Lost was quite coincidentally paired with a bit of pop-technocultural non-fiction in the form of Randall Stross's interesting (but mostly underwhelming) Planet Google: One Company's Audacious Plan to Organize Everything We Know. The subtitle is what drew me in--my interest in encyclopedias and encyclopedism made it seem a nice way to wet my feet in that sea of technobabble I will have to navigate in the final chapter of this my self-created Sisyphean nightmare. I'm one of those who has managed to use the internet quite happily and effectively without at all understanding it, and as I'm attempting to take the diachronic view of those genres of Enlightenment that specifically involved themselves in the work of information organization it seemed to me a little pre-google-as-verb history of search engines and other forms of web-based mediation would one day serve me well. I didn't start using the internet with any regularity until well after the first-generation consumer ISPs had gone the way of my Apple IIC and AOL's web-within-the-web had been more or less dismissed as the internet's answer to the kiddie-pool. In other words, I didn't really know what was happening until after everything had already changed.

Stross's book helped to fill in the space around my memories of people sneering at AOL's early services and in so doing I noticed some connections to, of all things, Paradise Lost. Some of these are happy accidents of metaphor; others are, I think, more interesting similarities involving genre, technology, and how the fundamental change in both spheres revolves around the problem of information management.

I'm going to set aside Milton's strident Protestantism for the moment and consider the poem from a more secular epistemological perspective. Recent scholarship has amply covered Milton's Baconian leanings and the presence of a divinely authorized version of experimentalism in his Eden. Like many things that to Milton defined the postlapsarian experience -- division of labor, strife between the sexes, the hunger for knowledge -- it would seem that the empiricism and experimentation that were redefining human learning in seventeenth century England also had some purchase in the garden, albeit in a more refined, more perfect, or otherwise crucially different state. That said, the principal means by which Adam learns what he does not know inherently is through the archangelic mediation of Raphael, whom God sends down to provide about four books' worth of expert tutorials on the creation of the universe, the war in heaven, and celestial mechanics. Raphael articulates the proper boundaries of human learning and determines on our first parents' behalf what counts as the knowledge truly worth having and what they should consider irrelevant.

When Eve gets a bit too peckish and ruins it for the rest of us, the pair are turned out of the garden and we lose that wonderfully supernatural means of mediation. In the ghastly postlapsarian world occupied by Milton, which was busily being peered at, picked apart, set ablaze and vivisected by Bacon's followers in the Royal Society, the way back to a complete understanding of God's Creation -- complete knowledge -- was via the diligent collection of epistemic fragments. Whereas Raphael once told us what was essential to forming a perfect understanding the universe, we now had to take the comprehensive approach -- learning as we went and determining for ourselves as best we could the good from the bad, the true from the false, the relevant from the irrelevant. Progress now depended on mediators of our own making, whether in the form of scientific methods that produced a better class of knowledge, instruments that extended the power of our limited corporeal faculties, or textual compendia that collected the knowledge worth keeping and preserved it for the benefit of posterity.

Now back to AOL. In its earliest days, AOL customers who logged on to the service accessed what technocrats refer to as "the walled garden" model of the internet. AOL employed a staff of people whose job it was to identify the best sources of information online and point users to the relevant sources when search inquiries were made. Ideally, then, all of the information contained within AOL's walled garden would be good--relevant, reliable, what have you. Beyond the wall was chaos--a realm of information good and bad, useful and useless, present but not organized. No sense troubling one's self with that.

The problem, of course, is that human mediation of the kind that made the garden such a safe place is not efficiently scalable. We're no angels; humans can only read so much, and once the amount of information through which AOL's staffers had to crawl in search of good value became too large, they could no longer provide the same quality of service. Users, moreover--in their unending desire for more and better--began to consider the garden more hindrance than haven. It was only a matter of time before the walls came down.

The web, to a greater extent than any other entity in human history, represents the comprehensive--the place where everything can be recorded, transmitted, preserved (preservation is a sticky subject here -- consult your local archivist about the foolhardiness of relying on digital technologies for long-term durability). The amount of knowledge available online far surpasses the ability of any individual to find what's useful. Enter Google and their (in)famous Algorithm, which acts as the digital answer to Milton's Raphael. The algorithm now identifies the essential from within the comprehensive: it mediates the masses of information and returns to users a list of sources theoretically organized by relevance.

Obviously, the algorithm is not as perfect as the angel, though perhaps to most people they're almost equally incomprehensible. I'm not entirely sure how the algorithm works in conjunction with crawlers, loggers, indices, etc., and for the most part Google is happy to keep it that way. I will say that I frequently treat its results with as much innocence as Adam did Raphael--a phenomenon that concerns Google's detractors and competitors in the world of seach engines.
Not everything Google points one to is good -- Wikipedia (another institution that conflates the comprehensive with the complete and has experienced its fair share of credibility concerns) frequently turns up at the top of search results -- and those in the know can work the system to their advantage.

And of course Raphael did not attempt to profit from click-through ads.

I have more to say about all this, but I don't (alas) know enough about it yet to do so.

Lost was excellent tonight, I thought.