“In 1973 Thomas Pynchon’s Gravity’s Rainbow was awarded the Nebula, the highest honor available in the field once known as “science fiction” — a term now mostly forgotten.
“Sorry, just dreaming… [T]hough Gravity’s Rainbow really was nominated for the 1973 Nebula, it was passed over for Arthur C. Clarke’s Rendezvous With Rama, which commentator Carter Scholz rightly deemed “less a novel than a schematic diagram in prose.” Pynchon’s nomination now stands as a hidden tombstone marking the death of the hope that science fiction was about to merge with the mainstream.”
From the opening to Jonathan Lethem’s 1998 Village Voice article, “The Squandered Promise of Science Fiction.”
Lethem describes the SF of the 60s and 70s as:
often word-drunk, applying modernist techniques willy-nilly to the old genre motifs, adding compensatory dollops of alienation and sexuality to characters who’d barely shed their slide rules… [It] also made possible books like Samuel Delany’s Dhalgren, Philip K. Dick’s A Scanner Darkly, Ursula LeGuin’s The Dispossessed, and Thomas Disch’s 334 — work to stand with the best American fiction of the 1970s, labels, categories, and genres aside. In a seizure of ambition, SF even flirted with renaming itself “speculative fabulation,” a lit-crit term both pretentiously silly and dead right. For what makes SF wonderful and complicated is that mix of speculation and the fabulous: SF is both think-fiction and dream-fiction.
Lethem goes on to describe how “fabulists like Borges, Abe, Cortazar, and Calvino flourished abroad,” while “a strain of literary puritanism quarantined imaginative and surreal writing from respectability here.” Until the 60s, SF provided the best outlet for those impulses, and for idea-driven novels, and for stories that would “acknowledge the technocratic impulse that was transforming contemporary culture,” while the genre as a whole gradually became more literary, adding “characterization, ambiguity, and reflexivity” to its repertoire.
But a funny thing happened on the way to the revolution. In the ’60s, just as SF’s best writers began to beg the question of whether SF might be literature, American literary fiction began to open to the modes it had excluded. Writers like Donald Barthelme, Richard Brautigan, and Robert Coover restored the place of the imaginative and surreal, while others like Don DeLillo and Joseph McElroy began to contend with the emergent technoculture. William Burroughs and Thomas Pynchon did a little of both. The result was that the need to recognize SF’s accomplishments dwindled away. Why seek in those gaudy paperbacks what was readily available in reputable packages? So what followed was mostly critical rejection, or indifference.
And he has some great lines a little further down:
But the literary traditions reinforcing that division are only part of the story. Among the factors arrayed against acceptance of SF as serious writing, none is more plain to outsiders than this: the books are so fucking ugly. Worse, they’re all ugly in the same way, so you can’t distinguish those meant for grown-ups from those meant for 12-year-olds. Sadly enough, that confusion is intentional, and the explanation brings us back again to the mid ’70s.
It’s now a commonplace in film criticism that George Lucas and Steven Spielberg together brought to a crashing halt the most progressive and interesting decade in American film since the ’30s. What’s eerie is that the same duo are the villains in SF’s tragedy as well, though you might want to add a third name, J. R. R. Tolkien. The vast popular success of the imagery and archetypes purveyed by those three savants of children’s literature expanded the market for “sci-fi”, a cartoonified, castrated, and deeply nostalgic version of the budding literature, a thousandfold. [...] The golden mean of an SF jacket since 1976 looks, well, exactly like the original poster for Star Wars. Men of the future were once again thinking with their swords — excuse me, light sabers.
The article was archived by a guy named David Myers, who chronicled Lethem’s every move and every publication up to October 2004, when he evidently lost interest in being a superfan. The hand-coded fansite itself makes for hours of interesting reading.