My cousin Paul is a genius. He graduated from a special high school for geniuses when he was sixteen years old. Today, one of Paul’s projects is working for Google on the company’s virtual reality application, Welcome to Light Fields. Paul is also one of the people who developed the technology used to create the avatars in the movie Avatar and an aging Brad Pitt in Benjamin Button, among others. The light fields app is a collection of algorithms used to capture, create, and render a virtual reality that you can then view through a special headset.
I don’t have a headset capable of viewing a virtual reality. For a long time, I had enough problems viewing actual reality. I was afraid to add another, even though I think Paul’s creations are amazing and important.
When I was young I spent a lot of time feeling as if I were just outside of the real world—as if I was not a part of anything, as if I wasn’t real. I had no idea then that my feelings of separateness and otherness were the natural results of having a neurodiverse brain. When I became upset and distraught, I’d write the same line over and over again in my journal: “All I ever wanted was to be real.” Looking back, that plea seems especially sad to me; my reaction to my struggles was to conclude I wasn’t even real. My efforts to insert myself into the “normal” world, to not be detected as a fraud, to not seem weird and different, exhausted me. I never completely succeeded. I remained perpetually on the outskirts, though, as I got older, I became much better at “camouflaging.”
Camouflaging refers to the way individuals on the autistic spectrum (predominantly women, according to studies) develop patterns, rituals, and conscious behavior alterations in order to operate in the neurotypical world undetected. I’m not one of those people who is especially awkward in social situations, nor am I on a place on the spectrum where I experience overwhelming compulsions to sit in a corner with knees up to my chest and rock back and forth. Not that I haven’t done exactly that at times in my life, but I’m generally adept at performing accepted social rituals—things like eye contact, friendly laughter, lots of “hmm hmm”–ing in the correct places. I’m not a fan of the term camouflaging in this instance, perhaps because, to me, the word implies something simple like wearing clothing meant to mimic a background. I think of it instead as a process of relighting, of altering the way I’m perceived so that it’s difficult to notice that maybe I don’t exactly fit into a particular environment.
For SIGGRAPH Asia 2019 (a respected and innovative computer graphics conference), Paul and his team showed a project called Relightables, a method of filming and photographing people that allows their images to be composited into other virtual environments, relighting them so that they appear as if they were actually photographed in that environment. This is not as easy as it may sound. In order to achieve the desired lighting effects, the person must be captured from all angles and the light reflections must be recorded. One of the breakthroughs in this technology was figuring out a way to capture a broader light spectrum so that artists didn’t have to spend hours color-correcting a composite image of a person. Initially, the LEDs used were only red, blue, and green. Adding cyan, amber, and white to the mix made a better representation of the full spectrum of daylight possible.
Paul believes it’s rather silly to have artists do things that can be accomplished by technology. He thinks artists are creators, innovators; technology should help them tap into new and immense aspects of their individual genius and creativity, not make them spend hours correcting something technologically based.
In the bizarre way my brain sometimes makes connections, I relate to this. I observe the world, mentally record information about how people react and move and interact, and then use that information to help my own self blend into situations that I might not otherwise. This, too, is not as easy as it may sound.
Light fields are necessary to make you think you are in a three-dimensional space as opposed to a two-dimensional space. Light reflects differently off near and faraway objects. It’s all about the light. Paul told me that. In order to achieve a digital creation of an object or person that the human brain interprets as “real,” you must photograph that person or object in a sphere of light, snapping multiple images in a very short time from every conceivable angle. In this way, you create normal maps that can be used to render digital versions of objects and persons that look real. These maps are essentially a collection of images and data about the way light reflects off of a thing from every angle and viewpoint, achievable because of the sphere.
I keep normal maps in my head of people functioning in various social situations like an office party or a coffee shop or dinner out with a group of folks they don’t know very well. With such maps, I can also be successfully integrated into a slew of circumstances.
I kept a diary all through my adolescence. Somewhere during my teenage years, those diaries turned into journals. I wrote in thin, lined, spiral, letter-size notebooks. I bought them in threes at the Oberlin College bookstore. The covers were tan, manila card stock with the college logo on the front in black ink. I have dozens of these notebooks packed away in a box in the corner of the top shelf of my living room closet, a shelf that’s difficult both to put things up onto and pull things down from. Sometimes I wonder how I ever had so many words to impart, so much to say. These days I enjoy silence and the lack of words much of the time.
I don’t see Paul very often. We’re technically second cousins. Our Aunt Pauline had a pool table in her basement, and we’d play when the family gathered for Thanksgiving. Paul was an exceptionally adept pool player, explaining the geometry and physics of where and how to aim a pool cue and send a ball where you wanted it to go. I was classified a “gifted” child, in the top one percent of my class, by the Ohio State Board of Education but I could tell, even way back then, that Paul’s brain operated on a whole other level. I was always shy, didn’t really know how to talk to people, and Paul’s intelligence intimidated me. But he was also kind, even as a teenager, and I realized that he was able to see the world in more than one way, that he navigated multiple realities—or at least what I perceived as multiple realities—in a way I didn’t think was possible.
What I mean is: Paul would stand at the edge of the pool table, cue stick in hand, talking about trajectories and velocities and weight and speed of objects, stringing together sentences that became increasingly jargon-loaded and technical, sentences I only partially understood. Then, he’d walk over to me and explain the same things again, without all the formulas and math. He showed me where to hit the ball in order for it to end up where I intended it to. Not many people can take complicated science and math concepts and reshape them so a person not deep in the world of mathematical formulas and hypotheses understands. He always seemed so bright to me, both literally and figuratively.
Today, Paul’s technology is realized in a physical contraption called Light Stage 6. (There were five others before this version.) A person filmed inside Light Stage 6 doing something—anything—can be projected into a digital landscape doing that thing and the human eye perceives that the person is actually in the digital environment. There’s no longer that uncanny, slightly off effect of earlier computer-generated surroundings, when you could tell a person wasn’t really where the filmmaker wanted you to think they were. Determining whether or not a person was filmed on a green screen used to be pretty easy; it wasn’t yet possible to alter the way light reflected off of something digital. Essentially, you couldn’t match the lighting effects so the two realities never quite aligned.
Light Stage 6 captures a person in thirty-three different light conditions in one-thirtieth of a second. This is achieved with programmable LEDs and a capture system of cameras and illuminators inside the light stage that replicate all sorts of lighting environments.
Many years ago, I had a conversation with Paul about clothing and wardrobe in film. I have worked as a tailor and pattern maker in the entertainment business for thirty years.
“You should look into working as a consultant on computer-generated images of clothing,” he told me, “Right now, we can successfully create digital humans, but it’s more difficult to dress them, to know where the shadows should fall on a specific piece of clothing.” There’s so much subtlety and light transitions in the folds and seams of materials that expertise in wardrobe construction is essential to understand how to digitally render a garment.
I didn’t follow through with this advice, though perhaps I should have. Paul’s most recent technology is paving the way for filmmakers to be able to do everything digitally, even direct an actor. If I’m being honest, this frightens me somewhat. I fear the lines between realities are becoming entirely too blurred. But, if I’m being even more honest, the lines between my realities have always been hazy.
I make up stories in my head that I tell myself over and over, intricate and complicated renditions of an imagined life. I see them all so clearly in my mind that I convince myself they are real, that they did happen.
Conversely, I have moments when I gaze at my reflection and feel as if I am looking down on myself from a distance. The light falls in a weird way across my face, just like when you’re watching some computer-generated human that looks almost real, but isn’t. Something is off, just by a little bit, the light’s reflection not quite right.
Sometimes I become confused about the things I’ve made up and things that actually happened. Sometimes I fear I made my entire life up; a virtual reality so expertly rendered that I cannot tell that it is, indeed, virtual.
If you’re not happy with the world you are in, you need to build a new one.
The technology exists to build virtual realities that are indistinguishable from actual reality. When I listen to Paul talk about light fields, I am always struck by how in awe of it he still seems to be—despite the fact that he’s one of its main creators. I have no idea if he actually feels awe; this is just my perception of how he feels. He seems to feel genuine joy explaining it all, and he’s so very good at it.
What I’ve slowly come to realize is that a virtual reality is real. People take issue with whether or not it can actually be touched—but that doesn’t make it less real, just real in a different way.
In addition to adolescent anorexia, I also suffered from self-harm by “cutting.” I still have a scar on my right thigh where I cut myself with a seam ripper. I was always attempting to verify my solidness, my tangible existence.
Technology exists now where anyone can create a virtual reality. There’s a new VR180 handheld capture system (thanks to Paul and other brilliant researchers at Google) made up of two fisheye cameras with one-hundred-and-eighty-degree viewing capabilities. That’s all anyone needs to start lighting and rendering their own virtual realities.
The name “Light Fields” conjures up vast expanses of flowered land bathed in sunlight. I imagine myself in the middle of one dancing, twirling, hair splayed out, the light bouncing, bringing out the red. I am one with this land, this world. I look down at my bare toes curling into the dirt, and even they are awash in light. There are no clouds, no trees offering shade, just light. I feel the soil between my toes, sandy and grainy, small pebbles pressing into the balls of my feet. I don’t need a virtual reality camera or capture system to achieve this and I’m not sure that I’d ever want one. But that’s just me.
After almost fifty years, I’ve finally come to realize the beauty and, I think, the necessity of virtual realities. There is more than one way to understand, view, and exist in the world. I’ve also stopped equating virtual with not real—which was always faulty logic anyhow. I mean, not-real reality is essentially a contradiction. Real is real, tangible or otherwise.
As relighting becomes more easily accomplished with technology, I’ve arrived at a weird sense of pride in my sometimes not-seamless integration into reality. I’m no longer performing frantic color correcting in order to make my composite self fit perfectly into a scene. I am okay with my light being slightly off. I kind of think that in this day and age it makes me even more real. Just as I always was, even when I didn’t know it.
The light is everywhere.
***
Rumpus original art by Cyrus Finegan.
