Soil micro-environments with augmented reality

For the final I experimented with projection and augmented reality (Unity & Vuforia) to tell the story of how plants remediate their environments. I narrowed this down to phytoremediation with sunflowers, specifically, to make this manageable for a week-long project. Sunflowers accumulate lead from the soil, but like all bioremediators, they then become toxic themselves. I also wanted to show the complexity of the micro-environments in soil. My grand aspiration was to create one of these experiences for each of the ways to remediate soil. 

I put everything together with some text and animations. Many of the soil images I put up on Vuforia as tests got much better tracking ratings when I made them brighter.  I made a big composite image of these trackable parts, and then used screenshots of the composite image as my image targets.


Gabe helped me find 3D models of bacteria and bugs, but I found they had so many vertices and faces they created a lag on the phone. I tried using Blender to “decimate” the objects but this made me want to throw my computer out the window. Instead, I used Blender to identify the objects with he fewest faces and used those. Gabe later suggested correcting this with shaders for mobile in Unity.

It might have been useful to add some more information with audio, since it seems better to avoid lots of text that you’d need to read. I would have liked to incorporate sound but couldn’t find a simple way to attach this to an event, like a found target or a rendered object. I’m not sure this would have added to the experience, since you can see multiple targets at the same time. It’s easier to attach sound to a camera, and maybe I could have added some ambient noise. 

My original project proposal:

Ready Player One

Ready Player One has been appropriately critiqued for being a superficial page-turner, propelled forward by pages on pages of cultural references and tired tropes. These aspects of the book made it exhausting to read, but there were nonetheless a some compelling ideas: the parallels of this VR-scape to the present day internet–particularly around its corporatization, and the details around the pervasiveness and integration of this future VR world.

Aspects of the immersive VR world where the book largely takes place reminded me of Sarah Rothberg’s description of a future work environment–that we may one day just put on headsets that function as our desktops. Wade Watts and his peers go to school in this VR world, but they also maintain robust personal lives, learn, play, and participate in this parallel economy. The reality that everyone is in fact connected to an actual, physical body is something to be exploited by those with power, who can kill. One troubling aspect in this respect was how little processing the characters do when they experience deaths of those close to them. Although, Wade does go to somewhat thoughtful measures to protect his physical body and his real identity. The digital ephemera of the dead person’s avatar is not something that’s addressed–something curiously lacking considering this is already a problem. The only person who gets this kind of forethought is Halliday.

Anonymity in this world is important, which was interesting to reflect upon since this has largely been lost on the modern, social internet. The commercial progress of the current internet seemed all the more apparent when taken to the extreme as it was in the book, where people go to great lengths and spend a lot of money to curate their VR lives. Halliday and OASIS represent some techno-utopian vision of a future VR-scape that has already failed in its lower-fidelity precursor (sad/boring/fiefdom/current internet).

3D Avatars & Unity

This week we scanned ourselves using structure sensors and Skanect, to create 3D avatars that we can animate in Mixamo. It was difficult to get a good scan: things to consider were keeping the sensor level, maintaining a wifi connection with the computer running Skanect, moving the sensor in the right direction at the right speed, and making sure to stay very still. Post-processing in Skanect allows you to color your scan, edit out the ground, and rotate the figure for importing into Mixamo.

While I maintained “claw hands” during my scan, I must have moved a little bit and my hands just messed up anyway. So when I rigged my figure, I used the fewest number of joints, giving me a mitten-hand effect.

As a next step we imported our Mixamo-animated Fuse characters & our own avatars into Unity, and experimented with creating scenes:

NYC sewage system: toilet projection

I recently went on a Newtown Creek audio tour, a project by ITP professor Marina Zurkow, & alums Rebecca Lieberman & Nick Hubbard, where I learned many things I didn’t know about the sewage processing facility there. I was already fascinated by how cities process sewage and where there are opportunities to intervene to create a more sustainable system. Among other things, projection mapping offers an opportunity to put video in unexpected locations, so I thought it would be interesting to put information about NYC sewage system at what is for many people, the most obvious place they interact with it: the bathroom.

I did a bit of research and found some information and a number of videos on the topic. I decided to use “How NYC Works – Wastewater treatment,” an easily downloadable video from vimeo. I also incorporated sounds of urination and flushing, and the sounds of rain (rain in New York is known to cause Combined Sewage Overflows).

As an initial experiment, it was very useful to see what worked well and what didn’t. Below are some still of my favorite parts.

Hansel & Gretel with Twine

I teamed up with Angela Wang to re-imagine the fairy tale “Hansel and Gretel.” You can play here.

Our first step was to deconstruct the story to its main elements, symbols, and themes. We wanted to maintain important aspects but play with other elements: character portrayal, setting, plot. After a lot of ideation that included ideas inspired by the Hansel & Gretel show at the Park Avenue Armory, physical installations, and 360 interaction in the web, we settled on using Twine.

We imagined parallel story lines and different endings–this was the most fun and time consuming part! It was fun to riff off of each other and modernize the creepy aspects of the story. By allowing the different themes and storylines to manifest, it highlighted the oddness of sharing this story with children for so many years.

Some of Gabe’s feedback was to delay the reveal of the story, and to incorporate more of the Twine elements. It was definitely challenging to incorporate all of the game-play aspects that Twine makes available. Aside from fixing things like some awkward language and typos, I think it would also be good to incorporate the 2nd set of questions in the story in a more cohesive way.

Since presenting in class, we’ve had other people play the game, and I think the response has been pretty positive. People find it funny and disturbing, which was our intention. There’s also a certain amount of surprise when people try to go back in the story and find a different path, only to find that they are led to an even more evil ending.

Ricoh Theta: in-class experiment

In class we experimented with 360 photo and video using the Ricoh Theta camera and software. I ran into issues transferring the footage onto my new macbook using Image Capture, and ended up needing to load the camera as a drive.

I took video from different parts of the journey to Bobst library and different areas of the stack, but I haven’t gotten a chance to edit different parts of the footage together. Unlike last year, vimeo now supports 360 video, so I took just one of the scenes, from on top of a glass case, and uploaded that:

Already programmed: responses

Connected, but alone?

Sherry Turkle argues in her TED talk, “Connected, but alone?” that too much texting is bad for us: she anticipates the unpopularity of the talk because this isn’t something people or companies want to hear. People are more comfortable with machines, which enable interaction without real emotional risk. But, they also can’t really empathize with us.

The argument that texting leaves out a lot of cues, and is a bad medium of conversation makes sense (even though I’d like to point out that it allows for other things, like video, photos, group chat, and sharing of virtual content that gives a different richness to these exchanges). I struggle with the assertion that we are more alone than we used to be. Maybe this is true, but I find it hard to imagine that the work required to maintain a basic standard of living, and the isolation of people that didn’t fit into a previously even-more-rigid social structure, allowed for greater social connection and less loneliness in years past (in Western societies). She gives anecdotes of cases where people she’s interviewed specifically avoided in-person interaction, but I wonder how common this really is, and what the alternative would have been before the internet and before texting. Maybe these individuals would have been even worse off.


We are all cyborgs now

Amber Case’s talk, “We are all cyborgs now,” is funny because she makes the absurdity of our new rituals around technology so apparent. She notes the difference between other human tools and computers is that computers act as an extension of our mental selves, rather than our physical selves. One point (which serves as the basis for one of Yueping’s projects!) is this idea that we find ourselves rummaging through this external brain, unable to find our documents or an old google search.

I really don’t need much convincing the we’re already cyborgs, which is her main thesis. I just loved how observant this talk is, and I wonder if other’s see these machines as extensions of our bodies in the same way?


Program or be programmed

Rushkoff’s talk, “Program or be programmed,” is a discussion of his book by the same name, where he makes the case for media literacy. That he feels he needs to persuade people of its importance is kind of ironic since this is the antidote for otherwise being blindly coerced by our history, social context, and capitalist/corporate environment.

I ended up watching his talk, “Open source democracy” (coerced by the youtube playlist) as well, because his historical accounts that bring us into our present socio-economic context and the proposals he makes for addressing the related issues are equally fascinating. That we should learn to code because it changes the way we think, and has real social/political/world consequences recalls Papert’s Mindstorms! And his call for local economies, local solutions, and local engagement recalls Bill McKibben’s Deep Economy.

Skeptically, I think even with greater awareness of how we are coerced, manipulated, “programmed,” I don’t think we have the power to fully reclaim our agency. Cognizance of these mechanisms only helps to the extent that we have power over our reality, and even if we are to argue in favor of free will, this would only give us control over a tiny proportion of our lives. Is it lucky then, if your programming allows you to see through the program?

Responses: Sharing on the internet

Breitbart-led right-wing media ecosystem altered broader media agenda

This analysis sought to examine the media ecosystems during the 2016 election cycle that led to Trump’s ascendance using “hyperlinking patterns, social media sharing patterns on Facebook and Twitter, and topic and language patterns in the content of the 1.25 million stories.” The study points to social media “as a backbone to transmit a hyper-partisan perspective to the world,” with Breitbart at the center. The authors present evidence of the role of social media platforms and the right-wing online media, something that I think many were attuned to at the time–after Trump’s election I cringed and navigated to Breitbart for the first time.

While the authors note (and this is clearly observable by living online) that disinformation is common, they found that political and media polarization online was asymmetric–the right-oriented people got more of their news from newer, more polarized sites. I didn’t realize that right-wing media was younger in general. In the mid-2000s, when I was horrified that the adults elected Bush (twice!) I had imagined the right as Fox-news consumers. Now, even Fox is vilified by this new right wing media. From the comparison of patterns on Facebook and Twitter the authors suggest, “human choices and political campaigning, not one company’s algorithm, were responsible for the patterns we observe,” although both are dependent on click-driven revenue. Regardless, the result is scary and unprecedented:

…the insulation of the partisan right-wing media from traditional journalistic media sources, and the vehemence of its attacks on journalism in common cause with a similarly outspoken president, is new and distinctive.

Rebuilding a basis on which Americans can form a shared belief about what is going on is a precondition of democracy, and the most important task confronting the press going forward.


Rebuilding the Web We Lost

It was so interesting to read Anil Dash’s perspective on the nature and promise of the old social web: from self-hosting your online identity, the importance of flickr, search, and non-monetized links, to norms around data use and tracking, and crowdfunding. My tween experience was mainly around AOL and AIM.  He wrote a companion piece outlining what could be done to bring the most valuable elements of what was lost back. The main things that stood out were diversity and funding: it seems like a combination of an insular industry and its ad-based funding streams resulted in these negative trends.

The biggest reason the social web drifted from many of the core values of that early era was the insularity and arrogance of many of us who created the tools of the time… Another way of looking at the exclusionary tendencies of typical Silicon Valley startups is by considering the extraordinary privilege of most tech tycoons as a weakness to be exploited.

Dash is hopeful about ways we can address these problems, suggesting that the insularity of Silicon Valley provides an opportunity to create something better that would compete with their flawed strategies. But to me this seems like such massive, ingrained problem that it will take several years if they are ever addressed. There are definitely people promoting diversity in tech, but I seldom hear this value espoused at high levels of the industry, where the power of culture change is manifest. The idea that we can promote blue-collar coding is actually somewhat alarming–like the value of the service is different dependent on who does it. It makes sense that industry partners are invested in teaching computer science for this exact reason.

And then there’s funding: is anyone working on funding websites a different way? Is this even possible for the conglomerates that have already taken over? Or do we have to wait for them to be unseated?

…the fundamental reason these sites refused to accommodate so many user demands is because of economics. Those sites make their revenues on models dictated by the terms of funding from the firms that backed them.


Habits of Leaking: Of Sluts and Network Cards

Wendy Chun’s piece addresses many aspects, traditions, and values of western culture, born out out of its colonial, racist, and patriarchal history, that give proper context to abuse (the focus is mainly abuse towards white women) on the web. One of the more salient arguments is around the right to space and the right to citizenship on the web, and how this is connected to the right to exist in public, to loiter.

We need an online public in which women are not victims but loiterers, actively engaging in its public sphere without a discourse of predators, pornographers, and slut-shamers waiting there to ruin them…we need to fight for the right to be vulnerable–to be in public–and not be attacked.

… mass loitering […] creates mixtures and possibilities that erode boundaries and establishes spaces that do not leak because boundaries are not compromised.

I was reminded of Jillian York’s piece on harassment and censorship, because she argues that women expressing themselves, existing, “shout[ing] louder over the din,” is one helpful way to address this abuse. A harmful and effective aspect of online abuse, before it begins to threaten physical bodies, is the way shame is utilized. There’s no a priori reason to feel shame as a victim outside of a culture that hoists shame upon women for existing in public–this does seem like something we could change to strip power of the people that weaponize this against women online, but if possible, will take a long time.

I’ll always remember how network cards work thanks to Chun’s description of how they operate “promiscuously.”

Net Neutrality & Everyone Online: responses

Network Neutrality, Broadband Discrimination

In 2002, Tim Wu proposed broadband discrimination as an alternative to open-access to reach the ultimate goal of network neutrality, and lays out why he believes this should be legislatively regulated as opposed to letting the market self-regulate. Wu describes a neutral network as “an Internet that does not favor one application (say, the world wide web), over others (say, email)” and open-access as “a structural requirement that would prevent broadband operators from bundling broadband service with Internet access from in-house Internet service providers.” Then, he describes a system that would “distinguish between classes of restrictions that should generally be allowable, and those that might raise suspicion” be managed through banning certain activity or price discrimination.

The point of the neutrality principle is not to interfere with the administration of the Internet Protocol side of a broadband carrier’s network. It is, rather, to prevent discrimination in that administration.

This paper raised many questions because of my lack of technical knowledge of the internet at the time and now, and the changes that have occurred in between. What follows is a series of dumb questions.

  • It’s unclear to me how “IP’s neutrality is actually a tradeoff between upward (application) and downward (connection) neutrality.” Does this just refer to fixed bandwidth?  
  • Why does “Delivering the full possible range of applications either [require] an impracticable upgrade of the entire network, or some tolerance of close vertical relationships.” First, why is it impracticable to upgrade networks? Is this referring to a solution within a certain timeframe? Surely networks get upgraded? Then, he references “vertical integration” and “vertical relationships” as potentially necessary, but I’m not sure why, or what an application that requires this might be.
  • Potentially related: the landscape of providers didn’t allow wifi in 2002! What changed between then and now? Did changes come about because of infrastructure improvements or changing norms with the providers themselves?
  • Not addressed (he does say broadband economics is out of scope): is there a problem with the suggested market approach to selling bandwidth? If there’s a limited supply, does that leave the people least able to pay without connectivity? Does this mean there should be bandwidth caps?

This paper made me very curious about solutions to fair network access now that this is increasingly considered a right. If part of the problem is infrastructure related, maybe upgrades in the public interest could be funded through social impact bonds?


Here comes everybody

Chapter three, “Everyone is a Media Outlet,” of Clay Shirky’s 2008 book describes the “amateurization” of journalism. Traditionally, the “news” has been “newsworthy events” and “events covered by the press” (I think this means things trained journalists have deemed important). Now that the internet lets just anyone have a blog or social media, people report on and write about things only professionals used to do. I’m not sure if Shirky thinks this is a problem in itself, but he does posit there is an issue with the field not understanding how their role as gatekeepers of news is dissolving.

What’s so great about elite gatekeepers and professionalism?

What Shirky presents as this sort of weird problem with remarks Senator Trent Lott made at Strom Thurmond’s 100th birthday party actually seems to me like a victory in delivering news that is important to subgroups of people that otherwise would have gone uncovered by professional journalists. In this instance the controversy affected right-wing bloggers and libertarian republicans, but if “the key to any profession is the relation of its members to one another” then ASNE’s Newsroom employment census shows the field of journalism may be missing some relationships:

“The percentage of minority journalists has hovered between 12 and 14 percent for more than a decade. In 1978, when ASNE launched its Newsroom Employment Census of professional full-time journalists, 3.95 percent were minorities.”

Perhaps the “internal consistency of professional judgement,” established by journalism is mostly good, except for when it blinds the field to large changes. I am more broadly skeptical of professional norms than this.

I was convinced that there should be a Federal Law that defines who is protected by journalistic activity and some definition of what this is. I hear this complication about bloggers but surely there can be some distinction made between important investigative activity and opinion pieces/editorial?

Chapter four “Publish, Then Filter” elaborates on patterns of communication that are changing. “Saying something to a few people we know used to be quite distinct from saying something to many people we don’t know” but now we have these “many to many” conversations online. The best part of this chapter was the discussion around filtering information the vast ecosystems of information, and how “communities of practice” that can form naturally benefit in great ways from new ease of communication.

Mirrors, tails, and labor: responses

Beyond the Mirror World

Philip Agre’s “Beyond the Mirror World” largely deals with the question of representing the real world with quantitative data. He gives examples of data use that seem to demonstrate the Capture Model he proposed in “Surveillance and Capture, Two Models of Privacy” and examines Gelernter’s idea that “the progress of computing will inevitably produce a single vast distributed computer system that contains a complete mirror image of the whole of reality.” He also addresses issues of privacy, and complications related to de-identifying data.

Perfect representation is impossible: we can at most measure proxies of things and activities, which reminded me of Borges’ “On Exactitude in Science“, which has to do with the development of a map so exact it becomes a perfect replica and is thus useless. Abstract representations are useful because they allow us to learn and express truths more broadly. Otherwise, what’s the point? We might as well just look around. It’s too hard to synthesize or extrapolate this way. The alternative is to examine the mirror-world in some software mediated way. Some data are hard to perceive in our physical reality: sound (because of its temporal property), radio waves, psychometric measures, etc. How Borges’ imagines perceiving infinity feels relevant in this way, too.


The Long Tail

It was delightfully nostalgic to read Chris Anderson’s The Long Tail, which examines the effect and potential of web-mediated media consumption in 2004. The sudden ease by which we could consume media in the late 1990’s and early 2000’s was exciting and overwhelming (I remember collecting a massive archive of music that I lost when my computer crashed–my music identity never recovered and by then no one owned music anymore–physical things are important). The big change was upending the 20-80 rule, since the internet allowed a market around more obscure media to be profitable.

While the options for consumers and the profits for visionary companies that developed algorithms to help people find niche content are elaborated upon, very little is said regarding the effect on the artists, the many companies cited as examples of old-school mainstream media providers, and whether we’re okay with new media companies collecting our personal data to market to us better. I feel a certain amount of resentment for the large media conglomerates that serve(d) as gatekeepers: I saw Triplets of Belleville and theaters and it’s one of 3 DVD’s I own. Hollywood still exists and smaller artists with distributed followings still struggle. Are algorithmic suggestions for similar content making us more isolated? And is the convenience worth the privacy trade-off?


Free Labor: Producing culture for the digital economy

Tiziana Terranova’s essay critiques the state of post-capitalist labor markets as they’ve developed specifically alongside the internet, arguing that the internet is reliant on free labor. It’s unclear to me whether it’s more true that, “the fruit of collective cultural labor [is] not simply appropriated, but voluntarily channeled and controversially structured within capitalist business practices” than it has been in the past. It seems that culture has always been produced on the fringes and slowly incorporated into the mainstream–if someone with investment capital pays attention then it becomes a commodity.

I think there’s also some validity to the argument that internet/culture/content work is devalued because of the “minoritarian, gendered, and raced character of the Internet population.”

“How many women have produced a fuckton of tech content for medium that hasn’t been compensated?” – @shanley, 2014

Nurses and teachers don’t see their wages increase in response to increased demand because of the demographics that dominate these fields (and when white/men do enter these fields they earn much more, but duh they always do). But with regards to knowledge work, a key distinguishing factor is its collaborative nature:

“The acknowledgment of the collective aspect of labor implies a rejection of the equivalence between labor and employment.”

I’d argue that employment has also been divorced from wages, and even more strongly dissociated from wealth: more/better labor, more “productive,” societally essential jobs, do not lead to more/better wages. And just because you’re employed doesn’t mean you’re a good person. Acknowledgement of these realities is part of why we see increased discussion around universal basic income.