ICM FINALS UPDATE: ANOTHER INSIGHT AND THREE.JS

intro

As in PCOMP I switched from flying solo for my project to a collaboration with Chian Huang: she showed me her project and was open to the idea about working together. Her concept was based on a web-based VR/3D experience with camera inputs including sound. I quickly agreed to the collaboration and after a brainstorming session we altered the initial idea and came up with a rough outline for our project: "Watch yourself watching others, watch others watching you - maybe."

keywords

social networks, surveillance, peer pressure

concept

The system should create a VR network structure where multiple agents can interact with each other  - all in google cardboard. As the latter technology will obscure the eyes, the participants are not sure whether they are being watched by the other participants. Audio will be disabled and body parts exchanged between the video feeds. The physical interchangeability of the participants highlights a distinct feature of new social networks: The user is just a datacloud, which can be sold off in parts to the highest bidders. Our obsession with watching ourselves and others is the fundamental interaction of virtual social networks. But we never really know what other network members see. 

Chian's initial sketch is based on a p5 example and augmented with music. 

We based our concept on those video-streams and used webRTC to have multiple participants in a virtual space. All of the sketches are running on a remote server in the cloud. 

23584255_1717900821595415_1844625378_o.jpg
23634077_1717901148262049_1978519266_o.jpg
23633583_1717904028261761_287846985_o.jpg
23633219_1717901038262060_2031692908_o.jpg

After experimenting with getting an external webRTC stream as a texture into p5/WebGL, we decided to switch to three.js  - it seems to offer better rendering and more flexibility with webRTC. 

Here a few three.js example sketches that we might use as starting points for our project:

Screen Shot 2017-11-15 at 6.02.15 AM.png

We still have not finally decided on the aesthetics of this "surveillance" - experience. We would like to keep it open until we both feel more comfortable with three.js  - which seems to be considerably more challenging than p5, but as well a lot more powerful when it comes to 3D rendering.