Understanding Networks: RESTful API and Controller

As outlined in the first blogpost, the assignment for our group was two-folded:

  • build an API & Server for our classmates Vidia and Lucas - they built an interactive limbo-stick after our specifications

  • build a physical controller for our classmates Simon and Keerthana - they built a dance-step projector

The assignment proved to be a great challenge, especially the coordination tasks that come with working as group with two other groups (that do the same) on two different projects over a longer period of time with no pre-defined work environment - but we finally succeeded!

Lucas and Vidia made a video for their limbo-stick machine, built after our API specs/deployed server.

And here a pic and code from the physical controller we built for Simon and Keerthana (and their dance machine): laser-cut/etched acrylic, lcd-screen, arduino mkr1000 wifi, potentiometers, bamboo box

wVoNicRHSQiiz2rabdSpAA.jpg

And a little prototype video of the controller in action (controlling the node-based dance step controller built by Simon and Keerthana):

Understanding Networks: RESTful API

update

assignment week1/2

Decide on your application, then describe and sketch the control surface and the specify the REST interface. Describe your application and present your REST API. Don’t build the project, just describe it and its functions. Write a blog post detailing this description. You’ll be assigned someone else’s specification to build this week.

concept

Inspired by the playfulness and shared group experience of parties Sandy, Kellee and I decided to create an interactive and RESTful Limbo-Stick.

context

A little bit of research on the origins of Limbo showed that the dance is considered the unofficial national dance of Trinidad & Tobago, together with steel drums and calypso as national heritage. In its original form the bar is raised slowly from the lowest levels, resembling the entry of death into life. When the dance gained more popularity this form was flipped - dancers are starting now high and the bar slowly lowers.

system overview

 
 

physical interface (draft)

Usual limbo setups look like our rough sketch:

 
 

RESTful API

For the RESTful API for the dance/game we followed an example to define our five basic states: “off”, “on”, “height adjustment”, “idle”, “collision”.

POST-requests

//*On/Off Mode*
POST /switchMain/{bool} 
//value: 1 to turn it on,  0 to turn limbo stick off

//*Height Adjustment Mode*
POST /height/{bool}
// Value: 1 to lower the height of the limbo stick one step on the predefined scale, 0 to keep current position

//*Collision Mode*
POST /collision/{bool}
//value: 1 if collision is detected, 0 if no collision is detected

//*Alarm Mode
POST /alarm/{bool}
//Value: 1 to turn it on, 0 to turn alarm off

//*Home Mode*
// If POST /height counter  > 6, go into home mode:
POST /home/{bool}
//value: 1 to home the stick to highest position on the predefined scale, 0 to keep it current position

GET-request

GET /state/

//returns
{
  "switchMain": {0, 1},
  "height": {0, 1},
  "collision": {0, 1},
  "alarm": {0,1},
  "home": {0, 1}
}

physcial interface/controller

 
Screen Shot 2018-11-19 at 8.08.17 PM.png
 

Understanding Networks: Packet Analysis

Assignment:

Capture and analyze traffic on your home network while going about your usual network activities. Present your results in summary form, using graphical analysis where appropriate.

background traffic analysis

I used wireshark throughout the assignment in the ITP-sandbox network.

After a few basic experiments with querying a very basic webpage and looking at the sent/received packets, I went back to zero: I started measuring just traffic when all applications are off over the course of 5 minutes. Surprisingly there is still a lot of background network traffic happening, especially application data, without me actively engaging in any online activities.

OS X source traffic (my IP: 128.122.6.150)

OS X destination traffic (my IP: 128.122.6.150)

Ubuntu source traffic (my IP: 128.122.6.149)

Ubuntu destination traffic (my IP: 128.122.6.149)

I used two operating systems to compare the background traffic: OS X and Ubuntu. Ubuntu only shows one third of overall network traffic compared to OS X.

The latter one shows a lot of connections to IP addresses starting with 17 - these are registered to Apple in Cupertino. A more detailed view shows a variety of different activities that my OS X operating system is performing and sending data back and forth between Apple servers and my machine:

Screen Shot 2018-10-27 at 3.59.12 PM.png
Screen Shot 2018-10-27 at 4.43.15 PM.png
Screen Shot 2018-10-27 at 4.42.20 PM.png
Screen Shot 2018-10-27 at 4.37.04 PM.png



Some further research on the sent TCP-packets showed that those processes refer to Apple cloud services (as I am using iCloud to sync my photos, calendar etc.) or programs like iTunes.

On my Ubuntu machine, I do not use any cloud service that that is tied to the operating system, therefore 1/3 of background activities.

As I got interested in this “hidden” traffic I did some further research on the TLS - Layer and used wireshark to go through each step of the TLS -protocol:

 
 
tls.png
 
 

Looking at the cipher change protocol specifically, I did further research on the encryption part (here the negotiated cypher-suite) of TLS - and finally understood Diffie-Hellman and RSA encryption. That was worth the extra hours … ! I have to confess that trying to sketch all components of RSA encryption still gives me headaches, compared to that Diffie-Hellman seems a bit more simple and elegant. To me it was not obvious to choose which one over the other, intuitively I would choose Diffie-Hellman over RSA. And I wondered why Apple is “downgrading” my (possible) SHA384 encryption to SHA 256 in the cipher suite negotiation in the protocol.

Screen Shot 2018-10-27 at 7.25.07 PM.png

After reading the Diffie-Hellman vs RSA post on stackexchange, I was a bit less confused: Non-ephemeral RSA encryption seems to be the industry standard for now as its generally faster to compute. Diffie-Hellman is more secure, but more difficult to compute.

I suspect Apple is using the lower encryption standard (RSA with SHA 256) due to the sheer volume of traffic on their servers.

usual daily network traffic analysis

For my usual daily network traffic analysis I had a couple of browser windows open, mail and terminal. In one of the browser windows I was running an online-radio station (nts.live). I captured data over the course of 5 minutes.

Here the output looking at the source-traffic (my IP: 128.122.6.150, running OS X):

Here the destination side of traffic:

Finding out on which data channel my online-radio station is running proofed to be difficult as the stream is hosted not under the website-ip address but on a different server. I suspected either one of the ip addresses below as they showed a continuously high traffic via TCP (initially I expected UDP but as the stream is https, TCP makes sense). Both pointed to amazon-servers.

Screen Shot 2018-10-29 at 11.30.57 PM.png

To find out which one might be the one hosting the stream, I just closed the tab running the online radio station. Here the results:

no_radio.png

To my surprise both were muted now and didn’t appear on the traffic overview anymore. As the website is hosting two streams at the same time, I guess it might load both from different servers even when I would only be able to listen to one? My fellow classmate Beverly pointed me into the right direction: I should check the sent packets in Chrome directly - and here two streams are loaded at the same time! This is for sure eating into the bandwidth …

Screen Shot 2018-10-29 at 11.48.07 PM.png

Surprisingly the connection to the Apple servers was somehow quiet during multiple wireshark-captures while running Chrome and Mail. The server-IP starting with 17 (apple server range) does not appear in the traffic overview at all. Why this is the case, is not quite clear to me. Maybe background processes are only run while no other traffic is using the bandwidth? I can only guess at that point.

Now enough of packet sniffing, TCP, TLS and UDP - I learned a lot and got a lot more interested in encryption, which will be the topic of a future class in a few weeks. Awesome!

Understanding Networks: Traceroute

assignment

Traceroute at least three of the sites you regularly visit. Do it from all of the locations you regularly connect from. Save the trace in a file, and make a map of the routes, indicating the network providers that show up every time. Identify who the major tier 1 providers are in your life. Identify the networks your traffic crosses in the course of your daily life. Figure out whose hands the data about your life goes through on a regular basis. Look for patterns from your network-browsing habits through analysis and graphing of your network traces.

setup

I traced four websites that I frequently visit:

I chose two networks/locations for tracing that I spend most of my time at:

  • university wifi (+ mobile network comparison in same location), at Washington Square/Manhattan

  • home wifi (+ mobile network comparison in same location) in Brooklyn

I wrote two custom tracerouting tools to automate the process as far as possible:

  • tracer.py: combined traceroute and isp-lookup/geoip for given list of urls, returns json-txt file

  • mapper.ipynb: matplotlib based tool using basemap to create traceroute maps from json-txt file

server footprint from home

 
 

I live in Brooklyn. This is where I spend the few hours that I am not on the floor at ITP.

Looking at the footprints I observed that connecting to my homepage (which I registered in Europe with Squarespace) from my homw wifi causes the trace to bounce back and forth between EU and US - until finally reaching the US - and the routes my signal travels on are owned by Time Warner Cable and Akamai Technologies:

The route looks similar when using a mobile connection, but with different providers: Zayo Bandwidth and Level 3 Technologies stick out, Akamai Technologies comes up again as ISP in Europe:

Looking at another footprint (nts.live), Time Warner Cable dominates the servers in the US:

The same trace with mobile emphasizes again Zayo Bandwidth for mobile-networks:

Connecting to NYU and Stackoverflow does not yield too many interesting results, both stay more or less close to or entirely in New York. The only strange behavior comes from trying to traceroute Stackoverflow on mobile - it does not allow to traceroute, the signal gets stuck in a local private network (in all locations).

Here the connection to Stackoverflow via wifi which travels to Denver and then comes back to New York:

 

server footprint from ITP/NYU

 

ITP/NYU is situated at Washington Square in Manhattan. Here I spend most of my time, logged into NYU wifi or the mobile network.

Comparing the traces of the two wifi-networks (home and university), the paths through the servers look different for my homepage - in the US the network provider is not clearly identifiable, NYU provides the last visible trace before it goes to Europe. There GTT Communications comes up as a provider:

The trace for nts.live shows a connection to Europe that does not come in the map from my home wifi, the network provider that pops up is Hibernia Networks Netherlands. Why? It might have to do with NTS having multiple IP-addresses, maybe the server was more easy to access from NYU. Maybe. I can only speculate at the moment. Anyway, here the map, accessed from NYU wifi:

On mobile the connection stays in the US (and again in the hands of Zayo Bandwidth as ISP):

Takeaways

To make it short - this is all very interesting! My online life is in the hands of very few network providers, they vary depending on which type of network I am connected to - and the routes vary sometimes substantially, detours to Europe are not always explainable for me. I really enjoyed understanding much more of this physical layer of the internet and how every request from a laptop of phone travels around the globe at -literally - light-speed.

I thoroughly enjoyed building custom tools in python for it and dive a little bit into data visualization with matplotlib and basemap - although I encountered quite a few challenges on my way: nested dictionaries are great, but need a logical setup, building tools takes way more time than actually collecting and visualizing the data.

Let’s finish this blogpost with a screenshot from parts of the json-data (a little bit obscured):

Understanding Networks: Ball Drop Game - Controller

I created a game controller with the RaspberryPi Zero W using websockets in Python and the GPIOs connected to switches. I chose this setup as the PiZero offers a terminal to run shell commands, can be accessed headless via ssh and offers the possibility to run Python scripts. It is also very portable and can be mounted on a small controller.

The connection to the game server via sockets was very smooth and reliable, the GPIO connection with the switches worked well - after quite a few hickups regarding the wiring of the switches: I initially wired all switches directly into one power source without separating them via resistors. As a result all switches fired with each other once one was triggered. After a few hours of hardware debugging (and de-soldering …), I mounted the resistors in between each power connection and the switch and everything worked smoothly.

I finally added a LED to indicate a successful socket connection to the server.

 

Here the code for the socket connection and GPIO wiring in Python:

import RPi.GPIO as GPIO # Import Raspberry Pi GPIO library
import socket
import time

s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.connect(('Server-IP',Port))

GPIO.setwarnings(False) # Ignore warning for now
GPIO.setmode(GPIO.BOARD) # Use physical pin numbering
GPIO.setup(35, GPIO.OUT) 
GPIO.output(35, GPIO.LOW)

#check connection
data=s.recv(1024)
if len(data) > 0:
 print('yay, we are connected')
 GPIO.output(35, GPIO.HIGH)

msg1='l'
msg2='r'
byt1=msg1.encode()
byt2=msg2.encode()
msg3='l'
msg4='r'
byt3=msg3.encode()
byt4=msg4.encode()

#for i in range(0,50):
 #       s.send(byt2)
  #      time.sleep(2)
   #     s.send(byt1)
    #    time.sleep(2)
#s.close()
def button_callback_l(channel):
    print("go left!")
    s.send(byt1)
def button_callback_r(channel):
    print("go right!")
    s.send(byt2)
def button_callback_u(channel):
    print("go up!")
    s.send(byt3)
def button_callback_d(channel):
    print("go down!")
    s.send(byt4)

GPIO.setup(11, GPIO.IN, pull_up_down=GPIO.PUD_DOWN) # Set pin 10 to be an input pin and set ini$
GPIO.add_event_detect(11,GPIO.FALLING,bouncetime=500, callback=button_callback_l) # Setup event o$
GPIO.setup(7,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.add_event_detect(7, GPIO.FALLING,bouncetime=500, callback=button_callback_r)
GPIO.setup(13,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.add_event_detect(13, GPIO.FALLING,bouncetime=500, callback=button_callback_u)
GPIO.setup(15,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.add_event_detect(15, GPIO.FALLING,bouncetime=500, callback=button_callback_d)
message = input("Press enter to quit\n\n") # Run until someone presses enter
GPIO.cleanup() # Clean up