NOTES.txt

-> fix fig 12 (old code)
-> create fixed testset
-> restructure IPY
-> create possibility to run on cluster
-> create classifiers
 -> Random Forest on parametrization
 -> NN
 -> CNN
 -> ResNet50
-> experiment with random width

-> create simple encodings for figure 1 of cleveland mcgill

-> create multi regression problem

->




#
# CHECK IF CROSSVALIDATION COMPLETED
#
#
#>>> import os
#>>> for root, a,b in os.walk('/n/regal/pfister_lab/PERCEPTION/RESULTS_NEW'):
#...   if root.endswith('Framed') and len(os.listdir(root)) < 80:
#...     print root
#... 


#
#
# ANGLE
## https://stackoverflow.com/questions/25227100/best-way-to-plot-an-angle-between-two-lines-in-matplotlib
#





#
# thats how we create the virtualenv
#
conda env create -f CONDAENV




#### SLURM TEMPLATE


#!/bin/bash

#
# #LAUNCH THE JOBS WITH:
# #for f in *.sbatch; do sbatch $f; done
#
#
# add all other SBATCH directives here...
#
#SBATCH -p serial_requeue
#SBATCH -n 1 # Number of cores
#SBATCH -N 1 # Ensure that all cores are on one machine
#SBATCH --mem=220000
#SBATCH -t 1-12:00
#SBATCH --mail-type=ALL
#SBATCH --mail-user=haehn@seas.harvard.edu
#SBATCH -o /n/home05/haehn/SLURM/gp/out-merger.txt
#SBATCH -e /n/home05/haehn/SLURM/gp/err-merger.txt

# add additional commands needed for Lmod and module loads here
source new-modules.sh

module load Anaconda/5.0.1-fasrc01

source activate CP

#
#
#
srun --pty -t 3-00:00 --export=ALL --ntasks=1 -p seas_dgx1 --gres=gpu --tunnel=44433:44433 --mem=32000 /bin/bash
# now on my machine
ssh -g -L 0.0.0.0:44433:localhost:44433 haehn@odyssey.fas.harvard.edu
# and then there
ssh -L 44433:localhost:44433 seasdgx104

# seasdgx104


#
# SSHFS
#
ssh -f haehn@odyssey.rc.fas.harvard.edu -L 3333:0.0.0.0:22 -N
sshfs -p 3333 haehn@127.0.0.1:/n/home05/haehn/Projects/CP CP_cluster/



#
# more related work
# CLEVELAND MCGILL
#
# chart generator: http://codementum.org/cleveland-mcgill/
#
## https://link.springer.com/chapter/10.1007/978-3-642-14600-8_46
## https://www.perceptualedge.com/articles/b-eye/data_visualization_bookshelf.pdf
## http://flowingdata.com/2010/03/20/graphical-perception-learn-the-fundamentals-first/
## https://github.com/tidyverse/ggplot2/wiki/Recommended-Reading
##
## https://webpages.uncc.edu/~ltharri1/files/chi_13.pdf
#
# last year's VADL workshop: https://vadl2017.github.io/


CURVATURE
http://members.chello.at/easyfilter/Bresenham.pdf










pos common scale
  var y, 
  var x, 
  var spot size

pos non-aligned scale
  var origin
  var y
  var x
  var spot size

length
  var length
  var x 
  var y
  var width

direction
  var direction
  var x 
  var y

angle
  var angle
  var x
  var y

area
  var area
  var x, 
  var y

volume
  var volume
  var x
  var y

curvature
  var curvature
  var x
  var y

shading
  var percent_complete
  var x
  var y

  

(12 * 9 * 4 + 4 * 9 * 2) * 5


#
# VIDEO


1. Evaluating Graphical Perception with CNNs

2. Which chart would you choose?

3. Cleveland and McGill studied this in 1984.

4. But today, not only humans perceive information. CNNs are used for many computer vision tasks.
   Can they perceive visual encodings?

5. We study how CNNs perceive low-level visuals such as lengths, curvature and position

6. We also replicate Cleveland and McGills position-angle experiment

7. .. the position-length experiment

8. .. and the bars and framed rectangles experiment

9. for this, we train over 2500 different networks

10. and report successes, failures, and insights for computing graphical perception with CNNs

11. all our code, data and results are freely available.    Thank you very much!



# Which chart would you choose? Probably the bar chart. 
#
# The difference was first studied by Cleveland and McGill.
#
# Nowadays not only us as humans perceive the world: many computer vision tasks are handled by CNNs
#
# Would CNNs prefer the bar chart?
#
# We tested multiple classifiers by replicating the experiments of cleveland mcgill:
#
# elementary perceptual tasks such as the measurement of curvature, angles, length and positions
#
# and the position-angle experiment which compares pie and bar charts
#
# as well as the position-length experiment 
#
# and the bar framed rectangles experiment built upon webers law of just noticable differences
#

# for this, we trained over 2500 different networks

# we report successes, failure, and insights for computing graphical perception using CNNs

# all our code, data, and results are publicly available





#
# Which chart would you choose? Hopefully the bar chart!
#
# This was first studied by Cleveland and McGill.
#
# But now, not only humans see the world. In fact many computer vision tasks are handled by CNNs.
#
# Can CNNs measure curvature and angles? Can they judge lengths or find positions on a common scale?
#
# Would they prefer bar charts over pie charts?
#
# please come to our talk to find out
#



####
# POS LENGTH
#
# TYPE 1 
    1.4 +- 0.14
  TYPE 2
    1.72 +- 0.2
  TYPE 3
    1.84 +- 0.16  
  TYPE 4
    2.35 +- 0.175
  TYPE 5
    2.72 +- 0.155
#
# POS ANGLE
#
# Type 1
    1.035 +- 0.125
  Type 2
    2.05 +- 0.115