cleanup + readme update

Author: chrischoy

.style.yapf

@@ -0,0 +1,4 @@
+[style]
+based_on_style = chromium
+column_limit = 100
+indent_width = 4

README.md

@@ -2,146 +2,93 @@
 
 This is the source code for the paper `3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction, ECCV 2016`. Given one or multiple views of an object, the network generate voxelized (voxel is 3D equivalent of pixel) reconstruction of the object in 3D.
 
+## Citing this work
 
-## Overview
-
-Given a set of images
-
-- [ShapeNet rendering](http://cvgl.stanford.edu/data2/ShapeNetRendering.tgz)
-- [ShapeNet voxelized models](http://cvgl.stanford.edu/data2/ShapeNetVox32.tgz)
-
-## Installation
-
-1. Download `ShapeNetCore` dataset
-2. Set the `MODEL_ROOT_PATH` in `lib/config.py` to the extracted `ShapeNetCore`
-3. Download model lists from the website. Some models do not have any faces.
-4. Generate dataset `json` file by running `python tools/gen_category_list.py`
-5. Voxelize all the models by running `python tools/voxelize_models.py`
-6. Render all the models by runnning `python tools/render_models.py`. To run this step, you have to setup `blender`.
-7. Set `cfg.DIR.MODEL_PATH`, `cfg.DIR.RENDERING_PATH` and `cfg.DIR.VOXEL_PATH` in `lib/config.py` accoringly
-8. Run experiments `bash ./experiments/script/mv_lstm_vec_net.sh`
-
-## Installation
-
-### CUDA Setup
-
-Follow the [instruction](http://deeplearning.net/software/theano/install.html) and set GPU + CUDA.
-
-1. `CUDA_ROOT=/path/to/cuda/root`
-2. add a cuda.root flag to THEANO_FLAGS, as in `THEANO_FLAGS='cuda.root=/path/to/cuda/root'`
-3. add a [cuda] section to your .theanorc file containing the option root = /path/to/cuda/root.
-
-Download `cuDNN3` and follow the [instruction](http://deeplearning.net/software/theano/library/sandbox/cuda/dnn.html)
-
-A non-intrusive installation is to set
-
-```
-export LD_LIBRARY_PATH=/home/user/path_to_CUDNN_folder/lib64:$LD_LIBRARY_PATH
-export CPATH=/home/user/path_to_CUDNN_folder/include:$CPATH
-export LIBRARY_PATH=/home/user/path_to_CUDNN_folder/lib64:$LD_LIBRARY_PATH
-```
-
-### Theano
-
-Install bleeding-edge Theano.
+If you find this work useful in your research, please consider citing:
 
 ```
-pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
-```
-
-#### Theanorc Setup
-
-```
-[global]
-floatX = float32
-base_compiledir = /var/tmp
-
-# Get an error message if cuDNN is not available
-optimizer_including = cudnn
+@article{choy20163d,
+  title={3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction},
+  author={Choy, Christopher B and Xu, Danfei and Gwak, JunYoung and Chen, Kevin and Savarese, Silvio},
+  journal={arXiv preprint arXiv:1604.00449},
+  year={2016}
+}
+
+@inproceedings{choy_eccv16,
+  author={Choy, Christopher B and Xu, Danfei and Gwak, JunYoung and Chen, Kevin and Savarese, Silvio},
+  title={3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction},
+  booktitle={European Conference on Computer Vision (ECCV)},
+  year={2016}
+}
 ```
 
-#### Issues
-
-- Upgrading from cuDNNv2
-
-    If you encounter
-
-    ```
-    ImportError: libcudnn.so.6.5: cannot open shared object file: No such file or directory
-    ```
-
-    Delete all binary files in the `base_compiledir` [link](http://deeplearning.net/software/theano/library/config.html#config.base_compiledir)
-
-
-### Blender
-
-Go to the blender website and download the [latest blender](https://developer.blender.org/diffusion/B/)
-
-```
-# read-only access
-git clone git://git.blender.org/blender.git
+## Overview
 
-# read/write access
-git clone git@git.blender.org:blender.git
+![Overview](imgs/overview.png)
+*Left: images found on Ebay, Amazon, Right: overview of `3D-R2N2`*
 
-cd blender
-git submodule update --init --recursive
-git submodule foreach git checkout master
-git submodule foreach git pull --rebase origin master
-```
+Traditionally, single view reconstruction and multi view reconstruction are disjoint problmes that has been dealt using different approaches. In this work, we first propose a unified framework for both single and multi view reconstruction using a `3D Recurrent Reconstruction Neural Network` (3D-R2N2).
 
-Then, follow the instruction on compiling the blender with the python module support [http://wiki.blender.org/index.php/User:Ideasman42/BlenderAsPyModule]
+| Schematics of `3D-Convolutional LSTM` | Inputs for each cell      |
+|:-------------------------------------:|:-------------------------:|
+| ![3D-LSTM](imgs/lstm_time.png)        | ![3D-LSTM](imgs/lstm.png) |
 
-Make sure to set the following flags correctly.
+We can feed images in a random order but the network is invariant to the order. The ciritical component that enables the network to be invariant to the order is the `3D-Convolutional LSTM` which we first proposed in this work. The `3D-Convolutional LSTM` selectively updates parts that are visible and keeps the parts that are self occluded (please refer to [http://cvgl.stanford.edu/3d-r2n2/](http://cvgl.stanford.edu/3d-r2n2/) for the supplementary material for analysis).
 
-```
-WITH_PYTHON_INSTALL=OFF
-WITH_PLAYER=OFF
-WITH_PYTHON_MODULE=ON
-...
-WITH_OPENCOLLADA=ON
-```
+## Datasets
 
-OpenCollada is optional if you wan to use collada (.dae) files. Install OpenCollada first.
+We used [ShapeNet](http://shapenet.cs.stanford.edu) models to generated rendered images and voxelized models which are available below (you can follow the installation instruction below to extract it on the default directory).
 
+- ShapeNet rendered images [ftp://cs.stanford.edu/cs/cvgl/ShapeNetRendering.tgz](ftp://cs.stanford.edu/cs/cvgl/ShapeNetRendering.tgz)
+- ShapeNet voxelized models [ftp://cs.stanford.edu/cs/cvgl/ShapeNetVox32.tgz](ftp://cs.stanford.edu/cs/cvgl/ShapeNetVox32.tgz)
 
-### OpenCollada
+## Installation
 
-This will requires gcc4.7
+The package requires python3. You can follow the direction below to install virtual environment within the repository or install anaconda for python 3.
+
+- Download the repository
+  ```
+  git clone https://github.com/chrischoy/3D-R2N2.git
+  ```
+- Setup virtual environment and install requirements
+  ```
+  cd 3D-R2N2
+  pip install virtualenv
+  virtualenv -p python3 py3
+  source py3/bin/activate
+  pip install -r requirements.txt
+  ```
+- Download trained network weight
+  ```
+  wget asdfasdf
+  ```
+- run the demo code
+  ```
+  python demo.py
+  ```
+
+### Training the network
+
+- Download datasets and place them in a folder named `ShapeNet`
+  ```
+  mkdir ShapeNet/
+  wget ftp://cs.stanford.edu/cs/cvgl/ShapeNetRendering.tgz
+  wget ftp://cs.stanford.edu/cs/cvgl/ShapeNetVox32.tgz
+  tar -xzf ShapeNetRendering.tgz -C ShapeNet/
+  tar -xzf ShapeNetVox32.tgz -C ShapeNet/
+  ```
+- Run experiments `bash ./experiments/script/mv_lstm_vec_net.sh`
+
+### Miscellaneous setup
+
+#### CUDA Setup
 
-```
-git clone https://github.com/KhronosGroup/OpenCOLLADA
-mkdir OpenCOLLADA-build
-cd OpenCOLLADA-build
-cmake ../OpenCOLLADA -DUSE_SHARED=ON
-make
-make install
-```
+Follow the [instruction](http://deeplearning.net/software/theano/install.html) and set GPU + CUDA.
 
-## Python 3 Requirements
+#### Theano
 
-On the root path, run
+Install bleeding-edge Theano using
 
 ```
-pip3 install -r requirements.txt
+pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
 ```
-
-
-# Dataset Setting
-
-- Download ShapeNet
-- Set variables in lib/config.py
-- Generate training dataset lists
-    ```
-    python tools/generate_category_list.py
-    ```
-- Generate rendering and voxelization
-- Run the training script
-
-
-# Erroneous Files
-
-03624134/67ada28ebc79cc75a056f196c127ed77/model.obj
-04090263/4a32519f44dc84aabafe26e2eb69ebf4/model.obj
-04074963/b65b590a565fa2547e1c85c5c15da7fb/model.obj
-

_init_paths.py

@@ -0,0 +1,6 @@
+# Erroneous Files
+
+03624134/67ada28ebc79cc75a056f196c127ed77/model.obj
+04090263/4a32519f44dc84aabafe26e2eb69ebf4/model.obj
+04074963/b65b590a565fa2547e1c85c5c15da7fb/model.obj
+

demo.py

@@ -0,0 +1,54 @@
+{
+    "04256520": {
+        "id": "04256520",
+        "name": "sofa,couch,lounge"
+    },
+    "02691156": {
+        "id": "02691156",
+        "name": "airplane,aeroplane,plane"
+    },
+    "03636649": {
+        "id": "03636649",
+        "name": "lamp"
+    },
+    "04401088": {
+        "id": "04401088",
+        "name": "telephone,phone,telephone set"
+    },
+    "04530566": {
+        "id": "04530566",
+        "name": "vessel,watercraft"
+    },
+    "03691459": {
+        "id": "03691459",
+        "name": "loudspeaker,speaker,speaker unit,loudspeaker system,speaker system"
+    },
+    "03001627": {
+        "id": "03001627",
+        "name": "chair"
+    },
+    "02933112": {
+        "id": "02933112",
+        "name": "cabinet"
+    },
+    "04379243": {
+        "id": "04379243",
+        "name": "table"
+    },
+    "03211117": {
+        "id": "03211117",
+        "name": "display,video display"
+    },
+    "02958343": {
+        "id": "02958343",
+        "name": "car,auto,automobile,machine,motorcar"
+    },
+    "02828884": {
+        "id": "02828884",
+        "name": "bench"
+    },
+    "04090263": {
+        "id": "04090263",
+        "name": "rifle"
+    }
+}

experiments/cfgs/local_shapenet.yaml

@@ -21,22 +21,13 @@ export THEANO_FLAGS="floatX=float32,device=gpu,assert_no_cpu_op='raise'"
 python main.py \
       --batch-size 24 \
       --iter 60000 \
-      --cfg ./experiments/cfgs/shapenet_1000.yaml \
-      --cfg ./experiments/cfgs/random_crop.yaml \
-      --cfg ./experiments/cfgs/no_random_background.yaml \
-      --cfg ./experiments/cfgs/max_5_views.yaml \
-      --cfg ./experiments/cfgs/local_shapenet.yaml \
       --out $OUT_PATH \
       --model $NET_NAME \
       ${*:1}
 
 python main.py \
       --test \
-      --batch-size 24 \
-      --cfg ./experiments/cfgs/shapenet_1000.yaml \
-      --cfg ./experiments/cfgs/no_random_background.yaml \
-      --cfg ./experiments/cfgs/max_5_views.yaml \
-      --cfg ./experiments/cfgs/local_shapenet.yaml \
+      --batch-size 1 \
       --out $OUT_PATH \
       --weights $OUT_PATH/weights.npy \
       --model $NET_NAME \