This is an attempt at predicting the dynamics of interacting objects with a deep network.
I wrote a simple 2d physics engine in C++ that simulates moment of inertia, fluid frictions, and elastic collisions, and a residual network in Lua/Torch that predicts the final configuration of a set of rectangles, given a starting configuration and the location where a force is applied.
Results and analysis are available in Fleuret (2016), and you can have a look at a 2min video.
This package is composed of a simple 2d physics simulator called ‘flatland’ written in C++, to generate the data-set, and a deep residual network ‘dyncnn’ written in the Lua/Torch7 framework.
You can run the reference experiment by executing the run.sh shell script.
It will
Generate the data-set of 40k triplets of images,
Train the deep network, and output validation results every 100 epochs. This takes ~30h on a GTX 1080 with cuda 8.0, cudnn 5.1, and recent torch.
Generate two pictures of the internal activations.
Generate a graph with the loss curves if gnuplot is installed.