From 2b5b12f440cf61195373ea33f2641668ce25f878 Mon Sep 17 00:00:00 2001
From: Vidar Tonaas Fauske <vidartf@gmail.com>
Date: Tue, 13 Nov 2018 17:43:46 +0100
Subject: [PATCH 1/4] Add a watch script

---
 js/package.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/js/package.json b/js/package.json
index dd0bcf7d..1855239c 100644
--- a/js/package.json
+++ b/js/package.json
@@ -20,7 +20,8 @@
     "build:labextension": "rimraf lab-dist && mkdirp lab-dist && cd lab-dist && npm pack ..",
     "build:all": "npm run build:labextension",
     "prepare": "npm run autogen",
-    "prepack": "npm run build:bundles-prod"
+    "prepack": "npm run build:bundles-prod",
+    "watch": "webpack -d -w"
   },
   "devDependencies": {
     "eslint": "^5.6.0",

From 25aa3b27b0b461cefd5832ab5e749ab7da48bbce Mon Sep 17 00:00:00 2001
From: Vidar Tonaas Fauske <vidartf@gmail.com>
Date: Tue, 13 Nov 2018 17:43:32 +0100
Subject: [PATCH 2/4] Add support for ipywebrtc

A lot of improvements in PRs to ipywebrtc will improve the experience here as well.
---
 examples/Capture.ipynb     | 294 +++++++++++++++++++++++++++++++++++++
 js/src/_base/Renderable.js |  79 ++++++++++
 2 files changed, 373 insertions(+)
 create mode 100644 examples/Capture.ipynb

diff --git a/examples/Capture.ipynb b/examples/Capture.ipynb
new file mode 100644
index 00000000..a00cdf99
--- /dev/null
+++ b/examples/Capture.ipynb
@@ -0,0 +1,294 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Capture outputs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This notebook will demonstrate how to capture still frames or videos from pythreejs using [ipywebrtc](https://ipywebrtc.readthedocs.io/en/latest/)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup an example renderer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pythreejs import *\n",
+    "import ipywebrtc\n",
+    "from ipywidgets import Output, VBox"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "view_width = 600\n",
+    "view_height = 400\n",
+    "\n",
+    "sphere = Mesh(\n",
+    "    SphereBufferGeometry(1, 32, 16),\n",
+    "    MeshStandardMaterial(color='red')\n",
+    ")\n",
+    "\n",
+    "cube = Mesh(\n",
+    "    BoxBufferGeometry(1, 1, 1),\n",
+    "    MeshPhysicalMaterial(color='green'),\n",
+    "    position=[2, 0, 4]\n",
+    ")\n",
+    "\n",
+    "camera = PerspectiveCamera( position=[10, 6, 10], aspect=view_width/view_height)\n",
+    "key_light = DirectionalLight(position=[0, 10, 10])\n",
+    "ambient_light = AmbientLight()\n",
+    "\n",
+    "scene = Scene(children=[sphere, cube, camera, key_light, ambient_light])\n",
+    "controller = OrbitControls(controlling=camera)\n",
+    "renderer = Renderer(camera=camera, scene=scene, controls=[controller],\n",
+    "                    width=view_width, height=view_height)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "renderer"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Capture renderer output to stream"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "stream = ipywebrtc.WidgetStream(widget=renderer, max_fps=30)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "If you want, you can preview the content of the stream with a video-viewer. This should simply mirror what you see in the renderer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [],
+   "source": [
+    "stream"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Capturing images\n",
+    "\n",
+    "To capture images from the stream, use the `ImageRecorder` widget from `ipywebrtc`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "recorder = ipywebrtc.ImageRecorder(filename='snapshot', format='png', stream=stream)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "There are two ways to capture images from the stream:\n",
+    "1. Manually from the browser by using the widget view of the recorder.\n",
+    "2. Programmatically using the .save()/download() method on the recorder."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Using the view"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "recorder"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here,clicking the \"Snapshot\" button will capture a new frame and sync it back to the kernel side. Clicking \"Download\" will download the current snapshot on the *client side*. When taking a snapshot, the image will also be synced to the *kernel side*. If the image has changed, any observers of the value trait of the image will trigger (i.e. `recorder.image.observe(callback, 'value')`):"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "out = Output()  # To capture print output\n",
+    "\n",
+    "@out.capture()\n",
+    "def on_capture(change):\n",
+    "    print('Captured image changed!')\n",
+    "recorder.image.observe(on_capture, 'value')\n",
+    "out"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Using kernel API:\n",
+    "\n",
+    "To request a snapshot from the kernel, set the `recording` attribute of the recorder to `True`. This will update the `image` attribute asynchronously. The easiest way to save this to the kernel side is to also set the `filename` attribute, and set `autosave` to `True`. This will cause the image to be saved as soon as it is available. This is equivalend to observing the image widget's `value` trait, and calling the `save()` method when the image changes."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "recorder.autosave = True\n",
+    "recorder.recording = True"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can also trigger a client-side download from the kernel by calling the `download()` method on the recorder:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "recorder.download()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Capturing video\n",
+    "\n",
+    "To capture a video from the stream, use the `VideoRecorder` from `ipywebrtc`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "video_recorder = ipywebrtc.VideoRecorder(stream=stream, filename='video', codecs='vp8')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "video_recorder"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here, clicking the \"Record\" button will start capturing the video. Once you click the \"Stop\" button (appears after clicking \"Record\"), the video will be displayed in the view, and it will be synced to the kernel. If the video has changed, any observers of the value trait of the video will trigger, similarly to that of the `ImageRecorder`. Clicking \"Download\" will download the current video on the client side.\n",
+    "\n",
+    "The kernel side API for the `VideoRecorder` is similar to that of the `ImageRecorder`, but you will also have to tell it when to stop:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "video_recorder.autosave = True\n",
+    "video_recorder.recording = True\n",
+    "# After executing this, try to interact with the renderer above before executing the next cell"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "video_recorder.recording = False\n",
+    "video_recorder.download()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/js/src/_base/Renderable.js b/js/src/_base/Renderable.js
index 2c108bee..01ca9d8c 100644
--- a/js/src/_base/Renderable.js
+++ b/js/src/_base/Renderable.js
@@ -1,6 +1,7 @@
 var _ = require('underscore');
 var widgets = require('@jupyter-widgets/base');
 var $ = require('jquery');
+var Promise = require('bluebird');
 
 var pkgName = require('../../package.json').name;
 var EXTENSION_SPEC_VERSION = require('../version').EXTENSION_SPEC_VERSION;
@@ -90,6 +91,83 @@ var RenderableModel = widgets.DOMWidgetModel.extend({
         this.trigger('childchange', this);
     },
 
+    /**
+     * Find a view, preferrably a live one
+     */
+    _findView: function() {
+        var viewPromises = Object.keys(this.views).map(function(key) {
+            return this.views[key];
+        }, this);
+        return Promise.all(viewPromises).then(function(views) {
+            for (var i=0; i<views.length; ++i) {
+                var view = views[i];
+                if (!view.isFrozen) {
+                    return view;
+                }
+            }
+            return views[0];
+        });
+    },
+
+    /**
+     * Interface for jupyter-webrtc.
+     */
+    captureStream: function(fps) {
+        var stream = new MediaStream();
+
+        var that = this;
+        var canvasStream = null;
+
+        function updateStream() {
+            return that._findView().then(function(view) {
+                if (canvasStream !== null) {
+                    // Stop and remove tracks from previous canvas
+                    stream.getTracks().forEach(function(track) {
+                        track.stop();
+                        stream.removeTrack(track);
+                        canvasStream.removeTrack(track);
+                    });
+                    canvasStream = null;
+                }
+                var canvas;
+                if (view.isFrozen) {
+                    canvas = document.createElement('canvas');
+                    canvas.width = view.$frozenRenderer.width();
+                    canvas.height = view.$frozenRenderer.height();
+                    var ctx = canvas.getContext('2d');
+                    ctx.drawImage(view.$frozenRenderer[0], 0, 0);
+                } else {
+                    canvas = view.renderer.domElement;
+                }
+                // Add tracks from canvas to stream
+                canvasStream = canvas.captureStream(fps);
+                canvasStream.getTracks().forEach(function(track) {
+                    stream.addTrack(track);
+                    if (track.requestFrame) {
+                        (function() {
+                            var orig = track.requestFrame.bind(track);
+                            track.requestFrame = function() {
+                                orig();
+                                // Ensure we redraw to make stream pickup first frame on Chrome
+                                // https://bugs.chromium.org/p/chromium/issues/detail?id=903832
+                                view.tick();
+                            };
+                            track.requestFrame();
+
+                        }());
+                    }
+                });
+
+                // If renderer status changes, update stream
+                that.listenToOnce(view, 'updatestream', updateStream);
+            });
+        }
+
+        return updateStream().then(function() {
+            return stream;
+        });
+    },
+
 }, {
     serializers: _.extend({
         clippingPlanes: { deserialize: unpackThreeModel },
@@ -199,6 +277,7 @@ var RenderableView = widgets.DOMWidgetView.extend({
         } else {
             this.renderer.setSize(width, height);
         }
+        this.trigger('updatestream');
     },
 
     updateProperties: function(force) {

From 2ef45830a6694d046c3add97b051440253b225e8 Mon Sep 17 00:00:00 2001
From: Vidar Tonaas Fauske <vidartf@gmail.com>
Date: Mon, 19 Nov 2018 15:28:15 +0100
Subject: [PATCH 3/4] Add ipywebrtc as examples dep

---
 setup.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/setup.py b/setup.py
index 31bed8b8..bdd3dddd 100644
--- a/setup.py
+++ b/setup.py
@@ -80,7 +80,8 @@
         'examples': [
             'scipy',
             'matplotlib',
-            'scikit-image'
+            'scikit-image',
+            'ipywebrtc',
         ],
         'docs': [
             'sphinx>=1.5',

From 646ad19ec33eaa020778d6068dd717dbc9052959 Mon Sep 17 00:00:00 2001
From: Vidar Tonaas Fauske <vidartf@gmail.com>
Date: Mon, 19 Nov 2018 16:07:32 +0100
Subject: [PATCH 4/4] Cleanup

---
 examples/Capture.ipynb | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/examples/Capture.ipynb b/examples/Capture.ipynb
index a00cdf99..e5cb8d15 100644
--- a/examples/Capture.ipynb
+++ b/examples/Capture.ipynb
@@ -4,13 +4,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Capture outputs"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
+    "# Capture outputs\n",
+    "\n",
     "This notebook will demonstrate how to capture still frames or videos from pythreejs using [ipywebrtc](https://ipywebrtc.readthedocs.io/en/latest/)."
    ]
   },