diff --git a/README.md b/README.md index 34f5c4e..a069a8f 100755 --- a/README.md +++ b/README.md @@ -1,10 +1,12 @@ # TinyML Application Development for Everyone -AIoT Dev Summit Logo +AIoT Devfest Logo Arduino Nano 33 BLE board -## Hands-on workshop at Arm AIoT Dev Summit December 2019 +## Hands-on workshop at AIoT Devfest January 2020 -In this tutorial we will teach a board to recognise gestures! We'll capture motion data from the [Arduino Nano 33 BLE Sense](https://store.arduino.cc/arduino-nano-33-ble-sense) board, import it into TensorFlow to train a model, and deploy a classifier onto the board using [TensorFlow Lite for microcontrollers](https://www.tensorflow.org/lite/microcontrollers/overview). +In this workshop we will teach an Arduino board to recognize gestures! We will capture motion data from the [Arduino Nano 33 BLE](https://store.arduino.cc/arduino-nano-33-ble) board[1](#note1), import it into TensorFlow to train a model, and deploy a classifier onto the board using [TensorFlow Lite for microcontrollers](https://www.tensorflow.org/lite/microcontrollers/overview). + +The hardware for this workshop has been provided by [Arduino](https://arduino.cc) ## Exercises @@ -20,4 +22,15 @@ In this tutorial we will teach a board to recognise gestures! We'll capture moti * [Exercise 9: Gesture Controlled USB Emoji Keyboard](exercises/exercise9.md) * [Exercise 10: Next Steps](exercises/exercise10.md) +This workshop material was developed by Sandeep Mistry and Don Coleman. + +Arduino Logo +Chariot Solutions Logo + +Previous versions + * https://github.com/sandeepmistry/aimldevfest-workshop-2019 + * https://github.com/arduino/AIoT-Dev-Summit-2019 + + +1: You can also use the [Arduino Nano 33 BLE Sense](https://store.arduino.cc/arduino-nano-33-ble-sense) for this workshop. diff --git a/aiot-logo.png b/aiot-logo.png deleted file mode 100644 index 700d51e..0000000 Binary files a/aiot-logo.png and /dev/null differ diff --git a/arduino_tinyml_workshop.ipynb b/arduino_tinyml_workshop.ipynb index bce8872..7ecf14f 100644 --- a/arduino_tinyml_workshop.ipynb +++ b/arduino_tinyml_workshop.ipynb @@ -21,14 +21,14 @@ "colab_type": "text" }, "source": [ - "\n", - "# Tiny ML on Arduino\n", + "\n", + "# TinyML on Arduino\n", "## Gesture recognition tutorial\n", " * Sandeep Mistry - Arduino\n", " * Don Coleman - Chariot Solutions\n", "\n", " \n", - "https://github.com/arduino/ArduinoTensorFlowLiteTutorials/" + "https://github.com/don/tinyml-workshop/" ] }, { @@ -510,7 +510,7 @@ "source": [ "## Encode the Model in an Arduino Header File \n", "\n", - "The next cell creates a constant byte array that contains the TFlite model. Import it as a tab with the sketch below." + "The next cell creates a constant byte array that contains the TFlite model. Import the generated model.h file as a tab into your Arduino sketch." ] }, { @@ -521,9 +521,9 @@ "colab": {} }, "source": [ - "!echo \"const unsigned char model[] = {\" > /content/model.h\n", - "!cat gesture_model.tflite | xxd -i >> /content/model.h\n", - "!echo \"};\" >> /content/model.h\n", + "!echo \"const unsigned char model[] __attribute__((aligned(4))) = {\" > /content/model.h\n", + "!cat gesture_model.tflite | xxd -i >> /content/model.h\n", + "!echo \"};\" >> /content/model.h\n", "\n", "import os\n", "model_h_size = os.path.getsize(\"model.h\")\n", diff --git a/exercises/exercise10.md b/exercises/exercise10.md index 5e58adc..3248a53 100755 --- a/exercises/exercise10.md +++ b/exercises/exercise10.md @@ -4,10 +4,10 @@ Now that you have things working, here are a few new things to try. - Add some more gestures to go with the :punch: and :flex: - 1. Load the IMU_Capture sketch on your Arduino Nano 33 BLE Sense. + 1. Load the IMU_Capture sketch on your Arduino Nano 33 BLE 1. Caputure some additional gestures for a new emoji. Perhaps a :smile:, :thumbsup:, :clap:, or :wave: 1. Load the new CSV files into your Colab model - 1. Put the trained model back on your Nano 33 BLE Sense. + 1. Put the trained model back on your Nano 33 BLE - Try increasing and decreasing the number of recordings per gesture, how does this impact performance? @@ -19,6 +19,7 @@ Now that you have things working, here are a few new things to try. - Check out the examples in the [TensorFlow Lite Getting Started Guide](https://www.tensorflow.org/lite/microcontrollers/get_started) + - Grab the [TinyML book](http://shop.oreilly.com/product/0636920254508.do) to continue at home diff --git a/exercises/exercise2.md b/exercises/exercise2.md index ad1539c..e962dad 100755 --- a/exercises/exercise2.md +++ b/exercises/exercise2.md @@ -1,10 +1,10 @@ # Exercise 2: Source Code -The [source code](https://github.com/arduino/AIoT-Dev-Summit-2019) for this workshop is available on Github in the Arduino organizaton. +The source code and documentation for this workshop available on Github. The code is in the [ArduinoSketches](../ArduinoSketches) folder. If you're familiar with git and Github, clone the repository to your computer. If that last sentence didn't make sense to you, we recommend that you download the code. -1. Go to https://github.com/arduino/AIoT-Dev-Summit-2019 +1. Go to https://github.com/don/tinyml-workshop 1. Click the green **Clone or download** button 1. Choose Download ZIP 1. Go to your downloads folder and extract all the files from the zip archive diff --git a/exercises/exercise3.md b/exercises/exercise3.md index 3a9fa83..0e24f93 100755 --- a/exercises/exercise3.md +++ b/exercises/exercise3.md @@ -1,10 +1,10 @@ # Exercise 3: Hardware -![Picture of Arduino Nano 33 BLE Sense board](images/ble-sense.jpg) +![Picture of Arduino Nano 33 BLE board](images/nano-33-ble.jpg) ## Unboxing and set up -1. Remove the Arduino Nano 33 BLE Sense board from the box +1. Remove the Arduino Nano 33 BLE board from the box 1. Plug the micro USB cable into the board and your computer 1. Open the Arduino IDE application on your computer 1. Choose the board `Tools -> Board -> Arduino Nano 33 BLE` @@ -21,7 +21,7 @@ 1. Open the serial monitor `Tools -> Serial Monitor` to see debug messages 1. If everything is working properly, the LED will start blinking quickly -__NOTE:__ The first time you upload a sketch to the Arduino Nano 33 BLE Sense, the USB port name might change. If this happens, you need to re-select the port using the `Tools -> Port` menu. +__NOTE:__ The first time you upload a sketch to the Arduino Nano 33 BLE, the USB port name might change. If this happens, you need to re-select the port using the `Tools -> Port` menu. Next [Exercise 4: Visualize the IMU Data](exercise4.md) diff --git a/exercises/exercise4.md b/exercises/exercise4.md index 8997197..beae694 100755 --- a/exercises/exercise4.md +++ b/exercises/exercise4.md @@ -2,7 +2,7 @@ The next step is to use an Arduino program you downloaded in [Exercise 2](exercise2.md) to capture motion data from the IMU. -1. Open __AIoT-Dev-Summit-2019/ArduinoSketches/IMU_Capture/IMU_Capture.ino__ in the Arduino IDE. +1. Open __tinyml-workshop/ArduinoSketches/IMU_Capture/IMU_Capture.ino__ in the Arduino IDE. 1. Compile the sketch and upload it to the board: `Sketch -> Upload` 1. Open the Serial Monitor: `Tools -> Serial Monitor` 1. Shake the board. The change in acceleration will start recording data from the IMU in the Serial Monitor diff --git a/exercises/exercise6.md b/exercises/exercise6.md index 16e0e8b..afed081 100755 --- a/exercises/exercise6.md +++ b/exercises/exercise6.md @@ -18,7 +18,7 @@ You can enable 3rd party cookies, or better yet, add an exception for `[*.]googl Open the [arduino_tinyml_workshop.ipynb](../arduino_tinyml_workshop.ipynb) notebook in Google Colab and follow the instructions in the notebook to prepare the data and train the model. -https://colab.research.google.com/github/arduino/AIoT-Dev-Summit-2019/blob/master/arduino_tinyml_workshop.ipynb +https://colab.research.google.com/github/don/tinyml-workshop/blob/master/arduino_tinyml_workshop.ipynb Next [Exercise 7: Classifying IMU Data](exercise7.md) diff --git a/exercises/exercise7.md b/exercises/exercise7.md index 7c5e15e..4ab8704 100755 --- a/exercises/exercise7.md +++ b/exercises/exercise7.md @@ -1,6 +1,6 @@ # Exercise 7: Classifying IMU Data -1. Open __AIoT-Dev-Summit-2019/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE. +1. Open __tinyml-workshop/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE. 1. Switch to the model.h tab 1. Replace the contents of model.h with the version you downloaded from Colab 1. Upload the sketch: `Sketch -> Upload` diff --git a/exercises/exercise8.md b/exercises/exercise8.md index 406a68c..455b38d 100755 --- a/exercises/exercise8.md +++ b/exercises/exercise8.md @@ -4,7 +4,7 @@ Now that we the code can recognize gestures, let's try printing out some emojis in addition to the text. -Open __AIoT-Dev-Summit-2019/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE if it's not already loaded. +Open __tinyml-workshop/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE if it's not already loaded. Create a new char* array named `EMOJIS`. Inside the array defines the emojis as a UTF-8 encoded unicode strings. This array must be the same length at the `GESTURE` array. The order of the emojis in the array must match the order of the gestures. diff --git a/exercises/exercise9.md b/exercises/exercise9.md index 5fa3f72..bb7ae31 100644 --- a/exercises/exercise9.md +++ b/exercises/exercise9.md @@ -33,6 +33,6 @@ Now you are ready to build a keyboard. Create a new sketch in the Arduino IDE. C ### Note -__NOTE:__ once you load code that runs the USB Keyboard, the Arduino IDE might not be able to see the serial port when you want to load new code. Double click the reset button on the Nano 33 BLE Sense before you run __Sketch -> Upload__. +__NOTE:__ once you load code that runs the USB Keyboard, the Arduino IDE might not be able to see the serial port when you want to load new code. Double click the reset button on the Nano 33 BLE before you run __Sketch -> Upload__. Next [Exercise 10: Next Steps](exercise10.md) \ No newline at end of file diff --git a/exercises/images/AI-IOT-devfest-AZ-2020-horiz.png b/exercises/images/AI-IOT-devfest-AZ-2020-horiz.png new file mode 100644 index 0000000..79f5e4d Binary files /dev/null and b/exercises/images/AI-IOT-devfest-AZ-2020-horiz.png differ diff --git a/exercises/images/ChariotSolutions.png b/exercises/images/ChariotSolutions.png new file mode 100644 index 0000000..71f97d9 Binary files /dev/null and b/exercises/images/ChariotSolutions.png differ diff --git a/exercises/images/clone-or-download.png b/exercises/images/clone-or-download.png index 667f661..d564153 100644 Binary files a/exercises/images/clone-or-download.png and b/exercises/images/clone-or-download.png differ diff --git a/exercises/images/nano-33-ble.jpg b/exercises/images/nano-33-ble.jpg new file mode 100644 index 0000000..7d40cdb Binary files /dev/null and b/exercises/images/nano-33-ble.jpg differ diff --git a/exercises/images/nano-33-ble_iso.jpg b/exercises/images/nano-33-ble_iso.jpg new file mode 100644 index 0000000..a38408f Binary files /dev/null and b/exercises/images/nano-33-ble_iso.jpg differ diff --git a/exercises/intro.md b/exercises/intro.md index 95cfd16..2bde62c 100644 --- a/exercises/intro.md +++ b/exercises/intro.md @@ -80,7 +80,7 @@ What you know -> | Supervised Learning | -> What you want to know Running ML models on microcontrollers. Low power, below 1 mW. -## Arduino Nano 33 BLE Sense board +## Arduino Nano 33 BLE board Based on the Nordic nRF52840 * Arm Cortex-M4F running at 64 MHz @@ -90,8 +90,8 @@ Based on the Nordic nRF52840 Onboard Sensors * IMU (measure motion: acceleration, gyro) -* Temperature, pressure, humidity, light, color -* PDM microphone + +This workshop also works on the Arduino Nano 33 BLE Sense board. The Sense version of the board adds additional sensors: temperature, pressure, humidity, light, color, PDM microphone. ## What is an IMU? @@ -105,7 +105,6 @@ This board uses the ST Micro LSM9DS1. We're only using the accelerometer and gyroscope for this project. - ## Workshop * Record IMU data @@ -124,7 +123,6 @@ We're only using the accelerometer and gyroscope for this project. ## IMU input data - The Arduino library enables the sensor to report 119 data points every second, this means a new set of data is received every 8.4ms