Skip to content

test #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# TinyML Application Development for Everyone

<a href="https://armsummit.bemyapp.com"><img src="aiot-logo.png" alt="AIoT Dev Summit Logo"></a>
<a href="https://aiotdevfest.org"><img src="exercises/images/AI-IOT-devfest-AZ-2020-horiz.png" alt="AIoT Devfest Logo" width="40%" style="padding-right:100px"></a>&nbsp;<a href="https://store.arduino.cc/arduino-nano-33-ble"><img src="exercises/images/nano-33-ble_iso.jpg" alt="Arduino Nano 33 BLE board" width="40%"></a>

## Hands-on workshop at Arm AIoT Dev Summit December 2019
## Hands-on workshop at AIoT Devfest January 2020

In this tutorial we will teach a board to recognise gestures! We'll capture motion data from the [Arduino Nano 33 BLE Sense](https://store.arduino.cc/arduino-nano-33-ble-sense) board, import it into TensorFlow to train a model, and deploy a classifier onto the board using [TensorFlow Lite for microcontrollers](https://www.tensorflow.org/lite/microcontrollers/overview).
In this workshop we will teach an Arduino board to recognize gestures! We will capture motion data from the [Arduino Nano 33 BLE](https://store.arduino.cc/arduino-nano-33-ble) board<sup>[1](#note1)</sup>, import it into TensorFlow to train a model, and deploy a classifier onto the board using [TensorFlow Lite for microcontrollers](https://www.tensorflow.org/lite/microcontrollers/overview).

The hardware for this workshop has been provided by [Arduino](https://arduino.cc)

## Exercises

Expand All @@ -20,4 +22,15 @@ In this tutorial we will teach a board to recognise gestures! We'll capture moti
* [Exercise 9: Gesture Controlled USB Emoji Keyboard](exercises/exercise9.md)
* [Exercise 10: Next Steps](exercises/exercise10.md)

This workshop material was developed by Sandeep Mistry and Don Coleman.

<a href="https://arduino.cc"><img src="exercises/images/Arduino_logo_R_highquality.png" alt="Arduino Logo" width=30%><a/>
<a href="https://chariotsolutions.com"><img src="exercises/images/ChariotSolutions.png" alt="Chariot Solutions Logo" width=30%></a>

Previous versions
* https://github.com/sandeepmistry/aimldevfest-workshop-2019
* https://github.com/arduino/AIoT-Dev-Summit-2019

<!-- TODO link to SLIDES -->

<a id="note1">1</a>: You can also use the [Arduino Nano 33 BLE Sense](https://store.arduino.cc/arduino-nano-33-ble-sense) for this workshop.
Binary file removed aiot-logo.png
Binary file not shown.
14 changes: 7 additions & 7 deletions arduino_tinyml_workshop.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@
"colab_type": "text"
},
"source": [
"<a href=\"https://www.arduino.cc/\"><img src=\"https://raw.githubusercontent.com/sandeepmistry/aimldevfest-workshop-2019/master/images/Arduino_logo_R_highquality.png\" width=200/></a>\n",
"# Tiny ML on Arduino\n",
"<a href=\"https://www.arduino.cc/\"><img src=\"https://raw.githubusercontent.com/don/tinyml-workshop/master/exercises/images/AI-IOT-devfest-AZ-2020-horiz.png\" width=200/></a>\n",
"# TinyML on Arduino\n",
"## Gesture recognition tutorial\n",
" * Sandeep Mistry - Arduino\n",
" * Don Coleman - Chariot Solutions\n",
"\n",
" \n",
"https://github.com/arduino/ArduinoTensorFlowLiteTutorials/"
"https://github.com/don/tinyml-workshop/"
]
},
{
Expand Down Expand Up @@ -510,7 +510,7 @@
"source": [
"## Encode the Model in an Arduino Header File \n",
"\n",
"The next cell creates a constant byte array that contains the TFlite model. Import it as a tab with the sketch below."
"The next cell creates a constant byte array that contains the TFlite model. Import the generated model.h file as a tab into your Arduino sketch."
]
},
{
Expand All @@ -521,9 +521,9 @@
"colab": {}
},
"source": [
"!echo \"const unsigned char model[] = {\" > /content/model.h\n",
"!cat gesture_model.tflite | xxd -i >> /content/model.h\n",
"!echo \"};\" >> /content/model.h\n",
"!echo \"const unsigned char model[] __attribute__((aligned(4))) = {\" > /content/model.h\n",
"!cat gesture_model.tflite | xxd -i >> /content/model.h\n",
"!echo \"};\" >> /content/model.h\n",
"\n",
"import os\n",
"model_h_size = os.path.getsize(\"model.h\")\n",
Expand Down
7 changes: 4 additions & 3 deletions exercises/exercise10.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ Now that you have things working, here are a few new things to try.

- Add some more gestures to go with the :punch: and :flex:

1. Load the IMU_Capture sketch on your Arduino Nano 33 BLE Sense.
1. Load the IMU_Capture sketch on your Arduino Nano 33 BLE
1. Caputure some additional gestures for a new emoji. Perhaps a :smile:, :thumbsup:, :clap:, or :wave:
1. Load the new CSV files into your Colab model
1. Put the trained model back on your Nano 33 BLE Sense.
1. Put the trained model back on your Nano 33 BLE

- Try increasing and decreasing the number of recordings per gesture, how does this impact performance?

Expand All @@ -19,6 +19,7 @@ Now that you have things working, here are a few new things to try.

- Check out the examples in the [TensorFlow Lite Getting Started Guide](https://www.tensorflow.org/lite/microcontrollers/get_started)

<!--
- Try the [Arudino Fruit to Emoji](https://github.com/arduino/ArduinoTensorFlowLiteTutorials/tree/master/FruitToEmoji) Tutorial

-->
- Grab the [TinyML book](http://shop.oreilly.com/product/0636920254508.do) to continue at home
4 changes: 2 additions & 2 deletions exercises/exercise2.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Exercise 2: Source Code

The [source code](https://github.com/arduino/AIoT-Dev-Summit-2019) for this workshop is available on Github in the Arduino organizaton.
The source code and documentation for this workshop available on Github. The code is in the [ArduinoSketches](../ArduinoSketches) folder.

If you're familiar with git and Github, clone the repository to your computer. If that last sentence didn't make sense to you, we recommend that you download the code.

1. Go to https://github.com/arduino/AIoT-Dev-Summit-2019
1. Go to https://github.com/don/tinyml-workshop
1. Click the green **Clone or download** button
1. Choose Download ZIP
1. Go to your downloads folder and extract all the files from the zip archive
Expand Down
6 changes: 3 additions & 3 deletions exercises/exercise3.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Exercise 3: Hardware

![Picture of Arduino Nano 33 BLE Sense board](images/ble-sense.jpg)
![Picture of Arduino Nano 33 BLE board](images/nano-33-ble.jpg)

## Unboxing and set up

1. Remove the Arduino Nano 33 BLE Sense board from the box
1. Remove the Arduino Nano 33 BLE board from the box
1. Plug the micro USB cable into the board and your computer
1. Open the Arduino IDE application on your computer
1. Choose the board `Tools -> Board -> Arduino Nano 33 BLE`
Expand All @@ -21,7 +21,7 @@
1. Open the serial monitor `Tools -> Serial Monitor` to see debug messages
1. If everything is working properly, the LED will start blinking quickly

__NOTE:__ The first time you upload a sketch to the Arduino Nano 33 BLE Sense, the USB port name might change. If this happens, you need to re-select the port using the `Tools -> Port` menu.
__NOTE:__ The first time you upload a sketch to the Arduino Nano 33 BLE, the USB port name might change. If this happens, you need to re-select the port using the `Tools -> Port` menu.

Next [Exercise 4: Visualize the IMU Data](exercise4.md)

Expand Down
2 changes: 1 addition & 1 deletion exercises/exercise4.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The next step is to use an Arduino program you downloaded in [Exercise 2](exercise2.md) to capture motion data from the IMU.

1. Open __AIoT-Dev-Summit-2019/ArduinoSketches/IMU_Capture/IMU_Capture.ino__ in the Arduino IDE.
1. Open __tinyml-workshop/ArduinoSketches/IMU_Capture/IMU_Capture.ino__ in the Arduino IDE.
1. Compile the sketch and upload it to the board: `Sketch -> Upload`
1. Open the Serial Monitor: `Tools -> Serial Monitor`
1. Shake the board. The change in acceleration will start recording data from the IMU in the Serial Monitor
Expand Down
2 changes: 1 addition & 1 deletion exercises/exercise6.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ You can enable 3rd party cookies, or better yet, add an exception for `[*.]googl

Open the [arduino_tinyml_workshop.ipynb](../arduino_tinyml_workshop.ipynb) notebook in Google Colab and follow the instructions in the notebook to prepare the data and train the model.

https://colab.research.google.com/github/arduino/AIoT-Dev-Summit-2019/blob/master/arduino_tinyml_workshop.ipynb
https://colab.research.google.com/github/don/tinyml-workshop/blob/master/arduino_tinyml_workshop.ipynb


Next [Exercise 7: Classifying IMU Data](exercise7.md)
2 changes: 1 addition & 1 deletion exercises/exercise7.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Exercise 7: Classifying IMU Data

1. Open __AIoT-Dev-Summit-2019/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE.
1. Open __tinyml-workshop/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE.
1. Switch to the model.h tab
1. Replace the contents of model.h with the version you downloaded from Colab
1. Upload the sketch: `Sketch -> Upload`
Expand Down
2 changes: 1 addition & 1 deletion exercises/exercise8.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Now that we the code can recognize gestures, let's try printing out some emojis in addition to the text.

Open __AIoT-Dev-Summit-2019/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE if it's not already loaded.
Open __tinyml-workshop/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino__ in the Arduino IDE if it's not already loaded.

Create a new char* array named `EMOJIS`. Inside the array defines the emojis as a UTF-8 encoded unicode strings. This array must be the same length at the `GESTURE` array. The order of the emojis in the array must match the order of the gestures.

Expand Down
2 changes: 1 addition & 1 deletion exercises/exercise9.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ Now you are ready to build a keyboard. Create a new sketch in the Arduino IDE. C

### Note

__NOTE:__ once you load code that runs the USB Keyboard, the Arduino IDE might not be able to see the serial port when you want to load new code. Double click the reset button on the Nano 33 BLE Sense before you run __Sketch -> Upload__.
__NOTE:__ once you load code that runs the USB Keyboard, the Arduino IDE might not be able to see the serial port when you want to load new code. Double click the reset button on the Nano 33 BLE before you run __Sketch -> Upload__.

Next [Exercise 10: Next Steps](exercise10.md)
Binary file added exercises/images/AI-IOT-devfest-AZ-2020-horiz.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added exercises/images/ChariotSolutions.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified exercises/images/clone-or-download.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added exercises/images/nano-33-ble.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added exercises/images/nano-33-ble_iso.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 3 additions & 5 deletions exercises/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ What you know -> | Supervised Learning | -> What you want to know

Running ML models on microcontrollers. Low power, below 1 mW.

## Arduino Nano 33 BLE Sense board
## Arduino Nano 33 BLE board

Based on the Nordic nRF52840
* Arm Cortex-M4F running at 64 MHz
Expand All @@ -90,8 +90,8 @@ Based on the Nordic nRF52840

Onboard Sensors
* IMU (measure motion: acceleration, gyro)
* Temperature, pressure, humidity, light, color
* PDM microphone

This workshop also works on the Arduino Nano 33 BLE Sense board. The Sense version of the board adds additional sensors: temperature, pressure, humidity, light, color, PDM microphone.

## What is an IMU?

Expand All @@ -105,7 +105,6 @@ This board uses the ST Micro LSM9DS1.

We're only using the accelerometer and gyroscope for this project.


## Workshop

* Record IMU data
Expand All @@ -124,7 +123,6 @@ We're only using the accelerometer and gyroscope for this project.

## IMU input data


The Arduino library enables the sensor to report 119 data points every second, this means a new set of data is received every 8.4ms


Expand Down