Skip to content

Commit 11a5ca6

Browse files
8bitkicksandeepmistry
8bitkick
authored and
sandeepmistry
committed
Add FruitToEmoji tutorial
1 parent fc371df commit 11a5ca6

File tree

7 files changed

+803
-0
lines changed

7 files changed

+803
-0
lines changed

‎FruitToEmoji/FruitToEmoji.ipynb

+351
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,351 @@
1+
{
2+
"nbformat": 4,
3+
"nbformat_minor": 0,
4+
"metadata": {
5+
"colab": {
6+
"name": "FruitToEmoji-GIT.ipynb",
7+
"provenance": [],
8+
"collapsed_sections": [],
9+
"toc_visible": true
10+
},
11+
"kernelspec": {
12+
"name": "python3",
13+
"display_name": "Python 3"
14+
}
15+
},
16+
"cells": [
17+
{
18+
"cell_type": "markdown",
19+
"metadata": {
20+
"id": "f92-4Hjy7kA8",
21+
"colab_type": "text"
22+
},
23+
"source": [
24+
"<a href=\"https://www.arduino.cc/\"><img src=\"https://raw.githubusercontent.com/sandeepmistry/aimldevfest-workshop-2019/master/images/Arduino_logo_R_highquality.png\" width=200/></a>\n",
25+
"# Tiny ML on Arduino\n",
26+
"## Classify objects by color tutorial\n",
27+
"\n",
28+
" \n",
29+
"https://github.com/arduino/ArduinoTensorFlowLiteTutorials/"
30+
]
31+
},
32+
{
33+
"cell_type": "markdown",
34+
"metadata": {
35+
"id": "uvDA8AK7QOq-",
36+
"colab_type": "text"
37+
},
38+
"source": [
39+
"## Setup Python Environment \n",
40+
"\n",
41+
"The next cell sets up the dependencies in required for the notebook, run it."
42+
]
43+
},
44+
{
45+
"cell_type": "code",
46+
"metadata": {
47+
"id": "Y2gs-PL4xDkZ",
48+
"colab_type": "code",
49+
"colab": {}
50+
},
51+
"source": [
52+
"# Setup environment\n",
53+
"!apt-get -qq install xxd\n",
54+
"!pip install pandas numpy matplotlib\n",
55+
"%tensorflow_version 2.x\n",
56+
"!pip install tensorflow"
57+
],
58+
"execution_count": 0,
59+
"outputs": []
60+
},
61+
{
62+
"cell_type": "markdown",
63+
"metadata": {
64+
"id": "9lwkeshJk7dg",
65+
"colab_type": "text"
66+
},
67+
"source": [
68+
"# Upload Data\n",
69+
"\n",
70+
"1. Open the panel on the left side of Colab by clicking on the __>__\n",
71+
"1. Select the Files tab\n",
72+
"1. Drag `csv` files from your computer to the tab to upload them into colab."
73+
]
74+
},
75+
{
76+
"cell_type": "markdown",
77+
"metadata": {
78+
"id": "kSxUeYPNQbOg",
79+
"colab_type": "text"
80+
},
81+
"source": [
82+
"# Train Neural Network\n",
83+
"\n",
84+
"\n",
85+
"\n"
86+
]
87+
},
88+
{
89+
"cell_type": "markdown",
90+
"metadata": {
91+
"id": "Gxk414PU3oy3",
92+
"colab_type": "text"
93+
},
94+
"source": [
95+
"## Parse and prepare the data\n",
96+
"\n",
97+
"The next cell parses the csv files and transforms them to a format that will be used to train the full connected neural network.\n",
98+
"\n"
99+
]
100+
},
101+
{
102+
"cell_type": "code",
103+
"metadata": {
104+
"id": "AGChd1FAk5_j",
105+
"colab_type": "code",
106+
"colab": {}
107+
},
108+
"source": [
109+
"import matplotlib.pyplot as plt\n",
110+
"import numpy as np\n",
111+
"import pandas as pd\n",
112+
"import tensorflow as tf\n",
113+
"import os\n",
114+
"import fileinput\n",
115+
"\n",
116+
"print(f\"TensorFlow version = {tf.__version__}\\n\")\n",
117+
"\n",
118+
"# Set a fixed random seed value, for reproducibility, this will allow us to get\n",
119+
"# the same random numbers each time the notebook is run\n",
120+
"SEED = 1337\n",
121+
"np.random.seed(SEED)\n",
122+
"tf.random.set_seed(SEED)\n",
123+
"\n",
124+
"CLASSES = [];\n",
125+
"\n",
126+
"for file in os.listdir(\"/content/\"):\n",
127+
" if file.endswith(\".csv\"):\n",
128+
" CLASSES.append(os.path.splitext(file)[0])\n",
129+
"\n",
130+
"CLASSES.sort()\n",
131+
"\n",
132+
"SAMPLES_WINDOW_LEN = 1\n",
133+
"NUM_CLASSES = len(CLASSES)\n",
134+
"\n",
135+
"# create a one-hot encoded matrix that is used in the output\n",
136+
"ONE_HOT_ENCODED_CLASSES = np.eye(NUM_CLASSES)\n",
137+
"\n",
138+
"inputs = []\n",
139+
"outputs = []\n",
140+
"\n",
141+
"# read each csv file and push an input and output\n",
142+
"for class_index in range(NUM_CLASSES):\n",
143+
" objectClass = CLASSES[class_index]\n",
144+
" df = pd.read_csv(\"/content/\" + objectClass + \".csv\")\n",
145+
" columns = list(df)\n",
146+
" # get rid of pesky empty value lines of csv which cause NaN inputs to TensorFlow\n",
147+
" df = df.dropna()\n",
148+
" df = df.reset_index(drop=True)\n",
149+
" \n",
150+
" # calculate the number of objectClass recordings in the file\n",
151+
" num_recordings = int(df.shape[0] / SAMPLES_WINDOW_LEN)\n",
152+
" print(f\"\\u001b[32;4m{objectClass}\\u001b[0m class will be output \\u001b[32m{class_index}\\u001b[0m of the classifier\")\n",
153+
" print(f\"{num_recordings} samples captured for training with inputs {list(df)} \\n\")\n",
154+
"\n",
155+
" # graphing\n",
156+
" plt.rcParams[\"figure.figsize\"] = (10,1)\n",
157+
" pixels = np.array([df['Red'],df['Green'],df['Blue']],float)\n",
158+
" pixels = np.transpose(pixels)\n",
159+
" for i in range(num_recordings):\n",
160+
" plt.axvline(x=i, linewidth=8, color=tuple(pixels[i]/np.max(pixels[i], axis=0)))\n",
161+
" plt.show()\n",
162+
" \n",
163+
" #tensors\n",
164+
" output = ONE_HOT_ENCODED_CLASSES[class_index]\n",
165+
" for i in range(num_recordings):\n",
166+
" tensor = []\n",
167+
" row = []\n",
168+
" for c in columns:\n",
169+
" row.append(df[c][i])\n",
170+
" tensor += row\n",
171+
" inputs.append(tensor)\n",
172+
" outputs.append(output)\n",
173+
"\n",
174+
"# convert the list to numpy array\n",
175+
"inputs = np.array(inputs)\n",
176+
"outputs = np.array(outputs)\n",
177+
"\n",
178+
"print(\"Data set parsing and preparation complete.\")\n",
179+
"\n",
180+
"# Randomize the order of the inputs, so they can be evenly distributed for training, testing, and validation\n",
181+
"# https://stackoverflow.com/a/37710486/2020087\n",
182+
"num_inputs = len(inputs)\n",
183+
"randomize = np.arange(num_inputs)\n",
184+
"np.random.shuffle(randomize)\n",
185+
"\n",
186+
"# Swap the consecutive indexes (0, 1, 2, etc) with the randomized indexes\n",
187+
"inputs = inputs[randomize]\n",
188+
"outputs = outputs[randomize]\n",
189+
"\n",
190+
"# Split the recordings (group of samples) into three sets: training, testing and validation\n",
191+
"TRAIN_SPLIT = int(0.6 * num_inputs)\n",
192+
"TEST_SPLIT = int(0.2 * num_inputs + TRAIN_SPLIT)\n",
193+
"\n",
194+
"inputs_train, inputs_test, inputs_validate = np.split(inputs, [TRAIN_SPLIT, TEST_SPLIT])\n",
195+
"outputs_train, outputs_test, outputs_validate = np.split(outputs, [TRAIN_SPLIT, TEST_SPLIT])\n",
196+
"\n",
197+
"print(\"Data set randomization and splitting complete.\")\n"
198+
],
199+
"execution_count": 0,
200+
"outputs": []
201+
},
202+
{
203+
"cell_type": "markdown",
204+
"metadata": {
205+
"colab_type": "text",
206+
"id": "v8qlSAX1b6Yv"
207+
},
208+
"source": [
209+
"## Build & Train the Model\n",
210+
"\n",
211+
"Build and train a [TensorFlow](https://www.tensorflow.org) model using the high-level [Keras](https://www.tensorflow.org/guide/keras) API."
212+
]
213+
},
214+
{
215+
"cell_type": "code",
216+
"metadata": {
217+
"id": "kGNFa-lX24Qo",
218+
"colab_type": "code",
219+
"colab": {}
220+
},
221+
"source": [
222+
"# build the model and train it\n",
223+
"model = tf.keras.Sequential()\n",
224+
"model.add(tf.keras.layers.Dense(8, activation='relu')) # relu is used for performance\n",
225+
"model.add(tf.keras.layers.Dense(5, activation='relu'))\n",
226+
"model.add(tf.keras.layers.Dense(NUM_CLASSES, activation='softmax')) # softmax is used, because we only expect one class to occur per input\n",
227+
"model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
228+
"history = model.fit(inputs_train, outputs_train, epochs=400, batch_size=4, validation_data=(inputs_validate, outputs_validate))\n",
229+
"\n"
230+
],
231+
"execution_count": 0,
232+
"outputs": []
233+
},
234+
{
235+
"cell_type": "markdown",
236+
"metadata": {
237+
"id": "guMjtfa42ahM",
238+
"colab_type": "text"
239+
},
240+
"source": [
241+
"### Run with Test Data\n",
242+
"Put our test data into the model and plot the predictions\n"
243+
]
244+
},
245+
{
246+
"cell_type": "code",
247+
"metadata": {
248+
"id": "V3Y0CCWJz2EK",
249+
"colab_type": "code",
250+
"colab": {}
251+
},
252+
"source": [
253+
"# use the model to predict the test inputs\n",
254+
"predictions = model.predict(inputs_test)\n",
255+
"\n",
256+
"# print the predictions and the expected ouputs\n",
257+
"print(\"predictions =\\n\", np.round(predictions, decimals=3))\n",
258+
"print(\"actual =\\n\", outputs_test)\n",
259+
"\n",
260+
"# Plot the predictions along with to the test data\n",
261+
"plt.clf()\n",
262+
"plt.title('Training data predicted vs actual values')\n",
263+
"plt.plot(inputs_test, outputs_test, 'b.', label='Actual')\n",
264+
"plt.plot(inputs_test, predictions, 'r.', label='Predicted')\n",
265+
"plt.show()"
266+
],
267+
"execution_count": 0,
268+
"outputs": []
269+
},
270+
{
271+
"cell_type": "markdown",
272+
"metadata": {
273+
"id": "j7DO6xxXVCym",
274+
"colab_type": "text"
275+
},
276+
"source": [
277+
"# Convert the Trained Model to Tensor Flow Lite\n",
278+
"\n",
279+
"The next cell converts the model to TFlite format. The size in bytes of the model is also printed out."
280+
]
281+
},
282+
{
283+
"cell_type": "code",
284+
"metadata": {
285+
"id": "0Xn1-Rn9Cp_8",
286+
"colab_type": "code",
287+
"colab": {}
288+
},
289+
"source": [
290+
"# Convert the model to the TensorFlow Lite format without quantization\n",
291+
"converter = tf.lite.TFLiteConverter.from_keras_model(model)\n",
292+
"tflite_model = converter.convert()\n",
293+
"\n",
294+
"# Save the model to disk\n",
295+
"open(\"gesture_model.tflite\", \"wb\").write(tflite_model)\n",
296+
" \n",
297+
"import os\n",
298+
"basic_model_size = os.path.getsize(\"gesture_model.tflite\")\n",
299+
"print(\"Model is %d bytes\" % basic_model_size)\n",
300+
" \n",
301+
" "
302+
],
303+
"execution_count": 0,
304+
"outputs": []
305+
},
306+
{
307+
"cell_type": "markdown",
308+
"metadata": {
309+
"id": "ykccQn7SXrUX",
310+
"colab_type": "text"
311+
},
312+
"source": [
313+
"## Encode the Model in an Arduino Header File \n",
314+
"\n",
315+
"The next cell creates a constant byte array that contains the TFlite model. Import it as a tab with the sketch below."
316+
]
317+
},
318+
{
319+
"cell_type": "code",
320+
"metadata": {
321+
"id": "9J33uwpNtAku",
322+
"colab_type": "code",
323+
"colab": {}
324+
},
325+
"source": [
326+
"!echo \"const unsigned char model[] = {\" > /content/model.h\n",
327+
"!cat gesture_model.tflite | xxd -i >> /content/model.h\n",
328+
"!echo \"};\" >> /content/model.h\n",
329+
"\n",
330+
"import os\n",
331+
"model_h_size = os.path.getsize(\"model.h\")\n",
332+
"print(f\"Header file, model.h, is {model_h_size:,} bytes.\")\n",
333+
"print(\"\\nOpen the side panel (refresh if needed). Double click model.h to download the file.\")"
334+
],
335+
"execution_count": 0,
336+
"outputs": []
337+
},
338+
{
339+
"cell_type": "markdown",
340+
"metadata": {
341+
"id": "1eSkHZaLzMId",
342+
"colab_type": "text"
343+
},
344+
"source": [
345+
"# Realtime Classification of Sensor Data on Arduino\n",
346+
"\n",
347+
"Now it's time to switch back to the tutorial instructions and run our new model on the [Arduino Nano 33 BLE Sense](https://www.arduino.cc/en/Guide/NANO33BLE)"
348+
]
349+
}
350+
]
351+
}

‎FruitToEmoji/README.md

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# FruitToEmoji
2+
3+
Classifies fruit using the RGB color and proximity sensors of the Arduino Nano 33 BLE Sense, using a TensorFlow Lite Micro model trained on data captured from the same hardware
4+

0 commit comments

Comments
 (0)