Skip to content

Commit d4ad93f

Browse files
committed
airline.ontime testcase
1 parent 2268b95 commit d4ad93f

File tree

2 files changed

+36
-5
lines changed

2 files changed

+36
-5
lines changed

README.md

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
* [MySQL Test Tables](#mysql-test-tables)
2020
* [ClickHouse Test Tables](#clickhouse-test-tables)
2121
* [Test Cases](#test-cases)
22-
* [airline.ontime Test Case](#airline-ontime-test-case)
22+
* [airline.ontime Test Case](#airlineontime-test-case)
2323

2424
---
2525

@@ -722,6 +722,13 @@ Main Steps
722722

723723
#### airline.ontime Data Set in CSV files
724724
Run [download script](run_airline_ontime_data_download.sh)
725+
You may want to adjust dirs where to keep `ZIP` and `CSV` file
726+
In `run_airline_ontime_data_download.sh` edit these lines:
727+
```bash
728+
ZIP_FILES_DIR="zip"
729+
CSV_FILES_DIR="csv"
730+
```
731+
725732
```bash
726733
./run_airline_ontime_data_download.sh
727734
```
@@ -964,12 +971,37 @@ CREATE TABLE IF NOT EXISTS `airline`.`ontime` (
964971

965972
#### airline.ontime Data Reader
966973
Run [datareader script](run_airline_ontime_data_reader.sh)
974+
You may want to adjust `PYTHON` path and source and target hosts and usernames
975+
```bash
976+
PYTHON=python3.6
977+
PYTHON=/home/user/pypy3.5-5.9-beta-linux_x86_64-portable/bin/pypy
978+
```
979+
```bash
980+
...
981+
--src-host=127.0.0.1 \
982+
--src-user=root \
983+
--dst-host=127.0.0.1 \
984+
...
985+
```
967986
```bash
968987
./run_airline_ontime_data_reader.sh
969988
```
970989

971990
#### airline.ontime Data Importer
972991
Run [data importer script](run_airline_ontime_import.sh)
992+
You may want to adjust `CSV` files location, number of imported files and MySQL user/password used for import
993+
```bash
994+
# looking for csv files in this dir
995+
FILES_TO_IMPORT_DIR="/mnt/nas/work/ontime"
996+
997+
# limit import to this number of files
998+
FILES_TO_IMPORT_NUM=3
999+
```
1000+
```bash
1001+
...
1002+
-u root \
1003+
...
1004+
```
9731005

9741006
```bash
9751007
./run_airline_ontime_import.sh

run_airline_ontime_data_download.sh

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
#!/bin/bash
22
# download airline.ontime test dataset
33

4+
ZIP_FILES_DIR="zip"
5+
CSV_FILES_DIR="csv"
6+
47
echo "Check required commands availability"
58
if command -v wget && command -v unzip && command -v clickhouse-client && command -v wc && command -v awk; then
69
echo "Looks like all required commands are available"
@@ -11,7 +14,6 @@ fi
1114

1215
echo "Download dataset"
1316

14-
ZIP_FILES_DIR="zip"
1517
echo "Create dir $ZIP_FILES_DIR for downloading zip files"
1618
mkdir "$ZIP_FILES_DIR"
1719

@@ -30,9 +32,6 @@ done
3032

3133
echo "Unzip dataset"
3234

33-
ZIP_FILES_DIR="zip"
34-
CSV_FILES_DIR="csv"
35-
3635
mkdir "$CSV_FILES_DIR"
3736

3837
if [ ! -d "$CSV_FILES_DIR" ]; then

0 commit comments

Comments
 (0)