Skip to content

Commit d2f74d1

Browse files
committed
docs polihsing
1 parent d4ad93f commit d2f74d1

File tree

1 file changed

+21
-7
lines changed

1 file changed

+21
-7
lines changed

README.md

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ pip install clickhouse-driver
4545

4646
Also the following (at least one of) MySQL privileges are required for this operation: `SUPER`, `REPLICATION CLIENT`
4747

48-
```sql
48+
```mysql
4949
CREATE USER 'reader'@'localhost' IDENTIFIED BY 'qwerty';
5050
CREATE USER 'reader'@'127.0.0.1' IDENTIFIED BY 'qwerty';
5151
CREATE USER 'reader'@'*' IDENTIFIED BY 'qwerty';
@@ -65,7 +65,7 @@ Also the following MySQL config options are required:
6565
```ini
6666
[mysqld]
6767
server-id = 1
68-
log_bin = /var/log/mysql/mysql-bin.log
68+
log_bin = /var/lib/mysql/bin.log
6969
expire_logs_days = 10
7070
max_binlog_size = 100M
7171
binlog-format = row #Very important if you want to receive write, update and delete row events
@@ -293,7 +293,7 @@ We have to separate test table into several ones because of this error, produced
293293
ERROR 1118 (42000): Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs
294294
```
295295

296-
```sql
296+
```mysql
297297
CREATE TABLE datatypes(
298298

299299
bit_1 BIT(1),
@@ -397,12 +397,12 @@ CREATE TABLE long_varbinary_datatypes(
397397
```
398398

399399

400-
```sql
400+
```mysql
401401
-- in order to be able to set timestamp = '1970-01-01 00:00:01'
402402
set time_zone='+00:00';
403403
```
404404

405-
```sql
405+
```mysql
406406
-- MIN values
407407
INSERT INTO datatypes SET
408408

@@ -500,7 +500,7 @@ INSERT INTO long_varbinary_datatypes SET
500500
;
501501
```
502502

503-
```sql
503+
```mysql
504504
-- MAX values
505505
INSERT INTO datatypes SET
506506

@@ -722,12 +722,24 @@ Main Steps
722722

723723
#### airline.ontime Data Set in CSV files
724724
Run [download script](run_airline_ontime_data_download.sh)
725+
725726
You may want to adjust dirs where to keep `ZIP` and `CSV` file
727+
726728
In `run_airline_ontime_data_download.sh` edit these lines:
727729
```bash
728730
ZIP_FILES_DIR="zip"
729731
CSV_FILES_DIR="csv"
730732
```
733+
You may want to adjust number of files to download (In case downloading all it may take some time).
734+
735+
Specify year and months range as you wish:
736+
```bash
737+
...
738+
echo "Download files into $ZIP_FILES_DIR"
739+
for year in `seq 1987 2017`; do
740+
for month in `seq 1 12`; do
741+
...
742+
```
731743
732744
```bash
733745
./run_airline_ontime_data_download.sh
@@ -737,7 +749,7 @@ Downloading can take some time.
737749
#### airline.ontime MySQL Table
738750
Create MySQL table of the following structure:
739751
740-
```sql
752+
```mysql
741753
CREATE DATABASE IF NOT EXISTS `airline`;
742754
CREATE TABLE IF NOT EXISTS `airline`.`ontime` (
743755
`Year` SMALLINT UNSIGNED, -- maps to UInt16,
@@ -971,6 +983,7 @@ CREATE TABLE IF NOT EXISTS `airline`.`ontime` (
971983
972984
#### airline.ontime Data Reader
973985
Run [datareader script](run_airline_ontime_data_reader.sh)
986+
974987
You may want to adjust `PYTHON` path and source and target hosts and usernames
975988
```bash
976989
PYTHON=python3.6
@@ -989,6 +1002,7 @@ PYTHON=/home/user/pypy3.5-5.9-beta-linux_x86_64-portable/bin/pypy
9891002
9901003
#### airline.ontime Data Importer
9911004
Run [data importer script](run_airline_ontime_import.sh)
1005+
9921006
You may want to adjust `CSV` files location, number of imported files and MySQL user/password used for import
9931007
```bash
9941008
# looking for csv files in this dir

0 commit comments

Comments
 (0)