Code

[WIP] ENH: add Pyarrow csv engine#38370

Code

Closed

arw2019 wants to merge 119 commits intopandas-dev:masterpandas-dev/pandas:masterfrom

arw2019:GH23697-arrow-csvarw2019/pandas:GH23697-arrow-csvCopy head branch name to clipboard

Commits

Commits on Feb 9, 2020

• add arrow engine to read_csv

  lithomas1

  committed
  f22ff46

  Copy full SHA for f22ff46

• fix failing test

  lithomas1

  committed
  8ae43e4

  Copy full SHA for 8ae43e4

• formatting and revert unnecessary change

  lithomas1

  committed
  09074df

  Copy full SHA for 09074df

• remove bloat and more formatting changes

  lithomas1

  committed
  6be276d

  Copy full SHA for 6be276d

• Whatsnew

  lithomas1

  committed
  df4fa7e

  Copy full SHA for df4fa7e

• Merge remote-tracking branch 'upstream/master' into add-arrow-engine

  lithomas1

  committed
  9cd9a6f

  Copy full SHA for 9cd9a6f

Commits on Feb 10, 2020

• Get tests up and running

  lithomas1

  committed
  ecaf3fd

  Copy full SHA for ecaf3fd

• Some fixes

  lithomas1

  committed
  b3c3287

  Copy full SHA for b3c3287

Commits on Feb 11, 2020

• Add asvs and xfail some tests

  lithomas1

  committed
  474baf4

  Copy full SHA for 474baf4

Commits on Feb 20, 2020

• address comments

  lithomas1

  committed
  2cd9937

  Copy full SHA for 2cd9937

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  48ff255

  Copy full SHA for 48ff255

• fix typo

  lithomas1

  committed
  3d15a56

  Copy full SHA for 3d15a56

• Merge branch 'add-arrow-engine' of github-other.com:lithomas1/pandas into add-arrow-engine

  lithomas1

  committed
  c969373

  Copy full SHA for c969373

Commits on Feb 29, 2020

• some fixes

  lithomas1

  committed
  98aa134

  Copy full SHA for 98aa134

Commits on Apr 5, 2020

• Fix bug

  lithomas1

  committed
  b9c6d2c

  Copy full SHA for b9c6d2c

• Fix merge conflicts

  lithomas1

  committed
  67c5db6

  Copy full SHA for 67c5db6

Commits on Apr 10, 2020

• New benchmark and fix more tests

  lithomas1

  committed
  7f891a6

  Copy full SHA for 7f891a6

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  11fc737

  Copy full SHA for 11fc737

• More cleanups

  lithomas1

  committed
  23425f7

  Copy full SHA for 23425f7

• Merge master

  lithomas1

  committed
  d9b7a1f

  Copy full SHA for d9b7a1f

Commits on Apr 11, 2020

• Merge branch 'add-arrow-engine' of github-other.com:lithomas1/pandas into add-arrow-engine

  lithomas1

  committed
  b8adf3c

  Copy full SHA for b8adf3c

• Formatting fixes and typo correction

  lithomas1

  committed
  01c0394

  Copy full SHA for 01c0394

Commits on Apr 12, 2020

• skip pyarrow tests if not installed

  lithomas1

  committed
  ba5620f

  Copy full SHA for ba5620f

• Address comments

  lithomas1

  committed
  2570c82

  Copy full SHA for 2570c82

Commits on Apr 14, 2020

• Get some more tests to pass

  lithomas1

  committed
  b3a1f66

  Copy full SHA for b3a1f66

Commits on Apr 17, 2020

• Fix some bugs and cleanups

  lithomas1

  committed
  d46ceed

  Copy full SHA for d46ceed

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  d67925c

  Copy full SHA for d67925c

Commits on May 20, 2020

• Perform version checks for submodule imports too

  lithomas1

  committed
  6378459

  Copy full SHA for 6378459

• Refresh with newer pyarrow

  lithomas1

  committed
  9d64882

  Copy full SHA for 9d64882

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  852ecf9

  Copy full SHA for 852ecf9

Commits on May 21, 2020

• Start xfailing tests

  lithomas1

  committed
  93382b4

  Copy full SHA for 93382b4

Commits on May 27, 2020

• Get all tests to run & some fixes

  lithomas1

  committed
  f1bb4e2

  Copy full SHA for f1bb4e2

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  14c13ab

  Copy full SHA for 14c13ab

Commits on May 29, 2020

• Lint and CI

  lithomas1

  committed
  7876b4e

  Copy full SHA for 7876b4e

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  4426642

  Copy full SHA for 4426642

Commits on Jun 3, 2020

• parse_dates support and fixups of some tests

  lithomas1

  committed
  008acab

  Copy full SHA for 008acab

Commits on Jun 13, 2020

• Date parsing fixes and address comments

  lithomas1

  committed
  2dddae7

  Copy full SHA for 2dddae7

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  261ef6a

  Copy full SHA for 261ef6a

Commits on Jun 29, 2020

• Clean/Address comments/Update docs

  lithomas1

  committed
  88e200a

  Copy full SHA for 88e200a

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  bf063ab

  Copy full SHA for bf063ab

• Fix typo

  Show description for ede2799
  

  lithomas1

  and

  jorisvandenbossche

  authored
  ede2799

  Copy full SHA for ede2799

Commits on Jul 8, 2020

• Fix doc failures

  lithomas1

  committed
  e8eff08

  Copy full SHA for e8eff08

Commits on Oct 22, 2020

• Merge remote-tracking branch 'upstream/master' into add-arrow-engine

  simonjayhawkins

  committed
  87cfcf5

  Copy full SHA for 87cfcf5

• wip

  simonjayhawkins

  committed
  55139ee

  Copy full SHA for 55139ee

• more xfails and skips

  simonjayhawkins

  committed
  c1aeecf

  Copy full SHA for c1aeecf

Commits on Oct 28, 2020

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  62fc9d6

  Copy full SHA for 62fc9d6

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for b53a620
  pre-commit-ci[bot]

  committed
  b53a620

  Copy full SHA for b53a620

• Fix typos

  lithomas1

  committed
  f13113d

  Copy full SHA for f13113d

• Doc fixes and more typo fixes

  lithomas1

  committed
  f9ce2e4

  Copy full SHA for f9ce2e4

Commits on Nov 2, 2020

• Green?

  lithomas1

  committed
  4158d6a

  Copy full SHA for 4158d6a

Commits on Nov 17, 2020

• Merge branch 'master' into add-arrow-engine

  lithomas1

  authored
  d34e75f

  Copy full SHA for d34e75f

Commits on Dec 8, 2020

• merge master

  arw2019

  committed
  6a37695

  Copy full SHA for 6a37695

• xfail tests

  arw2019

  committed
  10be581

  Copy full SHA for 10be581

• xfail test

  arw2019

  committed
  fcc7e04

  Copy full SHA for fcc7e04

• fix import

  arw2019

  committed
  d7959a1

  Copy full SHA for d7959a1

• xfail tests

  arw2019

  committed
  e37d126

  Copy full SHA for e37d126

Commits on Dec 9, 2020

• skip tests

  arw2019

  committed
  3bc4775

  Copy full SHA for 3bc4775

• merge

  arw2019

  committed
  7097bcb

  Copy full SHA for 7097bcb

• skip tests

  arw2019

  committed
  17a502d

  Copy full SHA for 17a502d

• C408 failure

  arw2019

  committed
  e27d7ef

  Copy full SHA for e27d7ef

• skip tests

  arw2019

  committed
  4e638e9

  Copy full SHA for 4e638e9

Commits on Dec 10, 2020

• simplify import_optional_dependency code

  arw2019

  committed
  4f7ebd0

  Copy full SHA for 4f7ebd0

• move whatsnew to 1.3

  arw2019

  committed
  69b3b42

  Copy full SHA for 69b3b42

• clean _get_options_with_defaults

  arw2019

  committed
  9d5cf24

  Copy full SHA for 9d5cf24

• clean _clean_options

  arw2019

  committed
  2d4a0aa

  Copy full SHA for 2d4a0aa

• clean _read

  arw2019

  committed
  e46b95d

  Copy full SHA for e46b95d

• extract kwd validation from __init__

  arw2019

  committed
  1844a6c

  Copy full SHA for 1844a6c

Commits on Dec 11, 2020

• revert mistaken refactor

  arw2019

  committed
  94178e4

  Copy full SHA for 94178e4

• typing

  arw2019

  committed
  13a2488

  Copy full SHA for 13a2488

• REF: ArrowParserWrapper.read

  arw2019

  committed
  a98cffd

  Copy full SHA for a98cffd

• REF: ArrowParserWrapper.read

  arw2019

  committed
  a32e3a5

  Copy full SHA for a32e3a5

Commits on Dec 12, 2020

• remove optional dependency code

  arw2019

  committed
  89416cc

  Copy full SHA for 89416cc

• Merge branch 'master' of https://github.com/pandas-dev/pandas into GH23697-arrow-csv

  arw2019

  committed
  a1bd010

  Copy full SHA for a1bd010

• REF: ArrowParserWrapper.read

  arw2019

  committed
  9687990

  Copy full SHA for 9687990

• REF: ArrowParserWrapper.read

  arw2019

  committed
  98f2061

  Copy full SHA for 98f2061

• REF: ArrowParserWrapper.read

  arw2019

  committed
  ec01fad

  Copy full SHA for ec01fad

• rewrite docs

  arw2019

  committed
  7b9572b

  Copy full SHA for 7b9572b

• rewrite docs

  arw2019

  committed
  6773a71

  Copy full SHA for 6773a71

• remove datetime hadling

  arw2019

  committed
  d63f5d0

  Copy full SHA for d63f5d0

• skiprows cannot be None

  arw2019

  committed
  9ff95ad

  Copy full SHA for 9ff95ad

• REF: ArrowParserWrapper.read

  arw2019

  committed
  6133a4c

  Copy full SHA for 6133a4c

• REF: ArrowParserWrapper.read

  arw2019

  committed
  454892f

  Copy full SHA for 454892f

• skip all pyarrow csv datetime tests

  arw2019

  committed
  e050394

  Copy full SHA for e050394

• rewrite benchmarks

  arw2019

  committed
  09fca60

  Copy full SHA for 09fca60

Commits on Dec 16, 2020

• merge master

  arw2019

  committed
  7aa5378

  Copy full SHA for 7aa5378

Commits on Dec 19, 2020

• merge master

  arw2019

  committed
  ac3cf7d

  Copy full SHA for ac3cf7d

• typo

  arw2019

  committed
  f9bf5f1

  Copy full SHA for f9bf5f1

• typo

  arw2019

  committed
  922bf4f

  Copy full SHA for 922bf4f

• test reorg

  arw2019

  committed
  1252a05

  Copy full SHA for 1252a05

• test reorg

  arw2019

  committed
  0af7291

  Copy full SHA for 0af7291

• test reorg

  arw2019

  committed
  361aab6

  Copy full SHA for 361aab6

• test reorg

  arw2019

  committed
  75de071

  Copy full SHA for 75de071

• test reorg

  arw2019

  committed
  a1dfcb2

  Copy full SHA for a1dfcb2

• test reorg

  arw2019

  committed
  2433170

  Copy full SHA for 2433170

• test reorg

  arw2019

  committed
  1a9f185

  Copy full SHA for 1a9f185

• test reorg

  arw2019

  committed
  16d37db

  Copy full SHA for 16d37db

Commits on Dec 31, 2020

• pyarrow_xfail->pyarrow_skip

  arw2019

  committed
  e124df0

  Copy full SHA for e124df0

• merge master

  arw2019

  committed
  75d099b

  Copy full SHA for 75d099b

• pyarrow_xfail->pyarrow_skip

  arw2019

  committed
  fe253ba

  Copy full SHA for fe253ba

• pyarrow_xfail->pyarrow_skip

  arw2019

  committed
  72c7c44

  Copy full SHA for 72c7c44

• xfail more tests

  arw2019

  committed
  2671007

  Copy full SHA for 2671007

• xfail more tests

  arw2019

  committed
  73ca5d4

  Copy full SHA for 73ca5d4

Commits on Jan 1, 2021

• merge master

  arw2019

  committed
  0666042

  Copy full SHA for 0666042

• update refactoredt tests

  arw2019

  committed
  639ca28

  Copy full SHA for 639ca28

Commits on Jan 2, 2021

• float precision tests

  arw2019

  committed
  1994fad

  Copy full SHA for 1994fad

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  566f1b4

  Copy full SHA for 566f1b4

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  cd9b300

  Copy full SHA for cd9b300

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  4a7dc0f

  Copy full SHA for 4a7dc0f

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  3b24fe7

  Copy full SHA for 3b24fe7

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  c33bf46

  Copy full SHA for c33bf46

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  dc9530b

  Copy full SHA for dc9530b

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  d83b2e0

  Copy full SHA for d83b2e0

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  6205bed

  Copy full SHA for 6205bed

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  c4b3bb7

  Copy full SHA for c4b3bb7

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  a77b33e

  Copy full SHA for a77b33e

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  04c8d21

  Copy full SHA for 04c8d21

• TST/REF: io/parsers/test_common.py

  arw2019

  committed
  8bb6959

  Copy full SHA for 8bb6959

• TST/REF: remove test_common.py

  arw2019

  committed
  d9478d6

  Copy full SHA for d9478d6

Commits on Jan 4, 2021

• merge master

  arw2019

  committed
  565f71f

  Copy full SHA for 565f71f