Skip to content

Commit ac15528

Browse files
authored
PERF: read_csv macro updates (#52632)
* Try simplifying delim macro * no dereference impl * whatsnew
1 parent 8b797e9 commit ac15528

File tree

2 files changed

+5
-3
lines changed

2 files changed

+5
-3
lines changed

doc/source/whatsnew/v2.1.0.rst

+1
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ Other enhancements
8888
- Improve error message when setting :class:`DataFrame` with wrong number of columns through :meth:`DataFrame.isetitem` (:issue:`51701`)
8989
- Improved error message when creating a DataFrame with empty data (0 rows), no index and an incorrect number of columns. (:issue:`52084`)
9090
- Let :meth:`DataFrame.to_feather` accept a non-default :class:`Index` and non-string column names (:issue:`51787`)
91+
- Performance improvement in :func:`read_csv` (:issue:`52632`) with ``engine="c"``
9192
-
9293

9394
.. ---------------------------------------------------------------------------

pandas/_libs/src/parser/tokenizer.c

+4-3
Original file line numberDiff line numberDiff line change
@@ -664,9 +664,7 @@ static int parser_buffer_bytes(parser_t *self, size_t nbytes,
664664
((!self->delim_whitespace && c == ' ' && self->skipinitialspace))
665665

666666
// applied when in a field
667-
#define IS_DELIMITER(c) \
668-
((!self->delim_whitespace && c == self->delimiter) || \
669-
(self->delim_whitespace && isblank(c)))
667+
#define IS_DELIMITER(c) ((c == delimiter) || (delim_whitespace && isblank(c)))
670668

671669
#define _TOKEN_CLEANUP() \
672670
self->stream_len = slen; \
@@ -721,6 +719,9 @@ int tokenize_bytes(parser_t *self,
721719
const char lineterminator = (self->lineterminator == '\0') ?
722720
'\n' : self->lineterminator;
723721

722+
const int delim_whitespace = self->delim_whitespace;
723+
const char delimiter = self->delimiter;
724+
724725
// 1000 is something that couldn't fit in "char"
725726
// thus comparing a char to it would always be "false"
726727
const int carriage_symbol = (self->lineterminator == '\0') ? '\r' : 1000;

0 commit comments

Comments
 (0)