Skip to content

Commit 1df9c2d

Browse files
committed
PERF: read_csv with memory_map=True when file encoding is UTF-8 (pandas-dev#43787)
1 parent e727a56 commit 1df9c2d

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

pandas/io/common.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -874,7 +874,7 @@ def __iter__(self) -> _MMapWrapper:
874874
def read(self, size: int = -1) -> str | bytes:
875875
# CSV c-engine uses read instead of iterating
876876
content: bytes = self.mmap.read(size)
877-
if self.decode:
877+
if self.decode and self.encoding != "utf-8":
878878
# memory mapping is applied before compression. Encoding should
879879
# be applied to the de-compressed data.
880880
final = size == -1 or len(content) < size

0 commit comments

Comments
 (0)