Skip to content

Commit 2a958af

Browse files
committed
Merge remote-tracking branch 'upstream/main' into 240323_57972_kurtosis_low_value_cutoff
2 parents 2e67416 + 5703f11 commit 2a958af

File tree

265 files changed

+4718
-5730
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

265 files changed

+4718
-5730
lines changed

.github/actions/run-tests/action.yml

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,9 @@
11
name: Run tests and report results
2-
inputs:
3-
preload:
4-
description: Preload arguments for sanitizer
5-
required: false
6-
asan_options:
7-
description: Arguments for Address Sanitizer (ASAN)
8-
required: false
92
runs:
103
using: composite
114
steps:
125
- name: Test
13-
run: ${{ inputs.asan_options }} ${{ inputs.preload }} ci/run_tests.sh
6+
run: ci/run_tests.sh
147
shell: bash -el {0}
158

169
- name: Publish test results

.github/workflows/unit-tests.yml

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -68,14 +68,6 @@ jobs:
6868
- name: "Pyarrow Nightly"
6969
env_file: actions-311-pyarrownightly.yaml
7070
pattern: "not slow and not network and not single_cpu"
71-
- name: "ASAN / UBSAN"
72-
env_file: actions-311-sanitizers.yaml
73-
pattern: "not slow and not network and not single_cpu and not skip_ubsan"
74-
asan_options: "ASAN_OPTIONS=detect_leaks=0"
75-
preload: LD_PRELOAD=$(gcc -print-file-name=libasan.so)
76-
meson_args: --config-settings=setup-args="-Db_sanitize=address,undefined"
77-
cflags_adds: -fno-sanitize-recover=all
78-
pytest_workers: -1 # disable pytest-xdist as it swallows stderr from ASAN
7971
fail-fast: false
8072
name: ${{ matrix.name || format('ubuntu-latest {0}', matrix.env_file) }}
8173
env:
@@ -161,18 +153,12 @@ jobs:
161153
- name: Test (not single_cpu)
162154
uses: ./.github/actions/run-tests
163155
if: ${{ matrix.name != 'Pypy' }}
164-
with:
165-
preload: ${{ matrix.preload }}
166-
asan_options: ${{ matrix.asan_options }}
167156
env:
168157
# Set pattern to not single_cpu if not already set
169158
PATTERN: ${{ env.PATTERN == '' && 'not single_cpu' || matrix.pattern }}
170159

171160
- name: Test (single_cpu)
172161
uses: ./.github/actions/run-tests
173-
with:
174-
preload: ${{ matrix.preload }}
175-
asan_options: ${{ matrix.asan_options }}
176162
env:
177163
PATTERN: 'single_cpu'
178164
PYTEST_WORKERS: 0

.github/workflows/wheels.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ jobs:
141141

142142
- name: Build normal wheels
143143
if: ${{ (env.IS_SCHEDULE_DISPATCH != 'true' || env.IS_PUSH == 'true') }}
144-
uses: pypa/cibuildwheel@v2.16.5
144+
uses: pypa/cibuildwheel@v2.17.0
145145
with:
146146
package-dir: ./dist/${{ startsWith(matrix.buildplat[1], 'macosx') && env.sdist_name || needs.build_sdist.outputs.sdist_file }}
147147
env:
@@ -150,7 +150,7 @@ jobs:
150150

151151
- name: Build nightly wheels (with NumPy pre-release)
152152
if: ${{ (env.IS_SCHEDULE_DISPATCH == 'true' && env.IS_PUSH != 'true') }}
153-
uses: pypa/cibuildwheel@v2.16.5
153+
uses: pypa/cibuildwheel@v2.17.0
154154
with:
155155
package-dir: ./dist/${{ startsWith(matrix.buildplat[1], 'macosx') && env.sdist_name || needs.build_sdist.outputs.sdist_file }}
156156
env:

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ repos:
7878
hooks:
7979
- id: pylint
8080
stages: [manual]
81-
args: [--load-plugins=pylint.extensions.redefined_loop_name]
81+
args: [--load-plugins=pylint.extensions.redefined_loop_name, --fail-on=I0021]
8282
- id: pylint
8383
alias: redefined-outer-name
8484
name: Redefining name from outer scope

Dockerfile

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ RUN apt-get install -y build-essential
99
RUN apt-get install -y libhdf5-dev libgles2-mesa-dev
1010

1111
RUN python -m pip install --upgrade pip
12-
RUN python -m pip install \
13-
-r https://raw.githubusercontent.com/pandas-dev/pandas/main/requirements-dev.txt
12+
COPY requirements-dev.txt /tmp
13+
RUN python -m pip install -r /tmp/requirements-dev.txt
14+
RUN git config --global --add safe.directory /home/pandas
1415
CMD ["/bin/bash"]

asv_bench/benchmarks/categoricals.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ def setup(self):
8888
)
8989

9090
for col in ("int", "float", "timestamp"):
91-
self.df[col + "_as_str"] = self.df[col].astype(str)
91+
self.df[f"{col}_as_str"] = self.df[col].astype(str)
9292

9393
for col in self.df.columns:
9494
self.df[col] = self.df[col].astype("category")

asv_bench/benchmarks/join_merge.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,23 @@ def time_i8merge(self, how):
328328
merge(self.left, self.right, how=how)
329329

330330

331+
class UniqueMerge:
332+
params = [4_000_000, 1_000_000]
333+
param_names = ["unique_elements"]
334+
335+
def setup(self, unique_elements):
336+
N = 1_000_000
337+
self.left = DataFrame({"a": np.random.randint(1, unique_elements, (N,))})
338+
self.right = DataFrame({"a": np.random.randint(1, unique_elements, (N,))})
339+
uniques = self.right.a.drop_duplicates()
340+
self.right["a"] = concat(
341+
[uniques, Series(np.arange(0, -(N - len(uniques)), -1))], ignore_index=True
342+
)
343+
344+
def time_unique_merge(self, unique_elements):
345+
merge(self.left, self.right, how="inner")
346+
347+
331348
class MergeDatetime:
332349
params = [
333350
[

asv_bench/benchmarks/timeseries.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ def setup(self):
183183
self.dt_ts = Series(5, rng3, dtype="datetime64[ns]")
184184

185185
def time_resample(self):
186-
self.dt_ts.resample("1S").last()
186+
self.dt_ts.resample("1s").last()
187187

188188

189189
class AsOf:

0 commit comments

Comments
 (0)