Skip to content

Remove unused asv directives and verify in the CI that we are not using them #23075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
datapythonista opened this issue Oct 10, 2018 · 3 comments · Fixed by #23265
Closed

Remove unused asv directives and verify in the CI that we are not using them #23075

datapythonista opened this issue Oct 10, 2018 · 3 comments · Fixed by #23265
Labels
Benchmark Performance (ASV) benchmarks CI Continuous Integration good first issue
Milestone

Comments

@datapythonista
Copy link
Member

Some of the directives being used in our benchmarks (asv_bench/benchmarks/) are deprecated in asv. e.g. in algorithms.py each of the benchmark classes defines goal_time (e.g. https://github.com/pandas-dev/pandas/blob/master/asv_bench/benchmarks/algorithms.py#L18), but it looks like asv just ignores that: https://asv.readthedocs.io/en/stable/benchmarks.html#benchmark-attributes

We should update the benchmarks to not include unused asv directives, and we should add to ./ci/code_checks.py a way to identify if unused ones are being used (ideally getting the list of expected directives from asv itself).

Originally from #22884 (comment)

@jbrockmendel can you review the issue description and edit if needed please

@datapythonista datapythonista added CI Continuous Integration Benchmark Performance (ASV) benchmarks labels Oct 10, 2018
@jbrockmendel
Copy link
Member

Looks right to me. @pv any thoughts on how best to handle this? Would it make sense to implement these checks within asv?

@eriknil
Copy link
Contributor

eriknil commented Oct 21, 2018

I'll take a look at this

@cpcloud
Copy link
Member

cpcloud commented Oct 21, 2018

Working with @eriknil at PyData NYC 2018 mini-sprints.

I don't think that it's possible to determine whether unused directives are being used. The set of unused asv directives is infinite and includes any attribute on the object that isn't one that is recognized by asv.

Are you proposing limiting the class variables that can be defined?

For example, how is it possible to know that goal_time was intended to be used by asv and isn't just an attribute used by something else on the class?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Benchmark Performance (ASV) benchmarks CI Continuous Integration good first issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants