Skip to content

Cache quote unpickling #12242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

nicolasstucki
Copy link
Contributor

No description provided.

@nicolasstucki nicolasstucki self-assigned this Apr 27, 2021
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 7 job(s) in queue, 1 running.

@smarter
Copy link
Member

smarter commented Apr 27, 2021

In what situations will the same quote be unpickled multiple times?

@nicolasstucki
Copy link
Contributor Author

I will try different variants. The current is using the smallest cashing possible, the cache lives during the expansion of a single macro but skips some due to a bug. This is enough for a recursive macro to be optimized. Though I am not sure if this difference will be noticeable with the current benchmarks. The first step will be to create a benchmark where this difference can be noticed. Also see #12228.

@nicolasstucki nicolasstucki force-pushed the add-cache-to-quote-unpickling branch from 87aad4c to c0e7736 Compare April 27, 2021 15:47
@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (3d8b6a1)

@nicolasstucki nicolasstucki force-pushed the add-cache-to-quote-unpickling branch from bae29e3 to bd2fd57 Compare April 28, 2021 07:27
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 1 job(s) in queue, 0 running.

@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (007dacd)

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 1 job(s) in queue, 0 running.

@liufengyun
Copy link
Contributor

@nicolasstucki You may need to clear the cache to see the result.

@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (007dacd)

@nicolasstucki
Copy link
Contributor Author

@nicolasstucki You may need to clear the cache to see the result.

I noticed

@nicolasstucki nicolasstucki force-pushed the add-cache-to-quote-unpickling branch from bd2fd57 to d6eed31 Compare April 28, 2021 08:33
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 1 job(s) in queue, 0 running.

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 2 job(s) in queue, 1 running.

@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (bb5e140)

1 similar comment
@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (bb5e140)

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 1 job(s) in queue, 1 running.

@nicolasstucki
Copy link
Contributor Author

The per run cache did not seem to help much in https://github.com/lampepfl/dotty/blob/master/tests/bench/power-macro/PowerInlined-1k.scala even though we expand 8 macros

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

2 similar comments
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 1 job(s) in queue, 1 running.

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 3 job(s) in queue, 1 running.

@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (c14ec43)

@nicolasstucki nicolasstucki force-pushed the add-cache-to-quote-unpickling branch from 34bfc22 to 94f069e Compare April 28, 2021 13:51
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 8 job(s) in queue, 0 running.

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 8 job(s) in queue, 0 running.

@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@nicolasstucki nicolasstucki force-pushed the add-cache-to-quote-unpickling branch from a394572 to 97d747a Compare April 28, 2021 14:22
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 8 job(s) in queue, 0 running.

1 similar comment
@dottybot
Copy link
Member

performance test scheduled: 8 job(s) in queue, 0 running.

@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (b1a3e6d)

@nicolasstucki nicolasstucki force-pushed the add-cache-to-quote-unpickling branch from 97d747a to 00115c6 Compare April 28, 2021 16:19
@nicolasstucki
Copy link
Contributor Author

test performance with #quotes please

@dottybot
Copy link
Member

performance test scheduled: 1 job(s) in queue, 1 running.

@dottybot
Copy link
Member

Performance test finished successfully:

Visit https://dotty-bench2.epfl.ch/12242/ to see the changes.

Benchmarks is based on merging with master (2d2038e)

@nicolasstucki nicolasstucki marked this pull request as ready for review April 29, 2021 05:47
Copy link
Contributor

@liufengyun liufengyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

// Copy the cached tree to make sure the all definitions are unique.
TreeTypeMap(oldOwners = List(owner), newOwners = List(owner)).apply(tree)
case _ =>
tree
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case we have optimizations that make the TreeTypeMap and identity function, do we have a test to guard against such optimizations?

Copy link
Contributor Author

@nicolasstucki nicolasstucki Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already do that optimization outside TreeTypeMap, for example in changeOwner.

We do have tests that would fail if this stops cloning the symbols.

@nicolasstucki nicolasstucki merged commit 48d0999 into scala:master Apr 29, 2021
@nicolasstucki nicolasstucki deleted the add-cache-to-quote-unpickling branch April 29, 2021 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

QuoteUnpicker.unpickleExpr and QuoteUnpicker.unpickleType are slow
4 participants