Skip to content

Copy model-related shared variables in model_fgraph #218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 25, 2023

Conversation

ricardoV94
Copy link
Member

@ricardoV94 ricardoV94 commented Jul 24, 2023

In earlier iterations of the do-blogpost it became clear that it's more intuitive for model-related shared variables to be copied, instead of shared across different models. Some of these variables are created by PyMC (dim_lengths) and users would have to know the internals not to be surprised about the behavior.

User defined MutableData is also copied, because unlike the underlying SharedVariables, nothing about the name and documentation suggests these variables are supposed to be "shared" across models (just that they can be mutated).

I have only allowed user-defined shared variables that are not model variables to be actually shared across model cloning.

This now works as (imo) expected:

import pymc as pm
from pymc_experimental.model_transform.conditioning import do

with pm.Model(coords_mutable={"i": [0]}) as m:
    x = pm.Normal("x")
    y = pm.Normal("y", x, dims=("i",))
    
new_m = do(m, {x: 0})
new_m.set_dim("i", new_length=5, coord_values=list(range(5)))

assert pm.draw(m["y"]).shape == (1,)  # Before this would also be (5,)
assert pm.draw(new_m["y"].shape) == (5,)

This PR also fixes a bug, as RNG shared variables weren't actually being copied and rendered independent with the old clone approach. The relevant test was missing an assert :( (see first commit of this PR)

@ricardoV94 ricardoV94 added bug Something isn't working enhancements New feature or request labels Jul 24, 2023
@ricardoV94 ricardoV94 changed the title Copy model-related shared vars in model_fgraph Copy model-related shared vars in model_fgraph Jul 24, 2023
twiecki
twiecki previously approved these changes Jul 24, 2023
Copy link
Member

@lucianopaz lucianopaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great work @ricardoV94

@@ -184,8 +184,7 @@ def fgraph_from_model(

# Replace RNG nodes so that seeding does not interfere with old model
for rng in find_rng_nodes(model_vars):
new_rng = rng.clone()
new_rng.set_value(rng.get_value(borrow=False))
new_rng = shared(rng.get_value(borrow=False))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why the old approach didn't work. You were creating a clone, then setting its value to a new generator. Why would wrapping a new shared around a generator work and the old approach not.

Copy link
Member Author

@ricardoV94 ricardoV94 Jul 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SharedVariable.clone reuses the same container. The only thing that's "cloned" is the Variable object: https://github.com/pymc-devs/pytensor/blob/673c1accf98659ba0457759431cb36eef4659f63/pytensor/compile/sharedvalue.py#L139-L149

memo[rng] = new_rng
# Replace the following shared variables in the model:
# 1. RNGs
# 2. MutableData (could increase memory usage significantly)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll have to wait and get some feedback. If people complain about this, we can review the memory sharing.

@ricardoV94 ricardoV94 changed the title Copy model-related shared vars in model_fgraph Copy model-related shared variables in model_fgraph Jul 25, 2023
@ricardoV94 ricardoV94 merged commit 4051624 into pymc-devs:main Jul 25, 2023
@ricardoV94 ricardoV94 deleted the copy_shared_vars branch August 7, 2023 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancements New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants