-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Do not include rvs in symbolic normalizing constant #7787
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not include rvs in symbolic normalizing constant #7787
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR ensures that random variables (RVs) are not included in the symbolic normalizing constant graph by folding shapes to constants, adds shape inference for minibatch RVs, includes a test for the new behavior, and fixes a small typo.
- Use
constant_fold
to derive batch shapes instead of carrying RVs intosymbolic_normalizing_constant
- Implement
infer_shape
onMinibatchRandomVariable
so shape propagation works correctly - Add a dedicated test (
assert_no_rvs
) to confirm no RVs appear in the symbolic normalizing constant - Correct a typo in the
constant_fold
comment
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
File | Description |
---|---|
tests/variational/test_opvi.py | Added test_symbolic_normalizing_constant_no_rvs with assert_no_rvs |
pymc/variational/opvi.py | Swapped direct .shape usage for constant_fold([...].shape) in scaling |
pymc/variational/minibatch_rv.py | Added infer_shape method to propagate shapes without evaluation |
pymc/pytensorf.py | Fixed typo in comment (constand_folding → constant_folding ) |
Comments suppressed due to low confidence (3)
tests/variational/test_opvi.py:284
- [nitpick] The test verifies no RVs are in the graph but doesn't assert that the symbolic normalizing constant still produces the expected scalar or tensor shape. Consider adding an assertion on the returned value or shape to guard against regressions.
def test_symbolic_normalizing_constant_no_rvs():
pymc/variational/opvi.py:1109
- Calling
constant_fold
inside the list comprehension for each RV will repeatedly clone and rewrite the graph, which may be costly. Consider computing all shapes once (e.g., collect inputs, callconstant_fold
outside the loop) or caching results before the comprehension.
get_scaling(
pymc/variational/opvi.py:1279
- This mirrored use of
constant_fold
in another list comprehension also risks redundant graph rewriting. Extract a helper or hoist the folding step to improve efficiency and reduce duplicated logic.
get_scaling(
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #7787 +/- ##
=======================================
Coverage 92.84% 92.84%
=======================================
Files 107 107
Lines 18378 18380 +2
=======================================
+ Hits 17063 17065 +2
Misses 1315 1315
🚀 New features to boost your workflow:
|
2551adf
to
0867cde
Compare
This is a more proper fix for the problem highlighted in #7778
The normalizing constant for MinibatchRVs included the graph of the shape of the RVs.
Even though the shape of the MinibatchRV can be derived without evaluating the draws, passing any graph with RVs to
pytensorf.compile
will automatically integrate the updates which requires evaluating the RV anyway. This PR makes sure we don't include the RVs only to get the symbolic normalizing constant.📚 Documentation preview 📚: https://pymc--7787.org.readthedocs.build/en/7787/