Skip to content

Fix smoother to remove a non-invertible design matrix bug #476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 11, 2020

Conversation

dshemetov
Copy link
Contributor

@dshemetov dshemetov commented Nov 10, 2020

The error arose in the case of imputation with fewer datapoints than polynomial degree. Fixes #448. I also modified the imputation to change the polynomial fit degree at that boundary edge case and to use quadratic fitting by default (this is because imputing with polynomial_fit_degree=0 does a really poor job, so shouldn't ever be used).

predicted_value = signal @ coeffs
return predicted_value

def savgol_coeffs(self, nl, nr):
def savgol_coeffs(self, nl, nr, poly_fit_degree):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add docstring for the new param.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

signal_imputed[ix - self.window_length : ix] @ coeffs
signal_imputed[ix] = self.savgol_predict(
signal_imputed[ix - self.window_length : ix],
2,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How big of a difference is it using a quadratic polynomial vs the previous window_length degree polynomial (assuming im reading this right)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, now I think I should make a different PR and start a discussion there for this. This change is unrelated to the matrix bug.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took this change out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍. the rest of the PR looks good. One thing that I like to do when fixing a bug is write out the test case that would cause the bug and code until it passes. Then just leave it in as a test case and you have some confidence that it's resolved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea, I'll try that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, will approve once a test is added

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And a few other bad input tests for the sake of code coverage (couldn't help myself :P).

@dshemetov dshemetov force-pushed the fix_smoother_test branch 4 times, most recently from b1de029 to 7a13567 Compare November 10, 2020 23:39
Copy link
Contributor

@chinandrew chinandrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: :shipit: :shipit:

@krivard krivard merged commit caba8c3 into main Nov 11, 2020
@krivard krivard deleted the fix_smoother_test branch November 11, 2020 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix matrix inversion bug on smoother unit tests
3 participants