newey-west adjustment not working properly in OLS

[syduan]

It looks newey-west adjustment is not working properly in OLS when 'cluster' is set to 'time' or 'entity'. Specifically, pandas.stats.plm.py lines 791-794 don't have any effect. Should that be replaced with: xox = math.newey_west(m, nw_lags, nobs, df, nw_overlap)?

Here is some code to reproduce the issue.

import numpy
from pylab import *
from pandas import *

T = 100
panel_size = 3
data_dimensions = [T, panel_size]

xs_per_y = WidePanel({
'predictor a' : numpy.random.normal(size=data_dimensions),
'predictor b' : numpy.random.normal(size=data_dimensions)
})

y = B_a + B_b + noise

ys = xs_per_y['predictor a'] + xs_per_y['predictor b'] + numpy.random.normal(size=data_dimensions)

print ols(y=ys, x=xs_per_y, pool=True, cluster = 'time')

we expect the following t-stats to be smaller, but they are the same as the previous OLS

print ols(y=ys, x=xs_per_y, pool=True, cluster = 'time', nw_lags=10)

[jseabold]

You should prefer statsmodels for statistical modeling. AFAIK, there's no one actively maintaining this code and the plan has been to remove it completely from pandas once everything is also available in statsmodels. @josef-pkt has been recently working on Newey-West and friends. Maybe he can show how to achieve this in statsmodels.

[josef-pkt]

I need to see whether I have examples lying around a bit later.

in statsmodels 0.5 we had http://statsmodels.sourceforge.net/devel/stats.html#sandwich-robust-covariances
which sometimes are a bit picky on specifying the cluster and time index.

current master has new method in OLSResults
http://statsmodels.sourceforge.net/devel/generated/statsmodels.regression.linear_model.OLSResults.get_robustcov_results.html

The options are strongly modeled after Stata. examples are in the test suite. But there is one more round of refactoring coming up. Feedback very welcome.

We don't have the other Panel data support yet in master.

[syduan]

Thanks.

[syduan]

Please let us know when you find an example to do both clustering and newey-west.

[syduan]

Pardon, what part is not maintained, and going to drop completely from pandas?

The entire ols function, or some specific part for Panel errors?

[jseabold]

All of the modeling code. So, yes, OLS, and the panel errors. You should use statsmodels for this in the future.

PS. Please trim your e-mails to keep the git issue readable.

[jreback]

closing as going to defer to statsmodels

in 0.14 @jseabold should we deprecate ols / fama_mcbeth ? (or maybe just a warning that these are more supported in statsmodels?)

[jseabold]

Hmm, I wouldn't deprecate until we're sure that everything is also supported in statsmodels. Opening an issue to move the code and deprecate would be a start. Any help here appreciated. I have no idea when I'll be able to find time to do this.

[jreback]

hahh...ok...will create an issue for it....pls feel free to make a check box of tasks

[jreback]

seems merged with 0 changes. maybe the old github bug rearing.

[jreback]

closed by #8360