Skip to content

NullPointerException in PagedSeq/StreamReader #70

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
alanbur opened this issue Sep 18, 2015 · 10 comments
Closed

NullPointerException in PagedSeq/StreamReader #70

alanbur opened this issue Sep 18, 2015 · 10 comments
Labels
bug

Comments

@alanbur
Copy link

alanbur commented Sep 18, 2015

I'm getting a NullPointerException when using the Scala JSON parser on a stream. If I read the contents of the stream into a String and parse that instead, then it completes with no error. That suggests it's not an error in the JSON parser, it is in the StreamReader and/or PagedSeq collection classes. The stack trace is:

Exception in thread "main" java.lang.NullPointerException
    at scala.collection.immutable.PagedSeq.slice(PagedSeq.scala:192)
    at scala.collection.immutable.PagedSeq.slice(PagedSeq.scala:205)
    at scala.util.parsing.input.StreamReader.rest(StreamReader.scala:56)
    at scala.util.parsing.input.StreamReader.rest(StreamReader.scala:48)
    at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:599)
    at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:597)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$$anonfun$rep1$1.applyp$1(Parsers.scala:718)
    at scala.util.parsing.combinator.Parsers$$anonfun$rep1$1.continue$1(Parsers.scala:724)
    at scala.util.parsing.combinator.Parsers$$anonfun$rep1$1.apply(Parsers.scala:728)
    at scala.util.parsing.combinator.Parsers$$anonfun$rep1$1.apply(Parsers.scala:712)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:249)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:249)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.lexical.Scanners$Scanner.<init>(Scanners.scala:45)
    at scala.util.parsing.combinator.lexical.Scanners$Scanner.rest(Scanners.scala:58)
    at scala.util.parsing.combinator.lexical.Scanners$Scanner.rest(Scanners.scala:42)
    at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:599)
    at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:597)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:237)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:237)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:237)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:237)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$Success.flatMapWithNext(Parsers.scala:143)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:234)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:234)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:237)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:237)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:249)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:249)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:217)
    at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:882)
    at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:882)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
    at scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:881)
    at com.oracle.psarc.common.JSONParser.parseRaw(JSONParser.scala:31)
    at com.oracle.psarc.common.JSONParser.parseFull(JSONParser.scala:42)
    at com.oracle.psarc.common.PathConfig$.load(PathConfig.scala:35)
    at IndexMain$.main(IndexMain.scala:27)
    at IndexMain.main(IndexMain.scala)
@liskin
Copy link
Contributor

liskin commented Sep 18, 2015

Seems to be a problem in PagedSeq. StreamReader calls seq.slice(off + 1) after only checking that seq.isDefinedAt(off) (which should be okay since otherwise there would be no way to create an empty slice) and PagedSeq fails to check if addMore actually adds more. I'll try to fix PagedSeq.

@alanbur
Copy link
Author

alanbur commented Sep 19, 2015

Thanks for such a quick fix But it appears Seth Tisue has deleted https://issues.scala-lang.org/browse/SI-9480, which I believe is the bug you submitted your pull request with the fix against...

@liskin
Copy link
Contributor

liskin commented Sep 19, 2015

Yeah, he did. Don't know why. @SethTisue, would you mind elaborating a bit? Anyway, the fix is here: scala/scala#4753

@SethTisue
Copy link
Member

I apologize. I messed this up. I hastily assumed this would turn out to be a parser combinators bug, and when you discovered it wasn't, I failed to register that, so I thought the original ticket had no further value.

I'll comment on the PR about our options now.

@SethTisue
Copy link
Member

@Ichoran, all: perhaps we should deprecate PagedSeq in 2.12? Is it an obscure, special-purpose thing that probably doesn't have other clients besides scala.util.parser.combinators?

@SethTisue SethTisue mentioned this issue Sep 19, 2015
Closed
@Ichoran
Copy link

Ichoran commented Sep 21, 2015

@SethTisue - I would recommend deprecating it if we can. It is a pretty weirdly-behaved collection even when it isn't throwing NPEs. I don't know its usage pattern, though.

lrytz pushed a commit to lrytz/scala that referenced this issue Sep 21, 2015
@liskin @lrytz
Fix NPE in PagedSeq.slice at end of seq
0fa6787 
See scala/scala-parser-combinators#70

Basically the same thing as SI-6615, including the fact everything
works okay if the PagedSeq is printed before calling slice.

It might seem strange that this allows taking slices that start beyond
the end, but

 - this was possible anyway if one forced the entire sequence, and

 - it is reasonable to be able to take a slice at the very end (not
   beyond it) and get an empty sequence, which is exactly what
   StreamReader in scala-parser-combinators does and gets an NPE.
lrytz pushed a commit to lrytz/scala that referenced this issue Sep 21, 2015
@liskin @lrytz
Fix NPE in PagedSeq.slice at end of seq
8cca2bc 
See scala/scala-parser-combinators#70

Basically the same thing as SI-6615, including the fact everything
works okay if the PagedSeq is printed before calling slice.

It might seem strange that this allows taking slices that start beyond
the end, but
 - this was possible anyway if one forced the entire sequence, and
 - it is reasonable to be able to take a slice at the very end (not
   beyond it) and get an empty sequence, which is exactly what
   StreamReader in scala-parser-combinators does and gets an NPE.
@lrytz lrytz mentioned this issue Sep 21, 2015
Merged
@SethTisue
Copy link
Member

I started a scala-language thread on the possible deprecation: https://groups.google.com/d/msg/scala-language/gq8iueZmjjg/QQGFZIZsAAAJ

@gourlaysama
Copy link
Contributor

This was fixed in 2.11.8 (scala/scala#4758).

@SethTisue
Copy link
Member

(note that it would still be nice to have #71 addressed so that PagedSeq can be removed from Scala 2.13)

@gourlaysama
Copy link
Contributor

Yes, I'm looking into what it can be replaced with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@SethTisue @liskin @alanbur @gourlaysama @Ichoran