Merge pull request #81 from SethTisue/december-2018-minutes

December 2018 advisory board minutes

Author: SethTisue

minutes/_posts/2018-12-05-december-5-2018.md

@@ -0,0 +1,392 @@
+---
+layout: contact
+---
+
+# Minutes of the 11th meeting of the Scala Center, Q4 2018
+
+Minutes are [archived](https://scala.epfl.ch/records.html) on the
+Scala Center website.
+
+## Summary
+
+The following agenda was distributed to attendees:
+[agenda](https://github.com/scalacenter/advisoryboard/blob/master/agendas/011-2018-q4.md).
+
+Spotify has joined the advisory board.  Julien Tournay is
+their representative.
+
+After this meeting, Lars Hupel is stepping down as community
+representative.  A replacement has yet to be nominated.
+
+Scala Center activities for the past quarter focused on
+Coursier, Almond (formerly Jupyter-Scala), Bloop and BSP,
+Zinc, build pipelining, 2.13 collections, Dotty, Metals,
+Scalameta, Scalafix, Scala.js, SIP meetings, MOOCs,
+Scala Days 2019, and ScalaIO.
+
+Full details on these activities are in
+[Sébastien's report](https://docs.google.com/document/d/1MwzPQtYIaJ4_5z7-StJI-gKiY-oV_6Or7PEvnJBR92k/edit?usp=sharing).
+
+Other topics discussed at the meeting included the Center's
+sponsorship terms.
+
+Two new proposals were made:
+
+* [SCP-019](https://github.com/scalacenter/advisoryboard/blob/master/proposals/019-scala-214-30-back-compat.md): Focusing on backwards compatibility for Scala 2.14 and Scala 3.0 (Twitter)
+    * voted on, not accepted
+* [SCP-020](https://github.com/scalacenter/advisoryboard/blob/master/proposals/020-sbt-transitive-dependencies-conflicts.md): SBT transitive dependency conflicts management improvements (Spotify)
+    * voted on and accepted
+
+SCP-20 was exceptionally accepted as a late submission
+from Spotify, as they were only confirmed as board members
+between the submission deadline and the meeting.
+
+## Date, Time and Location
+
+The meeting took place virtually, via Google Meet, on Wednesday,
+December 5, 2018 at 3:30pm (GMT).
+
+Minutes were taken by Seth Tisue (secretary).
+
+## Attendees
+
+Board members present:
+
+- James Belsey, Morgan Stanley
+- Eugene Burmako and Stu Hood, Twitter
+- Lars Hupel, community/Typelevel
+- Olga Makhasoeva, 47 Degrees
+- Adriaan Moors, Lightbend
+- Jonathan Perry, Goldman Sachs
+- Frederick Reiss, IBM
+- Julien Tournay, Spotify
+- Mark Waks (a.k.a. Justin du Coeur), community/Artima, on behalf of Bill Venners
+
+Apologies:
+
+- Thomas Gawlitza, SAP
+
+Officers:
+
+- Sébastien Doeraene (director), EPFL
+- Jon Pretty (chairperson), Propensive
+- Martin Odersky (technical advisor), EPFL
+- Seth Tisue (secretary), Lightbend
+
+## Proceedings
+
+As chairperson, Jon Pretty conducted the meeting.
+
+He welcomed Spotify as the newest advisory board member, and Julien
+Tournay as their representative.  Julien works for Spotify in Sweden.
+
+Jon also welcomed Olga Makhasoeva as 47 Degrees' new representative
+on the board.
+
+Filling in for Bill Venners at this meeting was Mark Waks, Bill's
+colleague at Artima.
+
+### Activities
+
+As the Center's Executive Director, Sébastien Doeraene summarized
+Scala Center activities since the last meeting.
+
+Most of Sébastien's remarks were based on his
+[detailed report](https://docs.google.com/document/d/1MwzPQtYIaJ4_5z7-StJI-gKiY-oV_6Or7PEvnJBR92k/edit?usp=sharing)
+on the Center's recent activities.
+
+The following notes are a supplement to Sébastien's report:
+
+Coursier isn't itself new, but it is a new project for the Scala
+Center.  Alexandre Archambault, who is now a full-time engineer at the
+Scala Center, is Coursier's original open-source developer.  Seb
+highlighted Alex's plan to add publishing support to Coursier and
+eventually integrate Coursier into sbt more directly as a faster
+replacement for Ivy.  "It's a big enabler" of productivity, "basic
+speeds of day-to-day tools" that Scala developers use.  A new
+documentation site for Coursier will be released "soon".
+
+Martin Duhemm has left the Scala Center.  The Dotty IDE work described
+in Seb's report was Martin's last project.
+
+Seb's report covers two different IDE projects, namely Dotty IDE and
+Metals.  Unlike Dotty IDE, Metals aims to support Scala 2. The two
+projects' features and UI "will be reconciled in the coming months,"
+says Seb.  Metals "doesn't have a lot of features" yet but "the
+features it has are really polished" and "it's really simple to
+install and to import projects."  The goal is to add features
+gradually, "to do one thing at a time but do it really well."
+
+Jon asked what quality level that Metals has attained so far.  Seb
+said that the handling of compiler diagnostics is already "perfect".
+The go-to-definition feature has good test coverage, but users may
+still uncover unknown bugs.  Importing builds, via Bloop, may fail on
+especially complex builds.  Julien offered that Metals "worked
+perfectly" so far on a 20-module project he's been trying it with,
+though it "takes a while" to initialize.
+
+Jon asked about Scala Days sponsorships.  Seb said that interested
+sponsors should get in touch with Darja at the Scala Center.
+
+Eugene asked what's next for build pipelining.  Seb said that Jorge
+will continue to work on it, next steps in that work include support
+for incremental compilation and mixed Scala/Java codebases.  "Once
+it's usable for incremental compilation we'll make a bigger fuss about
+it and advertise it much more widely," said Seb.
+
+Eugene also asked about the status of Dotty IDE.  Seb returned to the
+question of aligning the work on Dotty IDE with the work on Metals.
+In many ways they are similar (for example, they are both LSP based
+and both integrate with VS Code and other editors).  But right now the
+user must install "one or the other", installing both tends to "not
+quite work."  These projects need to become "one concerted effort"
+with a single unified download, which would then operate in either
+Scala 2 or Dotty mode depending on what kind of project you have.
+
+Stu asked about the status of
+[SCP-018](https://github.com/scalacenter/advisoryboard/blob/master/proposals/018-converging-214-30.md)
+("Converging the Intermediate Representation of Scala 2.14 and Scala
+3.0"), "was anyone assigned to work on that yet?"  Seb said the Center
+is looking for "a new person we could assign this project to."  (If a
+new person can't start soon, someone else might be assigned in the
+meantime.)
+
+### Financial report
+
+"We got the final numbers for the MOOC income for Q1 through Q3 of
+2018," said Seb.  This totaled just under 300K CHF, of which the
+Center's share is 215K CHF.  (The CHF/USD exchange rate is currently
+1-to-1.)  This is similar to past proceeds, so that part of the budget
+is "stable".
+
+With Spotify joining the board, there are now eight member companies.
+There may be enough budget to open an additional position, besides the
+one vacated by Martin Duhemm, but it's not certain yet.
+
+Jon asked if there was any possibility of using extra money to
+increase salaries, to help ensure retention of engineers.  Seb said
+that EPFL's rules around compensation are so strict that there's
+really no leeway.
+
+Jon also asked if the Scala Days 2019 (in Lausanne) has any financial
+risks or financial opportunities for the Center.  Seb said that "the
+finances of Scala Days are a completely separate budget".  Because
+there is only one Scala Days in 2019, a larger hall was obtained, on
+the assumption that attendance will be higher than two separate
+conferences on either side of the Atlantic.  The break-even point is
+900 attendees, which is not very many more than attended Scala Days
+Berlin in 2018.
+
+### Proposals
+
+Both proposals were presented discussed before either proposal was voted on.
+
+#### SCP-019: Focusing on backwards compatibility for Scala 2.14 and Scala 3.0
+
+Proposed by Stu Hood on behalf of Twitter.
+
+The theme of the proposal is the relative priority, during the transition
+of Scala 3, of backward compatibility (Scala 3's ability to consume
+libraries built using Scala 2) versus forward compatibility (Scala 2's
+ability to consume libraries built using Scala 3).
+
+See the [proposal text](https://github.com/scalacenter/advisoryboard/blob/master/proposals/019-scala-214-30-back-compat.md)
+for details.
+
+There was some discussion on the [pull request for the
+proposal](https://github.com/scalacenter/advisoryboard/pull/44)
+
+between Martin and Stu. This resulted in some [revisions to the
+proposal](https://github.com/scalacenter/advisoryboard/pull/47)
+shortly before the meeting.  These changes are included in the
+proposal text linked above, and the updated version is also what the
+board finally voted on.
+
+In presenting the proposal, Stu noted that an alternative to forward
+compatibility already exists, namely cross-compilation.  "Libraries
+can do what they already do," namely cross-compile and cross-publish,
+in this case against both Scala 2 and 3, as long as they only use the
+common subset of the language (which encompasses nearly everything in
+Scala 2).
+
+"This is not an anti-TASTy proposal", Stu said.  "This is a proposal
+to use whatever means necessary to allow backwards compatibility."  If
+Scala 3's support for Scala 2's pickle format is good enough to get
+good backward compatibility, then so be it.  But if Scala 2 emitting
+TASTy turns out to be a better way to achieve backward compat, then so
+be it also.  (Whereas Scala 2.14's proposed ability to _consume_ TASTy
+is less relevant to backwards compat.)
+
+Martin commented on the proposal.  He described Dotty's already
+existing Scala 2 pickle support as "a solved problem."  "Every Dotty
+program ever written uses Scala 2 pickle support" and it's never been
+a problem, so Martin hopes that the backward compatibility problem is
+actually already solved, but does that still need to be validated
+further?  To more fully validate it, we need to test it with "a lot of
+code".  There are two ways to get that much code, one is to wait for
+the community to catch up and write a lot of Scala 3 code, but that
+takes a long time.  Or you port a lot of existing Scala 2 code, but
+that takes effort, especially for porting macros, but also (for
+example) if the code hits differences in type inference.  The Scala
+Center can help Scala 2 projects port their code when the time comes,
+but in the meantime there isn't anything to do to improve binary
+compatibility, so Martin sees no need to do deprioritize the
+forward-compatibility parts of SCP-018.
+
+Jon asked Adriaan what the impact on Lightbend's 2.14 efforts would
+be.  Adriaan addressed that and also commented more generally on the
+new proposal. "The key point is that forward compatibility is also
+important" for Scala 3 "because in order to identify potential
+challenges with Scala 3, we need to be able to link both ways.
+Applications might not upgrade to Scala 3 right away, but libraries
+may be eager to upgrade."
+
+Adriaan also said that SCP-018 was intentionally "pretty vague" about
+the exact nature of the best path forward.  "It's the task of the Scala
+Center" to figure out exactly what to do, depending on what engineers
+are available, and any other factors that arise as the work progresses.
+Seb agreed that this new proposal (SCP-019) "is directing, more than
+usual, what the Center should do and how we should do it."
+
+Eugene asked what barriers exist, or are expected to exist, for
+cross-compiling libraries against Scala 2 and Scala 3.  Understanding
+that better could help identify the best approach to compatibility.
+Stu added that the Dotty community build is currently "leaf nodes",
+that is, libraries with few or no dependencies.  We won't know more
+about compatibility until it's expanded to cover inter-library
+dependencies better.  And Jon observed that we need to test
+applications that depend on the libraries, too.
+
+Martin says the blocker, currently, for expanding the scope of the
+Dotty community build is macros.  "Transitive dependencies using
+macros are absolutely everywhere as you go up the stack. That's
+why it's so difficult. Bottom dependencies need to be adapted to
+use new-style macros" before we can test downstream projects.
+So yes "there are porting problems", but "they are not related to
+binary compatibility. Macros are the problem."  But if the Scala
+Center can identify key macro-based libraries and help port them,
+that will be a "big win".
+
+Mark added that ScalaTest is currently being ported to Dotty, so that
+will be a "big, chewy test" of a widely-used library that is
+macro-based.  He also mentioned that we need to keep in mind that the
+world of Scala development at companies is very different than what we
+see in the open-source world.  For example, relatively few companies
+are writing their own macros, even though open-source macro-based
+libraries are widely used everywhere.  So we'll need to keep that in
+mind when gathering data on compatibility.
+
+Martin agrees with Stu that producing a "compatibility reference" is
+needed, so that the community has the information it needs to
+cross-compile and/or cross-consume between Scala 2 and 3, but he sees
+producing this as primarily LAMP's and Lightbend's job, rather than
+primarily the Scala Center's job.
+
+Seb explained some the motivation for valuing forward compatibility,
+as he sees it, though "not taking sides". He explained, "If we only
+support 3 depends on 2, but not the other way around, then library
+developers are stuck on 2 for the next 5 years, since if their
+codebase must cross-compile, they cannot benefit from Scala 3's
+features.  That means that in practice, the visible Scala ecosystem
+would remain on Scala 2 for a long time, and that could send a
+worrying message to the public."  Martin seconded this.  He said most
+of the users he talks to who are worried about Scala 3 are worried
+about slow adoption, but then the forward-compatibility part of
+SCP-018 strongly reassures them.
+
+Some complex discussion about macros and the overall compatibility
+followed, difficult to summarize.  The key point is that neither
+forward compatibility nor backwards compatibility provide macros
+that work across both Scala 2 and Scala 3.
+
+It's clear that library authors and users will need guidance to
+understand what will or won't work in what scenarios, and when it
+matters how macros are involved.
+
+Jon noted that even if SCP-019 isn't accepted, that doesn't close the
+door on other future proposals aimed at refining SCP-018, as the
+overall Scala 3 situation develops further.
+
+**Voting**: three in favor, four opposed (with Seb casting the
+tie-breaking vote, and promising to "take the proposal into account"
+regardless), three abstentions, one member not present. The proposal
+is not accepted.
+
+#### SCP-020: SBT transitive dependency conflicts management improvements
+
+Proposed by Julien Tournay on behalf of Spotify.
+
+See the [proposal text](https://github.com/scalacenter/advisoryboard/blob/master/proposals/020-sbt-transitive-dependencies-conflicts.md)
+for details.
+
+In presenting the proposal, Julien said that library version conflicts
+have been a recurring problem in deploying code at Spotify.  Sometimes
+"we only realize there is a conflict when we run our data pipelines,"
+sometimes not until after the job has been running for hours.
+
+Lars asked to what extent the proposed work on sbt should overlap or
+interact with the work the Center is now doing on Coursier.
+
+Julien said that Coursier isn't the default in sbt, so for now, "most
+builds" would have the same issue even if it were fixed in Coursier.
+But if Coursier fixes it and also becomes the de facto standard for Scala
+builds, then yes, "done."
+
+Seb observed that sbt does already print eviction warnings.  Lars
+asked how a build tool can do better than that, given the lack of an
+accepted standard way for libraries to signal binary
+compatibility. Not all libraries use semantic versioning, or do it the
+same way.  "The reason I sound pessimistic," said Lars, "is that I'm
+concerned about false positives," a tool reporting conflicts where no
+incompatibility exists.  But the proposals seems to give "leeway" to
+the Scala Center to do whatever's feasible and helpful.
+
+Seb suggested perhaps adding a "2b", between levels 2 and 3 in the
+proposal, which would involve "a more elaborate API for conflict
+resolution", since `ConflictManager`, even if it worked perfectly as
+designed, arguably isn't a flexible enough mechanism, for the same
+reasons that Lars brought up: "you want to resolve conflicts in a
+different way for different libraries."
+
+Mark spoke up in support of the proposal, because the problem is real,
+and the proposal gives the Scala Center flexibility to choose how to
+address it. "At least level 1 would totally be helpful."  Perhaps
+plugins such as sbt-dependency-graph could become part of sbt.
+
+Jonathan and James both expressed some doubt about the usefulness, in
+their shops at least, of upping the overall level of "intricacy"
+around dependencies, especially in polyglot environments, by
+introducing yet more conflict resolution rules and mechanisms.
+
+**Voting**: seven in favor, one opposed, one abstention,
+one member not present. The proposal is accepted.
+
+### Community feedback
+
+Lars Hupel announced that he is stepping down as community
+representative; Jon thanked him for his more than two years of
+service.  A replacement has yet to be nominated.  (Last time around,
+the spot was offered to Typelevel to fill; Jon anticipated that
+happening again, but it's up to the board.)
+
+Mark said that "he's glad to see a focus on tooling, since that is still
+the thing that comes up most often" in his conversations with community
+members.
+
+### Other business
+
+At the previous meeting, there was some discussion about perhaps
+changing the Center's sponsorship terms.  Jon asked Seb if this has
+been considered further yet, and also asked if it would be good to
+create a concept of a "developer sponsor" who would pay a smaller fee
+but make up for it by contributing development work directly.  But the
+board agreed there wasn't really time to discuss it.  James asked if
+the Center could make specific suggestions, to be voted on at the next
+meeting.  Seb agreed to raise the topic on the board's mailing list,
+for discussion in advance of the next meeting.
+
+## Conclusion
+
+Jon said that as usual, the next meeting will be in approximately
+three months, perhaps early march.  Then the meeting after that will
+be held in Lausanne, in conjunction with Scala Days.