Skip to content

Commit 4123d84

Browse files
committed
Initial version of parallel collections from the Scala standard library
Moved over from scala/scala @ 36d6e48da258648413e50e5a9d46c94cfc062634 with the necessary adaptations to make it work as a separate module on top of scala/scala#5603. - Since the sequential collections cannot implement `Parallelizable` anymore, all conversions have to be performed via extension methods provided by `scala.collection.parallel.CollectionConverters._` - `ParIterableLike.sequentially` was changed to use a `newCombiner` for building the parallel version from the temporary sequential collection. - `Combiner` has a new convenience method `fromSequential`. `par` in `Parallelizable` is now implemented on top of this. - All parallel collections tests are now in two subprojects, for JUnit and ScalaCheck tests respectively. The main codebase is moved to the `core` subproject. We are currently using a ScalaCheck build for 2.12 but it will become incompatible once we start getting serious with the new collections library, so this is not a long-term solution. We may have to insource ScalaCheck into this project as well, just as we did for scala/scala. At least the separation of ScalaCheck and JUnit tests will make it easy to temporarily disable the ScalaCheck tests if no compatible ScalaCheck build is available. - Running on the old Travis infrastructure to avoid test failures (see travis-ci/travis-ci#3775) - The build-related parts were copied from https://github.com/scala/scala-swing.
1 parent 22d8d9e commit 4123d84

File tree

155 files changed

+1219
-4821
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

155 files changed

+1219
-4821
lines changed

.gitignore

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#
2+
# Are you tempted to edit this file?
3+
#
4+
# First consider if the changes make sense for all,
5+
# or if they are specific to your workflow/system.
6+
# If it is the latter, you can augment this list with
7+
# entries in .git/info/excludes
8+
#
9+
10+
*.jar
11+
*~
12+
13+
# eclipse, intellij
14+
/.classpath
15+
/.project
16+
/.cache
17+
/.idea
18+
/.settings
19+
20+
# bak files produced by ./cleanup-commit
21+
*.bak
22+
23+
# Mac specific, but that is common enough a dev platform to warrant inclusion.
24+
.DS_Store
25+
26+
target/

.travis.yml

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
language: scala
2+
3+
sudo: required
4+
5+
env:
6+
global:
7+
- PUBLISH_JDK=oraclejdk8
8+
9+
script: admin/build.sh
10+
11+
jdk: oraclejdk8

LICENSE

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
Copyright (c) 2002-2016 EPFL
2+
Copyright (c) 2011-2016 Lightbend, Inc.
3+
4+
All rights reserved.
5+
6+
Redistribution and use in source and binary forms, with or without modification,
7+
are permitted provided that the following conditions are met:
8+
9+
* Redistributions of source code must retain the above copyright notice,
10+
this list of conditions and the following disclaimer.
11+
* Redistributions in binary form must reproduce the above copyright notice,
12+
this list of conditions and the following disclaimer in the documentation
13+
and/or other materials provided with the distribution.
14+
* Neither the name of the EPFL nor the names of its contributors
15+
may be used to endorse or promote products derived from this software
16+
without specific prior written permission.
17+
18+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
19+
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
20+
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
21+
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
22+
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
23+
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
24+
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
25+
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
26+
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
27+
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
28+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Scala Parallel Collections
2+
3+
This Scala standard library module contains the package `scala.collection.parallel` with all parallel collections that
4+
used to be part of `scala-library` before Scala 2.13.

admin/README.md

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
## Tag Driven Releasing
2+
3+
Copied from https://github.com/scala/scala-java8-compat/commit/4a6cfc97cd95227b86650410e1b632e5ff79335b.
4+
5+
### Background Reading
6+
7+
- http://docs.travis-ci.com/user/environment-variables/
8+
- http://docs.travis-ci.com/user/encryption-keys/
9+
- http://docs.travis-ci.com/user/encrypting-files/
10+
11+
### Initial setup for the repository
12+
13+
To configure tag driven releases from Travis CI.
14+
15+
1. Generate a key pair for this repository with `./admin/genKeyPair.sh`.
16+
Edit `.travis.yml` and `admin/build.sh` as prompted.
17+
2. Publish the public key to https://pgp.mit.edu
18+
3. Store other secrets as encrypted environment variables with `admin/encryptEnvVars.sh`.
19+
Edit `.travis.yml` as prompted.
20+
4. Edit `.travis.yml` to use `./admin/build.sh` as the build script,
21+
and edit that script to use the tasks required for this project.
22+
5. Edit `.travis.yml` to select which JDK will be used for publishing.
23+
24+
It is important to add comments in .travis.yml to identify the name
25+
of each environment variable encoded in a `:secure` section.
26+
27+
After all of these steps, your .travis.yml should contain config of the
28+
form:
29+
30+
language: scala
31+
env:
32+
global:
33+
- PUBLISH_JDK=openjdk6
34+
# PGP_PASSPHRASE
35+
- secure: "XXXXXX"
36+
# SONA_USER
37+
- secure: "XXXXXX"
38+
# SONA_PASS
39+
- secure: "XXXXXX"
40+
script: admin/build.sh
41+
42+
If Sonatype credentials change in the future, step 3 can be repeated
43+
without generating a new key.
44+
45+
Be sure to use SBT 0.13.7 or higher to avoid [#1430](https://github.com/sbt/sbt/issues/1430)!
46+
47+
### Testing
48+
49+
1. Follow the release process below to create a dummy release (e.g. 0.1.0-TEST1).
50+
Confirm that the release was staged to Sonatype but do not release it to Maven
51+
central. Instead, drop the staging repository.
52+
53+
### Performing a release
54+
55+
1. Create a GitHub "Release" (with a corresponding tag) via the GitHub
56+
web interface.
57+
2. Travis CI will schedule a build for this release. Review the build logs.
58+
3. Log into https://oss.sonatype.org/ and identify the staging repository.
59+
4. Sanity check its contents
60+
5. Release staging repository to Maven and send out release announcement.
61+

admin/build.sh

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
# prep environment for publish to sonatype staging if the HEAD commit is tagged
6+
7+
# git on travis does not fetch tags, but we have TRAVIS_TAG
8+
# headTag=$(git describe --exact-match ||:)
9+
10+
if [ "$TRAVIS_JDK_VERSION" == "$PUBLISH_JDK" ] && [[ "$TRAVIS_TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9-]+)? ]]; then
11+
echo "Going to release from tag $TRAVIS_TAG!"
12+
myVer=$(echo $TRAVIS_TAG | sed -e s/^v//)
13+
publishVersion='set every version := "'$myVer'"'
14+
extraTarget="+publish-signed"
15+
cat admin/gpg.sbt >> project/plugins.sbt
16+
cp admin/publish-settings.sbt .
17+
18+
# Copied from the output of genKeyPair.sh
19+
K=$encrypted_e923b9d88d53_key
20+
IV=$encrypted_e923b9d88d53_iv
21+
22+
openssl aes-256-cbc -K $K -iv $IV -in admin/secring.asc.enc -out admin/secring.asc -d
23+
fi
24+
25+
sbt "$publishVersion" clean update +test +publishLocal $extraTarget

admin/encryptEnvVars.sh

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/bin/bash
2+
#
3+
# Encrypt sonatype credentials so that they can be
4+
# decrypted in trusted builds on Travis CI.
5+
#
6+
set -e
7+
8+
read -s -p 'SONA_USER: ' SONA_USER
9+
travis encrypt SONA_USER="$SONA_USER"
10+
read -s -p 'SONA_PASS: ' SONA_PASS
11+
travis encrypt SONA_PASS="$SONA_PASS"

admin/genKeyPair.sh

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/bin/bash
2+
#
3+
# Generates a key pair for this repository to sign artifacts.
4+
# Encrypt the private key and its passphrase in trusted builds
5+
# on Travis CI.
6+
#
7+
set -e
8+
9+
# Based on https://gist.github.com/kzap/5819745:
10+
function promptDelete() {
11+
if [[ -f "$1" ]]; then
12+
echo About to delete $1, Enter for okay / CTRL-C to cancel
13+
read
14+
rm "$1"
15+
fi
16+
}
17+
for f in admin/secring.asc.enc admin/secring.asc admin/pubring.asc; do promptDelete "$f"; done
18+
19+
echo Generating key pair. Please enter 1. repo name 2. [email protected], 3. a new passphrase
20+
echo Be careful when using special characters in the passphrase, see http://docs.travis-ci.com/user/encryption-keys/#Note-on-escaping-certain-symbols
21+
cp admin/gpg.sbt project
22+
sbt 'set pgpReadOnly := false' \
23+
'set pgpPublicRing := file("admin/pubring.asc")' \
24+
'set pgpSecretRing := file("admin/secring.asc")' \
25+
'pgp-cmd gen-key'
26+
rm project/gpg.sbt
27+
28+
echo ============================================================================================
29+
echo Encrypting admin/secring.asc. Update K and IV variables in admin/build.sh accordingly.
30+
echo ============================================================================================
31+
travis encrypt-file admin/secring.asc
32+
rm admin/secring.asc
33+
mv secring.asc.enc admin
34+
35+
echo ============================================================================================
36+
echo Encrypting environment variables. Add each to a line in .travis.yml. Include a comment
37+
echo with the name of the corresponding variable
38+
echo ============================================================================================
39+
read -s -p 'PGP_PASSPHRASE: ' PGP_PASSPHRASE
40+
travis encrypt PGP_PASSPHRASE="$PGP_PASSPHRASE"
41+

admin/gpg.sbt

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
2+
addSbtPlugin("com.typesafe.sbt" % "sbt-pgp" % "0.8.3") // only added when publishing, see build.sh

admin/publish-settings.sbt

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
def env(key: String) = Option(System.getenv(key)).getOrElse("")
2+
3+
pgpPassphrase := Some(env("PGP_PASSPHRASE").toArray)
4+
5+
pgpPublicRing := file("admin/pubring.asc")
6+
7+
pgpSecretRing := file("admin/secring.asc")
8+
9+
credentials += Credentials("Sonatype Nexus Repository Manager", "oss.sonatype.org", env("SONA_USER"), env("SONA_PASS"))

build.sbt

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
import com.typesafe.tools.mima.plugin.{MimaPlugin, MimaKeys}
2+
3+
resolvers in ThisBuild += "scala-pr" at "https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/"
4+
5+
// this is the SHA of https://github.com/scala/scala/pull/5603 as of 2017-01-12
6+
// (that's the PR that removes the parallel collections)
7+
crossScalaVersions in ThisBuild := Seq("2.13.0-eb9fdcd-SNAPSHOT")
8+
9+
scalaVersion in ThisBuild := crossScalaVersions.value.head
10+
11+
version in ThisBuild := "1.0.0-SNAPSHOT"
12+
13+
scalacOptions in ThisBuild ++= Seq("-deprecation", "-feature")
14+
15+
cancelable in Global := true
16+
17+
lazy val core = project.in(file("core")).settings(scalaModuleSettings).settings(scalaModuleOsgiSettings).settings(
18+
name := "scala-parallel-collections",
19+
OsgiKeys.exportPackage := Seq(s"scala.collection.parallel.*;version=${version.value}"),
20+
mimaPreviousVersion := None
21+
)
22+
23+
lazy val junit = project.in(file("junit")).settings(
24+
libraryDependencies += "com.novocode" % "junit-interface" % "0.11" % "test",
25+
testOptions += Tests.Argument(TestFrameworks.JUnit, "-a", "-v"),
26+
fork in Test := true
27+
).dependsOn(testmacros, core)
28+
29+
lazy val scalacheck = project.in(file("scalacheck")).settings(
30+
libraryDependencies += "org.scalacheck" % "scalacheck_2.12" % "1.13.4",
31+
fork in Test := true,
32+
testOptions in Test += Tests.Argument(TestFrameworks.ScalaCheck, "-workers", "1", "-minSize", "0", "-maxSize", "4000", "-minSuccessfulTests", "5")
33+
).dependsOn(core)
34+
35+
lazy val testmacros = project.in(file("testmacros")).settings(
36+
libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
37+
)

src/library/scala/collection/Parallelizable.scala renamed to core/src/main/scala/scala/collection/Parallelizable.scala

+1-5
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,7 @@ trait Parallelizable[+A, +ParRepr <: Parallel] extends Any {
3737
*
3838
* @return a parallel implementation of this collection
3939
*/
40-
def par: ParRepr = {
41-
val cb = parCombiner
42-
for (x <- seq) cb += x
43-
cb.result()
44-
}
40+
def par: ParRepr = parCombiner.fromSequential(seq)
4541

4642
/** The default `par` implementation uses the combiner provided by this method
4743
* to create a new parallel collection.

0 commit comments

Comments
 (0)