Skip to content

Minimal scripting support #11379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Feb 15, 2021
Merged

Minimal scripting support #11379

merged 8 commits into from
Feb 15, 2021

Conversation

philwalk
Copy link
Contributor

This supercedes #11180

Implements scala3 scripting support with the minimum functionality required for supporting real world scripting:

  • should be familiar to scala2 script developers, following the same basic design.
  • hash-bang section is ignored by the compiler
  • error message line numbers are correct even with hash-bang section
  • no constraint on script name, if hash-bang section is present (extension allowed by not required)
  • script runs in compiler process, avoiding multiple jvm startups
  • on successful compile, ScriptingDriver calls client callback prior to calling script main method.
  • The -save command line option writes a jar file to the script parent directory (same as scala2)
  • jar file manifest is written with Main-Class and Class-Path properties (jar is executable)

The client callback is needed to implement the -save option with a single jvm startup.

3rd party tools are able to implement custom implementations following the example in dotty.tools.scripting.Main.

  1. pass @<argsFile> and -color:* options to compiler
  2. add -save|-savecompiled option
  3. recognize scripts with #!.*scala regardless of extension
  4. if -save is specified and <scriptPath>.jar file is
      newer than <scriptPath> execute it (no compile)
  5. set -Dscript.name for script execution paths (script and .jar)

changes to dotty.tools.scripting package:
  1. additional compiler args splitting and filtering
  2. renamed detectMainMethod to detectMainClassAndMethod, returns
     both main class name and reflect.Method object
  3. on -save option:
     a. if compile is successful, create same-name jar file in
        <scriptPath> parent directory
     b. "java.class.path" appended to context classpath with deduplication
     c. write "Main-Class" and "Class-Path" to jar manifest

added new tests to verify the following:
  1. one line and multi-line hash bang sections are ignored by compiler
  2. main class name in stack dump is as expected when main class is declared in script
  3. main class name in stack dump is as expected when main class is not declared in script
  4. script.name property matches scriptFile.getName
  5. verify that with -save option jar file with expected name is generated
  6. verify that without -save option, no jar file is generated
  7. generated jar file is executable via "java -jar <scriptFile>.jar"
  1. pass @<argsFile> and -color:* options to compiler
  2. add -save|-savecompiled option
  3. recognize scripts with #!.*scala regardless of extension
  4. if -save is specified and <scriptPath>.jar file is
      newer than <scriptPath> execute it (no compile)
  5. set -Dscript.path for script execution paths (script and .jar)

changes to dotty.tools.scripting package:
  1. additional compiler args splitting and filtering
  2. renamed detectMainMethod to detectMainClassAndMethod, returns
     both main class name and reflect.Method object
  3. on -save option:
     a. if compile is successful, create same-name jar file in
        <scriptPath> parent directory
     b. "java.class.path" appended to context classpath with deduplication
     c. write "Main-Class" and "Class-Path" to jar manifest

added new tests to verify the following:
  1. one line and multi-line hash bang sections are ignored by compiler
  2. main class name in stack dump is as expected when main class is declared in script
  3. main class name in stack dump is as expected when main class is not declared in script
  4. script.path property matches scriptFile.absPath
  5. verify that with -save option jar file with expected name is generated
  6. verify that without -save option, no jar file is generated
  7. generated jar file is executable via "java -jar <scriptFile>.jar"
@SethTisue
Copy link
Member

👏 I am eager to test this once it lands in a nightly

@philwalk
Copy link
Contributor Author

philwalk commented Feb 11, 2021

this PR resolves these issues:
#4747
#10858

and is related to these:
#10491
#11180

@anatoliykmetyuk anatoliykmetyuk self-assigned this Feb 12, 2021
@anatoliykmetyuk anatoliykmetyuk self-requested a review February 12, 2021 10:57
Copy link
Contributor

@anatoliykmetyuk anatoliykmetyuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, nice work! After the minor suggestions are addressed, this can be merged.

@philwalk

This comment has been minimized.

@anatoliykmetyuk

This comment has been minimized.

@philwalk
Copy link
Contributor Author

philwalk commented Feb 13, 2021

@anatoliykmetyuk - I have some critical changes to push soon:

  1. relative paths must be absolute in the jar manifest, so they're interpreted relative cwd rather than jar file location.
  2. after lots of manual testing, I refined the detection of manifest class-path relative path entries
  3. 3rd party tools have a use case for compiling without execution, so I added a "-compile-only" option
  4. more tests added to verify the new "-compile-only" option, plus some other tests and updated comments

The "-compile-only" option is necessary, for example to compile a script to a graalvm native-image without executing the script as part of the compilation.

+ convert relative to absolute entries in manifest for equivalent classpath.
  manifest classpath entries would be relative to jar file location, not cwd.
+ added "-compile-only" option for compile without calling script main
+ added tests to verify "-compile-only" option
+ cleanup tests, update comments.
+ extensive manual experimentation on Windows and Ubuntu
+ jar file startup latency is 1/3 to 1/4 of compile and invoke latency.
+ manual tests of graalvm native-image compile of generated script jars
Copy link
Contributor

@anatoliykmetyuk anatoliykmetyuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @philwalk!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants