Add PowerShell scripts #13592

philss · 2024-05-24T02:17:19Z

This is the first step to replace BAT scripts for Windows.

PowerShell works on Linux and MacOS too, so the scripts are supporting those systems as well.
~~The only requirement is that users have at least the version 7.2 LTS.~~

This feature was previously discussed with @josevalim :)

PS: I ran the test suite using "bin/elixir.ps1" in the lib/mix/test/test_helper.exs file (elixir_executable fun), and it's working good. I was not able to build and run on Windows yet, but the commands using the ELIXIR_CLI_DRY_RUN env var seems to be correct there :)

simonmcconnell · 2024-05-24T02:49:10Z

I really think we need to support Windows PoweShell 5.1. There are environments where you don't have access to install PoweShell, or you do but don't want to bundle PoweShell in your installer just to start Elixir.

What v7.2 specific functions are you using?

bin/mix.ps1

bin/elixirc.ps1

bin/elixir.ps1

simonmcconnell · 2024-05-24T03:47:11Z

bin/elixir.ps1

+  }
+}
+
+$ElixirParams = New-Object Collections.Generic.List[String]


Inconsistent array declaration. In elixirc.ps1 you're using @(), which I suggest you use everywhere.

The problem with @() in my case is that I want to add new elements and, sometimes, multiple elements. I wanted to add elements at the beginning of the array as well, and I couldn't find a way to do that with the @(). Do you know if it's possible?

$x = @() # append $x = $x + 1 # or $x = $x + @(1, 2, 3) # prepend $x = @(2) + $x

Either way is fine and most of the code uses the list style so maybe go with that. I thought it might be nice to be consistent across the scripts.

I changed to the += style. Thanks for the suggestion!

bin/elixir.ps1

philss

Thanks for the review! I could fix most of the suggestions in b06719e
I will continue to investigate some problems related to escaping.

I really think we need to support Windows PoweShell 5.1. There are environments where you don't have access to install PoweShell, or you do but don't want to bundle PoweShell in your installer just to start Elixir.

What v7.2 specific functions are you using?

@simonmcconnell I chose 7.2 just because it was the previous LTS version, and the version that I could install. I don't know yet what is not compatible with 5.1. I will investigate that as well.

bin/elixir.ps1

philss · 2024-05-24T17:46:14Z

bin/elixir.ps1

+  }
+}
+
+$ElixirParams = New-Object Collections.Generic.List[String]


The problem with @() in my case is that I want to add new elements and, sometimes, multiple elements. I wanted to add elements at the beginning of the array as well, and I couldn't find a way to do that with the @(). Do you know if it's possible?

bin/elixir.ps1

philss · 2024-05-24T17:59:06Z

bin/mix.ps1


-# Corrected arguments are ready to pass to batch file
-& $mixBatPath $newArgs
+Invoke-Expression "$Command $MixFile $($Args -join " ")"


I changed it to use Invoke-Expression, but it is not working well for when args have special chars. I'm investigating a better way. Perhaps the call (&) still the better option.

Hopefully we get to the point where we can use eval and rpc with quotes in a mix release.

As I said in a previous comment, I chose to use the "dot-sourcing" way of calling scripts, and define the args as variables.

This is much easier and, honestly, the only way I found to make this work without so much burden with escaping and quotations.

simonmcconnell · 2024-05-29T01:38:43Z

I chose 7.2 just because it was the previous LTS version, and the version that I could install. I don't know yet what is not compatible with 5.1. I will investigate that as well.

@philss I suggest you drop that check at the start and we ask CI to test on Windows PowerShell 5.1 and the LTS version. I'm happy to have a play with it on a Windows box too.

philss · 2024-05-29T04:37:50Z

I suggest you drop that check at the start and we ask CI to test on Windows PowerShell 5.1 and the LTS version. I'm happy to have a play with it on a Windows box too.

@simonmcconnell thanks! I removed the check.

philss · 2024-06-06T19:20:35Z

hey @josevalim and @simonmcconnell 👋 : I think this is ready for another pass.

bin/elixir.ps1

lib/elixir/test/elixir/kernel/cli_test.exs

josevalim

Beautiful. I have added some nits but other than that this is ready to go!

philss · 2024-06-10T17:27:49Z

@josevalim thank you! I have to check what is happening with the scripts on Windows. It looks like they are not being executed.

The idea is to replace the "bin/elixir.bat" script, and all other bat scripts in the future. The intention is to only support Windows for now.

PowerShell have some weird rules for parsing arguments. We need to take care of strings that have spaces, because the space char is considered a separator for arguments. We need to surround these strings with double-quotes, but we also need to escape the quotes inside first. When it's a normal execution, this escaping is just "doubling" the double-quotes. But when the execution is for the `--pipe-to` argument, this needs to be an Unix escaping done twice for double-quotes.

This is needed to preserve the contents of arguments that contain spaces when we call the main "bin/elixir.ps1" script using "Invoke-Expression".

Also test with ".ps1" on Unix-like SOs when "pwsh" binary is available.

We now define the prepended args in an array and we "source" the main script in order to process with those args. This way it's easier to just pass down the args and let the main script normalize them.

Since we are not planning to support releases for a while, we can simplify and remove these options.

It's not possible to execute them using "System.cmd/2" on Windows.

philss · 2024-06-10T22:41:30Z

@simonmcconnell we are probably close to a final version, so would be nice if you could give it a try on Windows and take another look. We had to disable the tests for the ".ps1" scripts on Windows because it won't execute due to that OS not recognizing the scripts as valid executables. I don't know if this can be fixed. So we are only running tests (of ".ps1" scripts) on Linux. Please let us know if you think this is possible to fix. Thanks!

simonmcconnell · 2024-06-11T00:47:45Z

@philss you might need to call System.cmd("powershell.exe" | "pwsh.exe", ["-ExecutionPolicy", "Bypass", "-NoProfile", "-File", "\"" <> path_to_ps1 <> "\""]). Everyone should have powershell but you can't guarantee they will have pwsh.

josevalim · 2024-06-11T06:36:11Z

@simonmcconnell we were hoping we could invoke the powershell script directly, as a .bat file, but we could not make that work, unfortunately. Using a separate executable would unfortunately complicate the suite. It makes me wonder if we should give up on the powershell scripts but, on the other hand, I don't think there are any alternatives besides pre-compiled executables. Setting assoc .ps1 unfortunately did not work either.

simonmcconnell · 2024-06-11T06:48:48Z

@josevalim AFAIK you can invoke it directly from a powershell shell, but not from a command prompt. You can set the shell you want to use in GitHub actions can't you?

simonmcconnell · 2024-06-11T06:51:43Z

scoop allows us to call scoop install .... It does this by having a .cmd file next to the .ps1 which checks which of pwsh and powershell to call the .ps1 with, which does all the work.

scoop.cmd

@rem C:\Users\simon\scoop\apps\scoop\current\bin\scoop.ps1
@echo off
where /q pwsh.exe
if %errorlevel% equ 0 (
    pwsh -noprofile -ex unrestricted -file "C:\Users\simon\scoop\apps\scoop\current\bin\scoop.ps1"  %*
) else (
    powershell -noprofile -ex unrestricted -file "C:\Users\simon\scoop\apps\scoop\current\bin\scoop.ps1"  %*
)

josevalim · 2024-06-11T06:54:38Z

If we start Elixir under Powershell, we still cannot call System.cmd directly. Which I guess makes sense, we wouldn't want the program to behave differently depending from where it started.

Your suggestion makes sense if we want to replace the existing .bat/.cmd files but there is a concern that always requiring powershell for starting Elixir will make things slower for those not running inside powershell.

simonmcconnell · 2024-06-11T07:08:41Z

If we start Elixir under Powershell, we still cannot call System.cmd directly

Sorry, I don't follow. From where can you not call System.cmd?

I think you're right on point 2 but I assume the majority of people are using powershell becuase it is the default these days. What are the benefits of migrating the batch scripts to powershell? Easier to write and escape things? I prefer a compiled executable.

josevalim · 2024-06-11T07:18:06Z

Sorry, I don't follow. From where can you not call System.cmd?

I cannot call System.cmd("some.ps1") even if Elixir is started under powershell. So the redirect is the only way to test it without changing existing code (or we change all tests to call powershell.exe).

Anyway, it is probably a bit too soon to move to .ps1, so I say we release this and plan to eventually force all .bat scripts to use powershell instead in one or two years from now. Btw, is the Windows Terminal powershell enable by default in latest releases? Or is it still running the command host?

simonmcconnell · 2024-06-11T08:13:05Z

PowerShell of some sort. I don't know if later versions use PowerShell 7+ or if it is always Windows Powershell.

josevalim · 2024-06-11T10:17:39Z

💚 💙 💜 💛 ❤️

simonmcconnell reviewed May 24, 2024

View reviewed changes

bin/mix.ps1 Outdated Show resolved Hide resolved

simonmcconnell reviewed May 24, 2024

View reviewed changes

bin/elixirc.ps1 Outdated Show resolved Hide resolved

simonmcconnell reviewed May 24, 2024

View reviewed changes