NativeScript CLI Use Modules from Node Dev Dependencies

[PanayotCankov]

NativeScript CLI Use Modules from Node Dev Dependencies

Why is this Needed

Currently using custom versions of the nativescript-cli modules is tedious. It requires, for example, referencing the "tns-ios" or "tns-android" packages through "--frameworkPath". The option requires different libraries based on the context it is used. Furthermore working on changes that impact multiple modules is tedious. And the following features will help keeping things up in source control.

Reference to Grunt

The following reorganization is inspired by Grunt. It is a build tool we are already using happily for most of the nativescript-cli modules.

The Grunt build system exposes a grunt command. Executing it will search up the directory tree for a:

gruntfile.js

When the file is located, it will be executed in node. Any grunt plugins required will be searched as standard node modules. Which means the gruntfile.js goes with a package.json. The modules are listed as dev dependencies:

{
  "name": "my-project-name",
  "version": "0.1.0",
  "devDependencies": {
    "grunt": "~0.4.5",
    "grunt-contrib-jshint": "~0.10.0",
    "grunt-contrib-nodeunit": "~0.4.1",
    "grunt-contrib-uglify": "~0.5.0"
  }
}

And the modules are usually installed locally using npm install. This leads to similar structure:

.
├── gruntfile.js
├── package.json
├── node_modules
│   └── grunt-exec
│       └── index.js
├── dist
│   └── compiler.js
└── src
    ├── mycode1.js
    └── mycode2.js

Now then, a subset of the nativescript-cli is pretty much a build system so it can benefit from similar approach. Currently, on build, we do transform the source files by eliminating platform postfixes, moving them from the app folder to the respective platform folder. In future we will have to provide hooks for different compilers (TypeScript, CoffeeScript etc.)

NativeScript-CLI

So following the grunt approach we can make the tns cli search up for a well known file, .tnsproject, tnsfile.js etc. and execute there. However the tns in that case should be a simple bootstrapper that would require its implementation using the node require logic. If the required libraries are not found locally, then tns should fallback to a globally installed one.

NOTE: If some nativescript-cli modules are not installed locally we should fallback to a globally installed one to preserve backward compatability.

This way the current usage should be preserved. However custom builds of the nativescript modules can be installed locally in node_modules.

It Will Help CLI Development

For example if you are working on the tns-ios package that also requires changes in the nativescript-cli module you should provide the following package.json.

{
  "name": "TNSApp",
  "version": "0.1.0",
  "devDependencies": {
    "nativescript": "~0.10.0",
    "tns-ios": "~0.10.0",
  }
}

And get the output of the nativescript and tns-ios builds onto the note_modules of the nativescript app and then the tns command should use the artifacts from the so locally installed custom builds.

.
├── nativescript.js
├── package.json
├── app
│   └── app.js
│       └── tns_modules
├── platforms
└── node_modules
    ├── tns_ios
    └── nativescript

Better Versioning and Better Source Control for the Users

The resulting project will be source control friendly. Currently we have binaries with the nativescript runtimes for Android and iOS that either get extracted within the project or symlinks are created.

• The first puts large files in source control, while the second may fail to round-trip through some systems.
• Tooling gets out of sync. For example code in the cli depends on certain versions of the runtimes.

With the suggested implementation the projects generated in platforms should query their binaries with some help from npm but the binaries themselves won't stay in source control.

When a repo containing a nativescript app is cloned, the developers can npm install the exact runtimes the app was written with. There would be nice for versioning and the apps will have clear indication what versions they were built with.

Furthermore it will ease us separate platform specific code. For example currently the code for tns debug ios, tns library add ios etc serve only for the NativeScript for iOS. With the offered approach they will easily move into the tns-ios package.

Downsides

The project scaffolding will be somewhat trickier. The tns create will have to install some additional files and further npm install may be necessary within the newly created app.

[rosen-vladimirov]

Great idea, @PanayotCankov . However I see one problem - in case I'm working on MacOS and my app's package.json is something like:

{
  "name": "TNSApp",
  "version": "0.1.0",
  "devDependencies": {
    "nativescript": "~0.10.0",
    "tns-ios": "~0.10.0",
  }
}

It will be useless on Windows as tns-ios package cannot be npm installed on Windows. So if I try sending it to somebody else, who is using Windows, he will not be be able to even try the app.

[teobugslayer]

@rosen-vladimirov - perhaps using optionalDependencies?

@PanayotCankov - we discussed removing the .tnsproject file and subsuming it into package.json - see #362

[PanayotCankov]

By the look of it we also have the versions in the .tnsproject file.

{
    "id": "org.nativescript.TNSApp7",
    "tns-ios": {
        "version": "0.10.0"
    }
}

And there allready plans to remove it, moving the settings in the package.json.

So after some iterations on the problem we narrowed it down to two things.

Do not Copy Binaries From The .NPM Cache

We do not want to copy binaries from the tns-ios locally in the platforms/ios folder. Instead we want the Xcode to use the binaries installed in the .npm cache. Which comes to a tiny integration between the search logic in the CLI and some Xcode params. We also have to make sure that iOS specific code that uses the tns-ios package should catch up the right version:

var tnsIosPackage = this.$npm.install(platformData.frameworkPackageName).wait();

Set up Development Environment

Where we can rapidly make changes to the nativescript-cli and tns-ios repo clones and have the changes propagated to our example projects.

What I had in mind was using link. With it you can npm link in the nativescript-cli and tns-ios repos, and npm link nativescript + npm link tns-ios in the example app. There may be other approaches easier to implement.

[atanasovg]

This seems a great idea to me and may not currently think of some downsides of this approach.

[RangerMauve]

I would argue against using the npm cache rather than a local copy of the binaries. The cache might change over time and is more or less global as I understand it, this might lead to issues if you're working on multiple projects, some of which are outdated.

[PanayotCankov]

During the development of the perf. profiler for iOS we were not able, using the real life scenario for the CLI, to:

1. Develop
2. Test

This is slowing us down. As an argument we now have the reason for the v1.2.1 hotfix:

https://github.com/NativeScript/nativescript-cli/pull/689/files

[hshristov]

💯
This is becoming an issue for us. Clients are working on different versions of NativeScript and we need to constantly switch between different version. Please raise its priority.

[rosen-vladimirov]

Currently you can add nativescript CLI as dev-dependency in your project.

npm i --save-dev nativescript

After that you can use ./node_modules/bin/.tns <command>. However this will fail in case you install nativescript-cli 1.3.0 and you try building for android. In order to fix this, just navigate to ./node_modules/nativescript/resources/tools/ and delete all .jars from there. This way you'll be able to build your project with

./node_modules/.bin/tns build android

These files are already removed from CLI's master branch, so with later versions, there will be no need to delete them manually.
As per the other suggestions - we are already using package.json and everything declared as dev-dependency will not be copied to tns_modules. The required runtime versions are saved inside special "nativescript" key of package.json, so I believe we have fulfilled the requirement.
Also in case you already have package.json, you can use tns install command that will get all of its dependencies and runtimes and add them to your project.

Should we close this issue?

[NelsonRock]

I have a issue:

When I do tns platform add android -> reponse java is not set to your path

All android support and library installed, 22, 23, android sdk builds tools 22, 23 as well

I'have installed node by nvm package manager , i have used node version 10.30, 10.35, 12.07 and reinstalled nativescript client in every version of node

Following path config in .bashrc

gradle path

GRADLE_HOME=/usr/share/gradle-2.3
PATH=$PATH:$GRADLE_HOME/bin
export GRADLE_HOME
export PATH

android path

ANDROID_HOME=/usr/share/android-sdk-linux
PATH=$PATH:$ANDROID_HOME/tools:ANDROID_HOME/platform-tools
export ANDROID_HOME
export PATH

java path

JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
JRE_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export JAVA_HOME
export JRE_HOME
export PATH

I paste the resultat from tns platform add android --log trace

nelson@developpeur:~/github/node/tes1$ tns platform add android --log trace
Exec uname -a
stdout: Linux developpeur 3.19.0-30-generic #34-Ubuntu SMP Fri Oct 2 22:08:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

stderr:
Exec npm -v
stdout: 1.4.28

stderr:
spawn: java -version
Exec ant -version
stdout:
stderr: /bin/sh: 1: ant: Permission denied

Exec node-gyp -v
stdout: v0.12.2

stderr:
Exec adb version
stdout:
stderr: /bin/sh: 1: adb: not found

Exec android -h
stdout:
Usage:
android [global options] action [action options]
Global options:
-s --silent : Silent mode, shows errors only.
-v --verbose : Verbose mode, shows errors, warnings and all messages.
--clear-cache: Clear the SDK Manager repository manifest cache.
-h --help : Help on a specific command.

                                                                Valid
                                                                actions
                                                                are
                                                                composed
                                                                of a verb
                                                                and an
                                                                optional
                                                                direct
                                                                object:

• sdk : Displays the SDK Manager window.

• avd : Displays the AVD Manager window.

• list : Lists existing targets or virtual devices.

• list avd : Lists existing Android Virtual Devices.

• list target : Lists existing targets.

• list device : Lists existing devices.

• list sdk : Lists remote SDK repository.

• create avd : Creates a new Android Virtual Device.

• move avd : Moves or renames an Android Virtual Device.

• delete avd : Deletes an Android Virtual Device.

• update avd : Updates an Android Virtual Device to match the folders
  of a new SDK.

• create project : Creates a new Android project.

• update project : Updates an Android project (must already have an
  AndroidManifest.xml).

• create test-project : Creates a new Android project for a test package.

• update test-project : Updates the Android project for a test package (must
  already have an AndroidManifest.xml).

• create lib-project : Creates a new Android library project.

• update lib-project : Updates an Android library project (must already have
  an AndroidManifest.xml).

• create uitest-project: Creates a new UI test project.

• update adb : Updates adb to support the USB devices declared in the
  SDK add-ons.

• update sdk : Updates the SDK by suggesting new platforms to install
  if available.

  stderr: Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
  Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
  Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar

Exec mono --version
stdout:
stderr: /bin/sh: 1: mono: not found

Exec git --version
stdout: git version 2.1.4

stderr:
Exec gradle -v

stdout:

Gradle 2.3

Build time: 2015-02-16 05:09:33 UTC
Build number: none
Revision: 586be72bf6e3df1ee7676d1f2a3afd9157341274

Groovy: 2.3.9
Ant: Apache Ant(TM) version 1.9.3 compiled on December 23 2013
JVM: 1.8.0_45-internal (Oracle Corporation 25.45-b02)
OS: Linux 3.19.0-30-generic amd64

stderr: Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar

Exec javac -version
stdout:
stderr: Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
javac 1.8.0_60

TypeError: Cannot read property '1' of null
at FiberFuture.Future.wait (/home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/node_modules/fibers/future.js:535:15)
at /home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/lib/nativescript-cli.js:17:45
- - - - -
at SysInfo.getSysInfo (/home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/lib/common/sysinfo.js:63:75)
at /home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/lib/common/dispatchers.js:21:46
at Function.settle (/home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/node_modules/fibers/future.js:249:26)
at FiberFuture.Future.wait (/home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/node_modules/fibers/future.js:534:10)
at /home/nelson/.nvm/v0.10.35/lib/node_modules/nativescript/lib/nativescript-cli.js:17:45

[ligaz]

Closing this as everything is addressed as outline by @rosen-vladimirov #415 (comment)