Upgrade npm and node version

[humitos]

There are some Python/JS libraries that requires a more up to date version of npm.

I've been using this workaround for now, but it would be good to have this implemented at the Docker image level.

# Workaround to upgrade ``npm`` since we have ``3.5.2`` in our
# Docker image and we need at least ``npm>=6.0``
import os
os.system('cd ~ ; mkdir bin ; npm install npm')

[tavianator]

I'm running into google/docsy#265 building the docs for my project, which means I need Node >= 10 installed.

[raucao]

I've been using this workaround for now, but it would be good to have this implemented at the Docker image level.

Also, just as a note, it doesn't help with upgrading node itself.

[raucao]

We're kind of stuck, unable to release a new major version of our software, until we can build the docs again. Would be fantastic if someone with experience in this Docker repo/system could update node.js to version 10 or newer sometime in the not-too-distant future.

(The current node.js LTS version is 14.15, which is 3 LTS major versions ahead of the one available on RTD.)

[stsewd]

I think we should just use nvm and install the latest LTS version of node instead of relying on the old one shipped with ubuntu

[raucao]

There's also a PPA: https://github.com/nodesource/distributions/blob/master/README.md#manual-installation

And for recent Ubuntu versions, snaps would be another option: https://github.com/nodejs/snap

[Nic30]

We're kind of stuck, unable to release a new major version of our software, until we can build the docs again. Would be fantastic if someone with experience in this Docker repo/system could update node.js to version 10 or newer sometime in the not-too-distant future.

Exactly, I can not use readthedocs.org because of this reason.

Basically because it is not possible to upgrade nodejs on readthedocs.org I can not release any new package as I need a link on the doc in advance.

Image based on Ubuntu 20.xx would be great is it know when it will be completed?
(readthedocs/build:8.0)

Also, I can not use conda:
https://readthedocs.org/projects/hwtlib/builds/13076605/

Collecting package metadata: ...working... done
Solving environment: ...working... Killed


Command killed due to excessive memory consumption

Edit:

I finally made it work.
I used anaconda and older version of npmjs (14.15.1) from official list https://conda.anaconda.org/conda-forge/linux-64/
My config: https://github.com/Nic30/hwtLib/blob/master/doc/environment.yml
Works reliably but takes much longer than native RTD image.

[pradyunsg]

This is a part of the things blocking me from moving https://pradyunsg.me/furo/ to ReadTheDocs.

It needs a reasonably new version of npm to generate the final assets (since those aren't checked in) and there's no good way to have that work on RTD. :(

I'm all ears for what's a good way to make it work -- I basically need to do npm install && gulp build && pip install . before building the docs.

[Nic30]

@pradyunsg see
https://github.com/Nic30/hwtLib/blob/master/doc/environment.yml
https://github.com/Nic30/hwtLib/blob/master/.readthedocs.yml
I somehow managed to make it work using anaconda. (I am calling npm install from python during the installation of other package https://github.com/Nic30/sphinx-hwt/blob/master/setup.py#L42 but it is probably possible to do it with anaconda env as well.)

[humitos]

Hi @pradyunsg!

This is a part of the things blocking me from moving pradyunsg.me/furo to ReadTheDocs.

I'd love to have furo theme on Read the Docs! 🤩

It needs a reasonably new version of npm to generate the final assets (since those aren't checked in) and there's no good way to have that work on RTD. :(
I'm all ears for what's a good way to make it work -- I basically need to do npm install && gulp build && pip install . before building the docs.

Unfortunately, currently there is no good way to do this --only the workaround that it's on the description of this issue 😞. A "nicer" workaround, is to use conda and install the version of npm and nodejs required; as other people mentioned.

Currently, we are using npm and nodejs packages that comes with Ubuntu 18.04 LTS (our current Docker images). The main problem here is that it's hard to upgrade a package to bring new features without breaking someone's else builds. Because of this reason, I tried to write down an initial proposal for the "new docker image structure" to allow us to upgrade packages on our image without affecting all the users --or reducing the amount of users affected as much as possible: readthedocs/readthedocs.org#7566

I'd say that most of the work from this repository is blocked on that discussion and it seems we don't have a clear path to move forward. I invite you all to read that PR and bring ideas if you know a better and cleaner way to solve this problem.

[raucao]

A "nicer" workaround, is to use conda and install the version of npm and nodejs required; as other people mentioned.

I still haven't seen a way to upgrade node.js itself (not npm), but maybe I missed it. As someone not familiar with Python and its build environments, can someone explain to the rest of us the specific solution for upgrading node.js in a current build perhaps?

[Nic30]

@raucao see my previous comment, I am using nodejs=14.15.1

[humitos]

We are working on new Docker images in #166 where node and npm versions won't come installed by default on those images. People wanting them will be able to install the version they need via the config file with something similar to build.os: ubuntu20 and build.languages.node: 14.

[choldgraf]

Just a note that we are running into this as well in jupyter-book/thebe#469 - that's a JS build that depends on a newer version of NPM. Just FYI!

[humitos]

@choldgraf I have a good part of the new implementation working locally. I'd like to use thebe project as a test for a documentation's project that depends on a newer nodejs version and do some QA with it. Is there a PR or branch that I can do from that project as a test that should be working once Read the Docs supports newer npm/nodejs versions?

I found that you made some changes in that PR to use GitHub Actions because of these limitations, but it seems that only this set of commits may be what I'm looking for https://github.com/executablebooks/thebe/pull/469/files/9149864f40168400e100e4e3be823785335ea8b9 --can you confirm this?

[choldgraf]

That sounds right to me! Perhaps @stevejpurves would be willing to coordinate with you to help test something out?

[stevejpurves]

@humitos that does look right, and I'd much rather go in that direction with a build running on RTD! I've created a clean branch/PR here with just the changes so far that we need.

jupyter-book/thebe#472

which is better to build on than the other, which went a bit exploratory.

btw... in those changes, I am using a local yarn install, as I was unable to install globally npm install -g yarn from the conf.py script. I'd much rather have yarn available globally. I'm not sure if you've considered letting folks put yarn on as part of this work? (FYI doing this on ubuntu20 via apt still requires the yarn apt repository to be added :/ )

[humitos]

Thanks @stevejpurves for your reply.

I've created a clean branch/PR here with just the changes so far that we need.

I'll do some tests with that PR and my local Read the Docs development instance and I will comment in that PR if I find any issue. Thanks!

I'd much rather have yarn available globally. I'm not sure if you've considered letting folks put yarn on as part of this work?

We are not planning to install yarn globally as part of this work. Besides, I'm not sure that this will be possible in the future because we are not planning to expose root access and I think that -g requires it.

I am using a local yarn install, as I was unable to install globally npm install -g yarn from the conf.py script.

In the future, running commands outside the conf.py may be possible if we implement readthedocs/readthedocs.org#8190 that will allow custom commands (via build.jobs.pre_build or similar)

[stevejpurves]

@humitos ok, thanks! I think the local yarn might still work for us in the interim, it's really as we need it to ensure deps are installed in line with the yarn.lock.

Is there anything I can do to help test on this one @humitos?

[humitos]

Is there anything I can do to help test on this one @humitos?

Not for now. I've commented in jupyter-book/thebe#472, so we can continue this talk there. Thanks!

[humitos]

Hi all! In #166 we implemented the ability to install node 14.x and 16.x --which is still in beta. Would you like to give it a try and let us know if it works? You need to add something like this to your .readthedocs.yaml file:

build:
  os: "ubuntu-20.04"
  tools:
    python: "3.9"
    nodejs: "16"

Note that if you are already defining the Python version via python.version you have to remove it from there.

Thanks!