Best way to deploy TypeScript function to Cloud Function?

[dmiranda2791]

I am using the TypeScript template to write the function and I faced some issues trying to deploy it using the gcloud functions deploy command.

I tried:

• The intuitive commando:

gcloud functions deploy pocReadFromBigQueryWriteToFirestore \
--runtime nodejs10 --trigger-http --allow-unauthenticated

Got the error

Deploying function (may take a while - up to 2 minutes)...failed.                                        
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: index.js does not exist; Error ID: e1b2ac36

• Then, passing a source argument pointing to the build/src directory.

gcloud functions deploy pocReadFromBigQueryWriteToFirestore \
--source=build/src \
--runtime nodejs12 --trigger-http --allow-unauthenticated

Got this error in the cli

Deploying function (may take a while - up to 2 minutes)...failed.                                        
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function failed on loading user code. This is likely due to a bug in the user code. Error message: Error: please examine your function logs to see the error cause: https://cloud.google.com/functions/docs/monitoring/logging#viewing_logs. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.

And this one on the functions logs:

2021-02-04T17:47:55.029ZpocReadFromBigQueryWriteToFirestore Provided module can't be loaded.
Default
2021-02-04T17:47:55.029ZpocReadFromBigQueryWriteToFirestore Did you list all required modules in the package.json dependencies?

Next, I tried copying the package.json into the build/src folder and rerun the same command as above.

Deploying function (may take a while - up to 2 minutes)...failed.                                        
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: > [email protected] postinstall /workspace/node_modules/protobufjs
> node scripts/postinstall


> [email protected] prepare /workspace
> yarn run compile

yarn run v1.22.4
$ tsc
/bin/sh: 1: tsc: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
npm ERR! code ELIFECYCLE
npm ERR! syscall spawn
npm ERR! file sh
npm ERR! errno ENOENT
npm ERR! [email protected] prepare: `yarn run compile`
npm ERR! spawn ENOENT
npm ERR! 
npm ERR! Failed at the [email protected] prepare script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /builder/home/.npm/_logs/2021-02-04T20_07_18_893Z-debug.log; Error ID: beaf8772

Finally, I got it working by copying package.json to build/src/package.json and removing the scripts prepare, pretest and posttest. So, I got to think and I wonder:

• Why and how this scripts affect the deployment?
• What is the best strategy to deployment?

Thanks!

[grant]

Hi, have you looked at other related issues?

• Cloud Function in Typescript #13
• Watching for changes & Typescript #52

There are many different configurations depending on how you like to write TS. In general, first build your TS into JS and then you can test the JS with the Functions Framework locally. That will ensure you don't get errors before you deploy. Make sure your build directory isn't ignored and that things work locally.

[dmiranda2791]

@grant thanks for pointing those out. I don't have issues with the development workflow in my local but do you know why the deployment fails when the function is deployed with this package.json? The error shows an execution of the prepare script but I don't know why is executed at deployment.

{
  "name": "cloud-function-poc",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "lint": "gts lint",
    "clean": "gts clean",
    "compile": "tsc",
    "fix": "gts fix",
    "prepare": "yarn run compile",
    "pretest": "yarn run compile",
    "posttest": "yarn run lint",
    "start": "functions-framework --source=build/src/ --target=pocReadFromBigQueryWriteToFirestore",
    "watch": "concurrently \"tsc -w\" \"nodemon --watch ./build/ --exec npm run start\""
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@google-cloud/bigquery": "^5.5.0",
    "@google-cloud/firestore": "^4.9.1",
    "@google-cloud/functions-framework": "^1.7.1",
    "eslint": "^7.2.0",
    "eslint-config-prettier": "^7.2.0"
  },
  "devDependencies": {
    "@types/express": "^4.17.11",
    "@types/node": "^14.11.2",
    "concurrently": "^5.3.0",
    "eslint-plugin-import": "^2.22.1",
    "gts": "^3.1.0",
    "nodemon": "^2.0.7",
    "typescript": "^4.0.3"
  }
}

[grant]

Hi,
The issue doesn't look like it's with the Functions Framework.

The main error posted above is:
/bin/sh: 1: tsc: not found

It looks like your function is trying to build the TS program before deploying, so you need typescript to be available. Right now it isn't installed because it's a dev dep. Maybe move it to a dep or be sure to run tsc before deploying to Cloud Functions.

[dmiranda2791]

Hey @grant, the thing is that I want to deploy the files that have already been built by TypeScript, which are in build/src, but when the function is being deployed it runs the prepare scripts by its self.

[grant]

@dmiranda2791 OK. Prebuilt JS files are great.

This seems like an issue with your function code.

In order for me to triage this issue, I need a minimal sample, full steps from start to finish of how you're deploying to GCF.

I don't know why index.js does not exist; Error ID: e1b2ac36. Maybe you're ignoring the build directory on accident when uploading to GCF. Check your gcloudignore and gitignore files. If you're ignoring JS files, you'll get this error. Create a gcloudignore file if you haven't done so:

https://cloud.google.com/functions/docs/deploying/filesystem

Either way, this isn't a Functions Framework issue, unless you see something I'm missing.

[grant]

I'm going to close this as the identified error does not pertain to the Functions Framework, rather it is user code. But feel free to keep commenting. As mentioned, clear reproduction steps would help.

I've provided a bunch of tips and possible errors in the code in comments above.

[ash67]

@grant I am having exactly the same issue as @dmiranda2791.

so when GTS complies typescript code into js and places the compiled JS code into the build/src folder. However, it does not copy package.json over to the build/src folder. So the possible solution is to manually copy the package.json file and remove prepare, pretest and posttest scripts.

so when deploying a function from the build/src folder, it works just fine.

I would like to ask if there was a better approach than this? Ideally the build/src should have compiled JS code with package.json file so the whole folder can be deployed.

[grant]

@grant I am having exactly the same issue as @dmiranda2791.

so when GTS complies typescript code into js and places the compiled JS code into the build/src folder. However, it does not copy package.json over to the build/src folder. So the possible solution is to manually copy the package.json file and remove prepare, pretest and posttest scripts.

so when deploying a function from the build/src folder, it works just fine.

I would like to ask if there was a better approach than this? Ideally the build/src should have compiled JS code with package.json file so the whole folder can be deployed.

Can you provide the list of steps to repro?

Would something like this in the package.json help?:

  "scripts": {
    "start": "functions-framework --source=build/src"
  }

[ash67]

@grant thanks for the prompt response!

Here are the steps:

1. I followed the steps mentioned here to create a project, write code in TS, run & test code locally.
2. My project had dependencies of other NPM modules so I added them in the package.json file
3. GTS created the build/src folder which contained the complied JS code without package.json file.
4. I deployed my function code from the build/src folder

Of course, it did not work because the build/src folder didn't contain the package.json which listed all my project dependencies.

The start script you suggested is already in my project and it works fine locally. It serves my function code at localhost:8080.

So should I deploy the pre-compiled code that is inside the build/src directory or deploy the TS code that would be compiled into JS when the function is being deployed?

[grant]

Thanks!

Yes, I see how something is broken in this flow. So I followed those steps, skipping the nodemon part. I then ran the FF locally - that worked.

Then I deployed:

gcloud functions deploy ash67 --runtime nodejs12 --trigger-http --allow-unauthenticated

And got the error:

Deploying function (may take a while - up to 2 minutes)...failed.                        
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: > @0.0.0 prepare /workspace
> npm run compile


> @0.0.0 compile /workspace
> tsc

sh: 1: tsc: not found

It looks like the gts init script compiles in prod, we probably don't want that because we don't want to install TS when building, let's build locally.

Remove line:

"prepare": "npm run compile",

Deploy:

gcloud functions deploy ash67 --runtime nodejs12 --trigger-http --allow-unauthenticated --entry-point helloWorld

There's a different error for not finding the function in the root dir. For a workaround, I created a index.js file in the root directory with this:

exports.helloWorld = require('./build/src/index').helloWorld;

Seems to work. Just remember to build/watch before deploying.

https://us-central1-test-grant.cloudfunctions.net/ash67

I'm not sure why configuring a source in the npm start FF script isn't working in prod, only locally. But this workaround seems pretty lightweight.

Hope that helps.

[ash67]

@grant

Thanks so much for providing this workaround. It works like a charm.

[marjan2k]

I was starting to test Functions Framework in Typescript and came across the same issue.
Removing the line "prepare": "npm run compile" fixed it.
Any reason why the prepare is kept there by default if it prevents deploy?

[johnnyoshika]

This is very interesting. I had the exact same problem as the OP, but as soon as I removed "prepare": "npm run compile" from package.json, deployment worked.

[mcorcuera]

I was having the same issue where my TS compiled field where under build/, but I was getting the error:

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: index.js does not exist; Error ID: e1b2ac36

I found out that the gcloud functions deploy command will take into account the file configured as main in the package.json. So having a configuration like this will make it work:

{
  "name": "cloud_functions_typescript",
  "version": "1.0.0",
  "description": "Cloud functions typescript",
  "main": "build/index.js",
  "scripts": { ...}
}

[hammadzz]

I hope I am wrong, but there is no way to stop Google Cloud Functions from installing your npm packages for you. I don't want GCF doing CI/CD work. It shouldn't be concerned about my dependencies, they should come packaged in the zip upload to GCF.

I understand that GCF doing the install of dependencies fixes the issue with native npm modules that work on your local machine but won't in GCF. Although that is an easy fix in CI/CD by using the right docker image / container to install npm modules inside.

For this reason I opted for Cloud Run instead to overcome this issue. If anyone knows a work around do let us know.

[johannesgiani]

@grant maybe that workaround could be mentioned in the typescript docs that @ash67 referred to.

Proposal:

• remove prepare script in package.json
• adjust main in package.json

"main": "build/src/index.js",

[grant]

@johannesgiani PRs are welcome. The TypeScript doc is a community example doc that I'm sure could be improved / updated. :)

[toddbaert]

Unless I'm very confused, the configuration of the scripts in the framework is a bit problematic combined with what runs automatically in Cloud Build (npm ci --quiet (NODE_ENV=production)).

npm ci --quiet (NODE_ENV=production): does NOT install dev dependencies
prepare: runs after install, including npm ci as above, and in this case runs npm run compile
compile: runs tsc, which often requries dev dependencies, such as typescript itself, and are not present because of the aforementioned production ci run.

This is easily observed locally by deleting your node modules and running npm ci --prodction. How is this supposed to work? What am I missing?

[johannesgiani]

@grant sure let me try that, but for some reason I get remote: Permission to GoogleCloudPlatform/functions-framework-nodejs.git denied to johannesgiani when pushing my branch (using username and access token for public repo). I setup the open source agreement following the contributing docs. Anything else to setup?

@toddbaert did you try just removing the prepare script that was generated by gts?

[grant]

@grant sure let me try that, but for some reason I get remote: Permission to GoogleCloudPlatform/functions-framework-nodejs.git denied to johannesgiani when pushing my branch (using username and access token for public repo). I setup the open source agreement following the contributing docs. Anything else to setup?

Please use a fork and create a PR. I think that error states you've tried to push a branch from a clone, not fork.

[toddbaert]

@johannesgiani I can certainly remove the script, but I'm trying to make a larger point that the way the scrips are written is fundamentally flawed. A prepare step with a tsc in it is a bad idea, because it will always cause errors after a production NPM install assuming people are using devDependencies idiomatically.

[xav-ie]

A prepare step with a tsc in it is a bad idea, because it will always cause errors after a production NPM install assuming people are using devDependencies idiomatically.

Were you ever able to figure this out? @hammadzz