Code 404 for sending requests with images (stored in Google Storage)

[IagoRabelo]

Description of the bug:

I'm having trouble sending requests to analyze images hosted on Google Storage. Today my application already works with sending images converted to base64, however, when switching to the hosted image link, I encounter a 404 error.

Actual vs expected behavior:

My application currently works using the @google/generative-ai lib and the following methods:
`
const chatSession = model.startChat({
generationConfig,
history: context,
})

const resp = await chatSession.sendMessage(["Perform the analysis and answer the questions in the json format provided"])
`

It is currently working with a context in this format:
[ { "role": "user", "parts": [ { "text": "OCR_NF: [\nOCR reading content\n]" }, { "inlineData": { "data": "iVBORw0K....kJggg==", "mimeType": "image/png" } } ] } ]
However, now, changing it to read the file from Google Storage, it looks like this:
[ { "role": "user", "parts": [ { "text": "OCR_DOC: [\nOCR reading content\n]" }, { "fileData": { "fileUri": "gs://bucket_name/file_name.png", "mimeType": "image/png" } } ] } ]
However, the response is always a 404 code:
{ "response": { "status": 400, "statusText": "Bad Request" }, "verify": false }
Observations:

1. The files are being saved correctly;
2. The link was checked and is correct;
3. It is within the same Gemini project;
4. To save the file I am using the @google-cloud/storage lib with the methods: storage.bucket, bucket.upload;

First time posting here, I hope I was clear enough.

Any other information you'd like to share?

No response

[hkt74]

Hi @IagoRabelo

Unfortunately, this SDK doesn't currently support inferencing directly with files from Google Cloud Storage (which requires GCP authenticate). However, you may find the Vertex AI Node.js SDK helpful, as it offers the functionality you're looking for, see example