Cannot read property 'pipeThrough' of undefined error on ChatSession.sendMessageStream but not ChatSession.sendMessage

[svdgoor]

Description of the bug:

I followed the stack in the code:
Error originates from processStream
This is exclusively called from generateContentStream
Which is called from sendMessageStream (The function I can see outside of the library)

Lets look at why exactly response.body is undefined.
response is generated by a call to makeModelRequest
Now this function, and a helper function it calls are defined here].
I can't figure out how it is possible that body is undefined, while response.ok is true.

Please help :)
Streaming is so cool!

Actual vs expected behavior:

const apiKey = get your own key;
const genAI = new GoogleGenerativeAI(apiKey);
const model = genAI.getGenerativeModel({model: "gemini-1.5-flash"});
const chat = model.startChat({});
try {
  const request = await chat.sendMessageStream("Hello");
  for await (const chunk of request.stream) {
    if (chunk.promptFeedback) {
      error(`${chunk.promptFeedback.blockReason} ${chunk.promptFeedback.safetyRatings} ${chunk.promptFeedback.blockReasonMessage}`);
    }
  }
  const response = await request.response;
  debug(`Test Stream Response: ${JSON.stringify(response)}`);
} catch (e) {
  error(e);
}
try {
  const request = await chat.sendMessage("Hello");
  const response = request.response;
  debug(`Test Non-Stream Response: ${JSON.stringify(response)}`);
} catch (e) {
  error(e);
}

Expected result:

Test Stream Response: {"candidates":[{"content":{"parts":[{"text":"Hello there! How can I help you today?\n"}],"role":"model"},"finishReason":"STOP","avgLogprobs":-0.0011556809768080711}],"usageMetadata":{"promptTokenCount":2,"candidatesTokenCount":11,"totalTokenCount":13},"modelVersion":"gemini-1.5-flash"}
Test Non-Stream Response: {"candidates":[{"content":{"parts":[{"text":"Hello there! How can I help you today?\n"}],"role":"model"},"finishReason":"STOP","avgLogprobs":-0.0011556809768080711}],"usageMetadata":{"promptTokenCount":2,"candidatesTokenCount":11,"totalTokenCount":13},"modelVersion":"gemini-1.5-flash"}

Actual result:

Cannot read property 'pipeThrough' of undefined
Test Non-Stream Response: {"candidates":[{"content":{"parts":[{"text":"Hello there! How can I help you today?\n"}],"role":"model"},"finishReason":"STOP","avgLogprobs":-0.0011556809768080711}],"usageMetadata":{"promptTokenCount":2,"candidatesTokenCount":11,"totalTokenCount":13},"modelVersion":"gemini-1.5-flash"}

Any other information you'd like to share?

"@google/generative-ai": "^0.21.0",

[hkt74]

the streaming method returns an iterable stream of response, could you try print the response from each chunk? see the sample code for chat streaming

[svdgoor]

This should still work though, regardless of whether the content is also available in the chunks.

const genAI = new GoogleGenerativeAI(apiKey);
const model = genAI.getGenerativeModel({model: "gemini-1.5-flash"});
const chat = model.startChat({});
try {
  debug("Test Stream Start");
  const request = await chat.sendMessageStream("Hello");
  try {
    for await (const chunk of request.stream) {
      if (chunk.promptFeedback) {
        error(`${chunk.promptFeedback.blockReason} ${chunk.promptFeedback.safetyRatings} ${chunk.promptFeedback.blockReasonMessage}`);
      }
      debug(`Test Stream Chunk: ${chunk.text()}`);
    }
  } catch (e) {
    error("Error with retrieving stream from request", e);
    if (e instanceof Error) {
      error(e.stack);
    }
  }
  const response = await request.response;
  debug(`Test Stream Response: ${JSON.stringify(response)}`);
} catch (e) {
  error("Error with retrieving response from request", e);
  if (e instanceof Error) {
    error(e.stack);
  }
}
try {
  const request = await chat.sendMessage("Hello");
  const response = request.response;
  debug(`Test Non-Stream Response: ${JSON.stringify(response)}`);
} catch (e) {
  error(e);
}

Log

 LOG  11:28:45 AM | DEBUG : [app/dev/state.tsx] Test Stream Start
 LOG  11:28:45 AM | ERROR : [app/dev/state.tsx] Error with retrieving response from request Cannot read property 'pipeThrough' of undefined
 LOG  11:28:45 AM | ERROR : [app/dev/state.tsx] TypeError: Cannot read property 'pipeThrough' of undefined
    at processStream (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:150491:48)
    at ?anon_0_ (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:150670:27)
    at next (native)
    at asyncGeneratorStep (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:5859:19)
    at _next (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:5873:29)
    at tryCallOne (address at InternalBytecode.js:1:1180)
    at anonymous (address at InternalBytecode.js:1:1874)
    at apply (native)
    at anonymous (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:14855:26)
    at _callTimer (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:14734:17)
    at _callReactNativeMicrotasksPass (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:14779:17)
    at callReactNativeMicrotasks (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:14985:44)
    at __callReactNativeMicrotasks (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:2644:48)
    at anonymous (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:2417:45)
    at __guard (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:2616:15)
    at flushedQueue (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:2416:21)
    at callFunctionReturnFlushedQueue (http://131.155.185.206:8081/node_modules/expo-router/entry.bundle//&platform=android&dev=true&hot=false&lazy=true&transform.engine=hermes&transform.bytecode=true&transform.routerRoot=app:2401:33)
 LOG  11:28:46 AM | DEBUG : [app/dev/state.tsx] Test Non-Stream Response: {"candidates":[{"content":{"parts":[{"text":"Hello there! How can I help you today?\n"}],"role":"model"},"finishReason":"STOP","avgLogprobs":-0.0011375560848550363}],"usageMetadata":{"promptTokenCount":2,"candidatesTokenCount":11,"totalTokenCount":13},"modelVersion":"gemini-1.5-flash"}

As you can see it doesn't return any chunks.

[hkt74]

your code snippet actually works for me.

 const genAI = new GoogleGenerativeAI(apiKey);
 const model = genAI.getGenerativeModel({model: "gemini-1.5-flash"});
 const chat = model.startChat({});
 try {
   console.log("Test Stream Start");
   const request = await chat.sendMessageStream("Hello");
   try {
     for await (const chunk of request.stream) {
       if (chunk.promptFeedback) {
         console.log(`${chunk.promptFeedback.blockReason} ${chunk.promptFeedback.safetyRatings} ${chunk.promptFeedback.blockReasonMessage}`);
       }
       console.log(`Test Stream Chunk: ${chunk.text()}`);
     }
   } catch (e) {
     console.log("Error with retrieving stream from request", e);
     if (e instanceof Error) {
      console.log(e.stack);
     }
   }
   const response = await request.response;
   console.log(`Test Stream Response: ${JSON.stringify(response)}`);
 } catch (e) {
   console.log("Error with retrieving response from request", e);
   if (e instanceof Error) {
    console.log(e.stack);
   }
 }

console output:

Test Stream Start
Test Stream Chunk: Hello
Test Stream Chunk:  there! How can I help you today?

Test Stream Response: {"candidates":[],"usageMetadata":{"promptTokenCount":2,"candidatesTokenCount":11,"totalTokenCount":13}}

could you share your npm version and node version? did you install the package from local repo or the public npm registry? thanks!

[svdgoor]

I'm currently away for holiday. Please leave the issue open for 2 weeks so I can rerun my tests.

Very interesting...

[svdgoor]

Yeah it's still not working. Exact same code.
npm@10.9.0, Welcome to Node.js v20.17.0.

Installed from public. Installed all my dependencies completely from scratch today again, no difference,

[hkt74]

The stack trace you provided is difficult to analyze. It's unclear whether the issue originates from the SDK or the API.

To isolate the problem, could you please try running the code snippet independently of your application?

thanks!

[svdgoor]

Currently unable to perform this test, will loop back around

[H9660]

@svdgoor @hkt74 This works well on my machine though. (Windows 10)

import { GoogleGenerativeAI } from "@google/generative-ai";

const genAI = new GoogleGenerativeAI(process.env.API_KEY);

const model = genAI.getGenerativeModel({model: "gemini-1.5-flash"});
const chat = model.startChat({});
try {
  const request = await chat.sendMessageStream("Hello");
  for await (const chunk of request.stream) {
    if (chunk.promptFeedback) {
      error(`${chunk.promptFeedback.blockReason} ${chunk.promptFeedback.safetyRatings} ${chunk.promptFeedback.blockReasonMessage}`);
    }
  }
  const response = await request.response;
  console.log(`Test Stream Response: ${JSON.stringify(response)}`);
} catch (e) {
  console.error(e);
}
try {
  const request = await chat.sendMessage("Hello");
  const response = request.response;
  console.log(`Test Non-Stream Response: ${JSON.stringify(response)}`);
} catch (e) {
    console.error(e);
}

[svdgoor]

Hey @H9660 your API key is in the message. Resit it!