Skip to content

Commit c9f7113

Browse files
dlarocquedlarocque
committed
Merge branch 'main' into dl/genai
2 parents ea28656 + e99683b commit c9f7113

File tree

81 files changed

+822
-206
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+822
-206
lines changed

‎.changeset/fast-mangos-chew.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@firebase/vertexai': patch
3+
---
4+
5+
Pass `GenerativeModel`'s `BaseParams` to created chat sessions. This fixes an issue where `GenerationConfig` would not be inherited from `ChatSession`.

‎.changeset/nice-plants-thank.md

Lines changed: 0 additions & 10 deletions
This file was deleted.

‎.changeset/nine-pugs-crash.md

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎.changeset/perfect-camels-try.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'firebase': minor
3+
'@firebase/vertexai': minor
4+
---
5+
6+
Add support for Gemini multimodal output

‎common/api-review/vertexai.api.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,7 @@ export { Date_2 as Date }
192192
export interface EnhancedGenerateContentResponse extends GenerateContentResponse {
193193
// (undocumented)
194194
functionCalls: () => FunctionCall[] | undefined;
195+
inlineDataParts: () => InlineDataPart[] | undefined;
195196
text: () => string;
196197
}
197198

@@ -372,6 +373,8 @@ export interface GenerationConfig {
372373
// (undocumented)
373374
presencePenalty?: number;
374375
responseMimeType?: string;
376+
// @beta
377+
responseModalities?: ResponseModality[];
375378
responseSchema?: TypedSchema | SchemaRequest;
376379
// (undocumented)
377380
stopSequences?: string[];
@@ -726,6 +729,15 @@ export interface RequestOptions {
726729
timeout?: number;
727730
}
728731

732+
// @beta
733+
export const ResponseModality: {
734+
readonly TEXT: "TEXT";
735+
readonly IMAGE: "IMAGE";
736+
};
737+
738+
// @beta
739+
export type ResponseModality = (typeof ResponseModality)[keyof typeof ResponseModality];
740+
729741
// @public (undocumented)
730742
export interface RetrievedContextAttribution {
731743
// (undocumented)

‎docs-devsite/vertexai.enhancedgeneratecontentresponse.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ export interface EnhancedGenerateContentResponse extends GenerateContentResponse
2424
| Property | Type | Description |
2525
| --- | --- | --- |
2626
| [functionCalls](./vertexai.enhancedgeneratecontentresponse.md#enhancedgeneratecontentresponsefunctioncalls) | () =&gt; [FunctionCall](./vertexai.functioncall.md#functioncall_interface)<!-- -->\[\] \| undefined | |
27+
| [inlineDataParts](./vertexai.enhancedgeneratecontentresponse.md#enhancedgeneratecontentresponseinlinedataparts) | () =&gt; [InlineDataPart](./vertexai.inlinedatapart.md#inlinedatapart_interface)<!-- -->\[\] \| undefined | Aggregates and returns all [InlineDataPart](./vertexai.inlinedatapart.md#inlinedatapart_interface)<!-- -->s from the [GenerateContentResponse](./vertexai.generatecontentresponse.md#generatecontentresponse_interface)<!-- -->'s first candidate. |
2728
| [text](./vertexai.enhancedgeneratecontentresponse.md#enhancedgeneratecontentresponsetext) | () =&gt; string | Returns the text string from the response, if available. Throws if the prompt or candidate was blocked. |
2829
2930
## EnhancedGenerateContentResponse.functionCalls
@@ -34,6 +35,16 @@ export interface EnhancedGenerateContentResponse extends GenerateContentResponse
3435
functionCalls: () => FunctionCall[] | undefined;
3536
```
3637
38+
## EnhancedGenerateContentResponse.inlineDataParts
39+
40+
Aggregates and returns all [InlineDataPart](./vertexai.inlinedatapart.md#inlinedatapart_interface)<!-- -->s from the [GenerateContentResponse](./vertexai.generatecontentresponse.md#generatecontentresponse_interface)<!-- -->'s first candidate.
41+
42+
<b>Signature:</b>
43+
44+
```typescript
45+
inlineDataParts: () => InlineDataPart[] | undefined;
46+
```
47+
3748
## EnhancedGenerateContentResponse.text
3849
3950
Returns the text string from the response, if available. Throws if the prompt or candidate was blocked.

‎docs-devsite/vertexai.generationconfig.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ export interface GenerationConfig
2727
| [maxOutputTokens](./vertexai.generationconfig.md#generationconfigmaxoutputtokens) | number | |
2828
| [presencePenalty](./vertexai.generationconfig.md#generationconfigpresencepenalty) | number | |
2929
| [responseMimeType](./vertexai.generationconfig.md#generationconfigresponsemimetype) | string | Output response MIME type of the generated candidate text. Supported MIME types are <code>text/plain</code> (default, text output), <code>application/json</code> (JSON response in the candidates), and <code>text/x.enum</code>. |
30+
| [responseModalities](./vertexai.generationconfig.md#generationconfigresponsemodalities) | [ResponseModality](./vertexai.md#responsemodality)<!-- -->\[\] | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
3031
| [responseSchema](./vertexai.generationconfig.md#generationconfigresponseschema) | [TypedSchema](./vertexai.md#typedschema) \| [SchemaRequest](./vertexai.schemarequest.md#schemarequest_interface) | Output response schema of the generated candidate text. This value can be a class generated with a [Schema](./vertexai.schema.md#schema_class) static method like <code>Schema.string()</code> or <code>Schema.object()</code> or it can be a plain JS object matching the [SchemaRequest](./vertexai.schemarequest.md#schemarequest_interface) interface. <br/>Note: This only applies when the specified <code>responseMIMEType</code> supports a schema; currently this is limited to <code>application/json</code> and <code>text/x.enum</code>. |
3132
| [stopSequences](./vertexai.generationconfig.md#generationconfigstopsequences) | string\[\] | |
3233
| [temperature](./vertexai.generationconfig.md#generationconfigtemperature) | number | |
@@ -75,6 +76,21 @@ Output response MIME type of the generated candidate text. Supported MIME types
7576
responseMimeType?: string;
7677
```
7778

79+
## GenerationConfig.responseModalities
80+
81+
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
82+
>
83+
84+
Generation modalities to be returned in generation responses.
85+
86+
- Multimodal response generation is only supported by some Gemini models and versions; see [model versions](https://firebase.google.com/docs/vertex-ai/models)<!-- -->. - Only image generation (`ResponseModality.IMAGE`<!-- -->) is supported.
87+
88+
<b>Signature:</b>
89+
90+
```typescript
91+
responseModalities?: ResponseModality[];
92+
```
93+
7894
## GenerationConfig.responseSchema
7995

8096
Output response schema of the generated candidate text. This value can be a class generated with a [Schema](./vertexai.schema.md#schema_class) static method like `Schema.string()` or `Schema.object()` or it can be a plain JS object matching the [SchemaRequest](./vertexai.schemarequest.md#schemarequest_interface) interface. <br/>Note: This only applies when the specified `responseMIMEType` supports a schema; currently this is limited to `application/json` and `text/x.enum`<!-- -->.

‎docs-devsite/vertexai.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,7 @@ The Firebase AI Web SDK.
131131
| --- | --- |
132132
| [BackendType](./vertexai.md#backendtype) | An enum-like object containing constants that represent the supported backends for the Firebase AI SDK. This determines which backend service (Vertex AI Gemini API or Gemini Developer API) the SDK will communicate with.<!-- -->These values are assigned to the <code>backendType</code> property within the specific backend configuration objects ([GoogleAIBackend](./vertexai.googleaibackend.md#googleaibackend_class) or [VertexAIBackend](./vertexai.vertexaibackend.md#vertexaibackend_class)<!-- -->) to identify which service to target. |
133133
| [POSSIBLE\_ROLES](./vertexai.md#possible_roles) | Possible roles. |
134+
| [ResponseModality](./vertexai.md#responsemodality) | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
134135
| [VertexAIError](./vertexai.md#vertexaierror) | Error class for the Firebase AI SDK.<!-- -->For more information, refer to the documentation for the new [AIError](./vertexai.aierror.md#aierror_class)<!-- -->. |
135136
| [VertexAIModel](./vertexai.md#vertexaimodel) | Base class for Firebase AI model APIs.<!-- -->For more information, refer to the documentation for the new [AIModel](./vertexai.aimodel.md#aimodel_class)<!-- -->. |
136137

@@ -140,6 +141,7 @@ The Firebase AI Web SDK.
140141
| --- | --- |
141142
| [BackendType](./vertexai.md#backendtype) | Type alias representing valid backend types. It can be either <code>'VERTEX_AI'</code> or <code>'GOOGLE_AI'</code>. |
142143
| [Part](./vertexai.md#part) | Content part - includes text, image/video, or function call/response part types. |
144+
| [ResponseModality](./vertexai.md#responsemodality) | <b><i>(Public Preview)</i></b> Generation modalities to be returned in generation responses. |
143145
| [Role](./vertexai.md#role) | Role is the producer of the content. |
144146
| [Tool](./vertexai.md#tool) | Defines a tool that model can call to access external knowledge. |
145147
| [TypedSchema](./vertexai.md#typedschema) | A type that includes all specific Schema types. |
@@ -299,6 +301,22 @@ Possible roles.
299301
POSSIBLE_ROLES: readonly ["user", "model", "function", "system"]
300302
```
301303

304+
## ResponseModality
305+
306+
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
307+
>
308+
309+
Generation modalities to be returned in generation responses.
310+
311+
<b>Signature:</b>
312+
313+
```typescript
314+
ResponseModality: {
315+
readonly TEXT: "TEXT";
316+
readonly IMAGE: "IMAGE";
317+
}
318+
```
319+
302320
## VertexAIError
303321

304322
Error class for the Firebase AI SDK.
@@ -343,6 +361,19 @@ Content part - includes text, image/video, or function call/response part types.
343361
export type Part = TextPart | InlineDataPart | FunctionCallPart | FunctionResponsePart | FileDataPart;
344362
```
345363

364+
## ResponseModality
365+
366+
> This API is provided as a preview for developers and may change based on feedback that we receive. Do not use this API in a production environment.
367+
>
368+
369+
Generation modalities to be returned in generation responses.
370+
371+
<b>Signature:</b>
372+
373+
```typescript
374+
export type ResponseModality = (typeof ResponseModality)[keyof typeof ResponseModality];
375+
```
376+
346377
## Role
347378

348379
Role is the producer of the content.

‎integration/compat-interop/package.json

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,20 +8,20 @@
88
"test:debug": "karma start --browsers Chrome --auto-watch"
99
},
1010
"dependencies": {
11-
"@firebase/app": "0.11.5",
12-
"@firebase/app-compat": "0.2.54",
13-
"@firebase/analytics": "0.10.12",
14-
"@firebase/analytics-compat": "0.2.18",
15-
"@firebase/auth": "1.10.1",
16-
"@firebase/auth-compat": "0.5.21",
17-
"@firebase/functions": "0.12.3",
18-
"@firebase/functions-compat": "0.3.20",
19-
"@firebase/messaging": "0.12.17",
20-
"@firebase/messaging-compat": "0.2.17",
21-
"@firebase/performance": "0.7.2",
22-
"@firebase/performance-compat": "0.2.15",
23-
"@firebase/remote-config": "0.6.0",
24-
"@firebase/remote-config-compat": "0.2.13"
11+
"@firebase/app": "0.12.1",
12+
"@firebase/app-compat": "0.3.1",
13+
"@firebase/analytics": "0.10.13",
14+
"@firebase/analytics-compat": "0.2.19",
15+
"@firebase/auth": "1.10.2",
16+
"@firebase/auth-compat": "0.5.22",
17+
"@firebase/functions": "0.12.4",
18+
"@firebase/functions-compat": "0.3.21",
19+
"@firebase/messaging": "0.12.18",
20+
"@firebase/messaging-compat": "0.2.18",
21+
"@firebase/performance": "0.7.3",
22+
"@firebase/performance-compat": "0.2.16",
23+
"@firebase/remote-config": "0.6.1",
24+
"@firebase/remote-config-compat": "0.2.14"
2525
},
2626
"devDependencies": {
2727
"typescript": "5.5.4"

‎integration/firestore/package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,8 @@
1414
"test:memory:debug": "yarn build:memory; karma start --auto-watch --browsers Chrome"
1515
},
1616
"dependencies": {
17-
"@firebase/app": "0.11.5",
18-
"@firebase/firestore": "4.7.11"
17+
"@firebase/app": "0.12.1",
18+
"@firebase/firestore": "4.7.12"
1919
},
2020
"devDependencies": {
2121
"@types/mocha": "9.1.1",

‎integration/messaging/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
"test:manual": "mocha --exit"
1010
},
1111
"devDependencies": {
12-
"firebase": "11.6.1",
12+
"firebase": "11.7.1",
1313
"chai": "4.5.0",
1414
"chromedriver": "119.0.1",
1515
"express": "4.21.2",

‎packages/analytics-compat/CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
# @firebase/analytics-compat
22

3+
## 0.2.19
4+
5+
### Patch Changes
6+
7+
- Updated dependencies [[`ea1f913`](https://github.com/firebase/firebase-js-sdk/commit/ea1f9139e6baec0269fbb91233fd3f7f4b0d5875), [`0e12766`](https://github.com/firebase/firebase-js-sdk/commit/0e127664946ba324c6566a02b393dafd23fc1ddb)]:
8+
- @firebase/util@1.11.1
9+
- @firebase/analytics@0.10.13
10+
- @firebase/component@0.6.14
11+
312
## 0.2.18
413

514
### Patch Changes

‎packages/analytics-compat/package.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@firebase/analytics-compat",
3-
"version": "0.2.18",
3+
"version": "0.2.19",
44
"description": "",
55
"author": "Firebase <[email protected]> (https://firebase.google.com/)",
66
"main": "dist/index.cjs.js",
@@ -22,7 +22,7 @@
2222
"@firebase/app-compat": "0.x"
2323
},
2424
"devDependencies": {
25-
"@firebase/app-compat": "0.2.54",
25+
"@firebase/app-compat": "0.3.1",
2626
"rollup": "2.79.2",
2727
"@rollup/plugin-json": "6.1.0",
2828
"rollup-plugin-typescript2": "0.36.0",
@@ -52,10 +52,10 @@
5252
},
5353
"typings": "dist/src/index.d.ts",
5454
"dependencies": {
55-
"@firebase/component": "0.6.13",
56-
"@firebase/analytics": "0.10.12",
55+
"@firebase/component": "0.6.14",
56+
"@firebase/analytics": "0.10.13",
5757
"@firebase/analytics-types": "0.8.3",
58-
"@firebase/util": "1.11.0",
58+
"@firebase/util": "1.11.1",
5959
"tslib": "^2.1.0"
6060
},
6161
"nyc": {

‎packages/analytics/CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
# @firebase/analytics
22

3+
## 0.10.13
4+
5+
### Patch Changes
6+
7+
- Updated dependencies [[`ea1f913`](https://github.com/firebase/firebase-js-sdk/commit/ea1f9139e6baec0269fbb91233fd3f7f4b0d5875), [`0e12766`](https://github.com/firebase/firebase-js-sdk/commit/0e127664946ba324c6566a02b393dafd23fc1ddb)]:
8+
- @firebase/util@1.11.1
9+
- @firebase/installations@0.6.14
10+
- @firebase/component@0.6.14
11+
312
## 0.10.12
413

514
### Patch Changes

‎packages/analytics/package.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@firebase/analytics",
3-
"version": "0.10.12",
3+
"version": "0.10.13",
44
"description": "A analytics package for new firebase packages",
55
"author": "Firebase <[email protected]> (https://firebase.google.com/)",
66
"main": "dist/index.cjs.js",
@@ -39,15 +39,15 @@
3939
"@firebase/app": "0.x"
4040
},
4141
"dependencies": {
42-
"@firebase/installations": "0.6.13",
42+
"@firebase/installations": "0.6.14",
4343
"@firebase/logger": "0.4.4",
44-
"@firebase/util": "1.11.0",
45-
"@firebase/component": "0.6.13",
44+
"@firebase/util": "1.11.1",
45+
"@firebase/component": "0.6.14",
4646
"tslib": "^2.1.0"
4747
},
4848
"license": "Apache-2.0",
4949
"devDependencies": {
50-
"@firebase/app": "0.11.5",
50+
"@firebase/app": "0.12.1",
5151
"rollup": "2.79.2",
5252
"@rollup/plugin-commonjs": "21.1.0",
5353
"@rollup/plugin-json": "6.1.0",

‎packages/app-check-compat/CHANGELOG.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,21 @@
11
# @firebase/app-check-compat
22

3+
## 0.3.22
4+
5+
### Patch Changes
6+
7+
- Updated dependencies [[`51e7b48`](https://github.com/firebase/firebase-js-sdk/commit/51e7b489d8aadd531453f882421903da8727b19d)]:
8+
- @firebase/app-check@0.9.1
9+
10+
## 0.3.21
11+
12+
### Patch Changes
13+
14+
- Updated dependencies [[`3789b5a`](https://github.com/firebase/firebase-js-sdk/commit/3789b5ad16ffd462fce1d0b9c2e9ffae373bc6eb), [`ea1f913`](https://github.com/firebase/firebase-js-sdk/commit/ea1f9139e6baec0269fbb91233fd3f7f4b0d5875), [`0e12766`](https://github.com/firebase/firebase-js-sdk/commit/0e127664946ba324c6566a02b393dafd23fc1ddb)]:
15+
- @firebase/app-check@0.9.0
16+
- @firebase/util@1.11.1
17+
- @firebase/component@0.6.14
18+
319
## 0.3.20
420

521
### Patch Changes

‎packages/app-check-compat/package.json

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@firebase/app-check-compat",
3-
"version": "0.3.20",
3+
"version": "0.3.22",
44
"description": "A compat App Check package for new firebase packages",
55
"author": "Firebase <[email protected]> (https://firebase.google.com/)",
66
"main": "dist/index.cjs.js",
@@ -34,16 +34,16 @@
3434
"@firebase/app-compat": "0.x"
3535
},
3636
"dependencies": {
37-
"@firebase/app-check": "0.8.13",
37+
"@firebase/app-check": "0.9.1",
3838
"@firebase/app-check-types": "0.5.3",
3939
"@firebase/logger": "0.4.4",
40-
"@firebase/util": "1.11.0",
41-
"@firebase/component": "0.6.13",
40+
"@firebase/util": "1.11.1",
41+
"@firebase/component": "0.6.14",
4242
"tslib": "^2.1.0"
4343
},
4444
"license": "Apache-2.0",
4545
"devDependencies": {
46-
"@firebase/app-compat": "0.2.54",
46+
"@firebase/app-compat": "0.3.1",
4747
"rollup": "2.79.2",
4848
"@rollup/plugin-commonjs": "21.1.0",
4949
"@rollup/plugin-json": "6.1.0",

0 commit comments

Comments
 (0)