Skip to content

feat(vertexai): Migrate to AI and add GoogleAI support #8931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
May 12, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c3ec037
feat(vertexai): Migrate to `GenAI` and add GoogleAI support
dlarocque Apr 15, 2025
4e831d1
Convert `GenAI` to new Firebase AI naming
dlarocque Apr 17, 2025
5a78b76
Convert backend types to classes
dlarocque Apr 22, 2025
02600d0
Cleanup
dlarocque Apr 22, 2025
ff62500
Move GoogleAI types to single file
dlarocque Apr 22, 2025
48fc75c
Format
dlarocque Apr 22, 2025
cee1fae
Encode/decode instance identifiers directly to/from backends
dlarocque Apr 22, 2025
475c81a
Update changeset
dlarocque Apr 23, 2025
730f460
Cleanup
dlarocque Apr 23, 2025
0e94110
fix(vertexai): pass `GenerativeModel`'s `BaseParams` to `ChatSession`
dlarocque Apr 24, 2025
15d9699
Fix generative model tests to use fakeAI
dlarocque Apr 28, 2025
6a1e02d
Add `packages/firebase/ai` directory for legacy resolvers
dlarocque Apr 29, 2025
dbdb762
Cleanup docs
dlarocque Apr 29, 2025
41b0385
[vertexai] Use json of unary-success-citations (#8981)
rlazo Apr 29, 2025
1d3b922
Use default location if location is empty string
dlarocque Apr 30, 2025
ee3e2a1
Merge branch 'main' into dl/genai
dlarocque Apr 30, 2025
44870ba
Merge branch 'dl/genai' of https://github.com/firebase/firebase-js-sd…
dlarocque Apr 30, 2025
4b6ab32
Replace 'vertexAI' error prefix with 'AI'
dlarocque May 1, 2025
1a41a92
Use Gemini Developer API and Gemini API in Vertex AI naming in docs
dlarocque May 2, 2025
cc1726d
update changeset
dlarocque May 2, 2025
6d8b119
Merge branch 'main' into dl/genai
dlarocque May 2, 2025
a516771
Update to Vertex AI Gemini API
dlarocque May 2, 2025
ea28656
Revert eslint change to fix firestore docs????
dlarocque May 6, 2025
c9f7113
Merge branch 'main' into dl/genai
dlarocque May 8, 2025
9529e08
docs fixes
dlarocque May 11, 2025
c4878c1
update toc
dlarocque May 11, 2025
07f68ad
docs review fixes
dlarocque May 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/tall-zoos-stare.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'firebase': minor
'@firebase/vertexai': minor
---

Add support for the Google AI API, enabling usage in a free tier, and migrate from `VertexAI` naming to `GenAI`.
137 changes: 95 additions & 42 deletions common/api-review/vertexai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,18 @@ export class ArraySchema extends Schema {
toJSON(): SchemaRequest;
}

// @public
export type Backend = GoogleAIBackend | VertexAIBackend;

// @public
export const BackendType: {
readonly VERTEX_AI: "VERTEX_AI";
readonly GOOGLE_AI: "GOOGLE_AI";
};

// @public
export type BackendType = (typeof BackendType)[keyof typeof BackendType];

// @public
export interface BaseParams {
// (undocumented)
Expand Down Expand Up @@ -239,6 +251,60 @@ export interface FunctionResponsePart {
text?: never;
}

// @public
export interface GenAI {
app: FirebaseApp;
backend: Backend;
// @deprecated
location: string;
}

// @public
export class GenAIError extends FirebaseError {
constructor(code: GenAIErrorCode, message: string, customErrorData?: CustomErrorData | undefined);
// (undocumented)
readonly code: GenAIErrorCode;
// (undocumented)
readonly customErrorData?: CustomErrorData | undefined;
}

// @public
const enum GenAIErrorCode {
API_NOT_ENABLED = "api-not-enabled",
ERROR = "error",
FETCH_ERROR = "fetch-error",
INVALID_CONTENT = "invalid-content",
INVALID_SCHEMA = "invalid-schema",
NO_API_KEY = "no-api-key",
NO_APP_ID = "no-app-id",
NO_MODEL = "no-model",
NO_PROJECT_ID = "no-project-id",
PARSE_FAILED = "parse-failed",
REQUEST_ERROR = "request-error",
RESPONSE_ERROR = "response-error",
UNSUPPORTED = "unsupported"
}

export { GenAIErrorCode }

export { GenAIErrorCode as VertexAIErrorCode }

// @public
export abstract class GenAIModel {
// @internal
protected constructor(genAI: GenAI, modelName: string);
// @internal (undocumented)
protected _apiSettings: ApiSettings;
readonly model: string;
// @internal
static normalizeModelName(modelName: string, backendType: BackendType): string;
}

// @public
export interface GenAIOptions {
backend: Backend;
}

// @public
export interface GenerateContentCandidate {
// (undocumented)
Expand Down Expand Up @@ -323,8 +389,8 @@ export interface GenerativeContentBlob {
}

// @public
export class GenerativeModel extends VertexAIModel {
constructor(vertexAI: VertexAI, modelParams: ModelParams, requestOptions?: RequestOptions);
export class GenerativeModel extends GenAIModel {
constructor(genAI: GenAI, modelParams: ModelParams, requestOptions?: RequestOptions);
countTokens(request: CountTokensRequest | string | Array<string | Part>): Promise<CountTokensResponse>;
generateContent(request: GenerateContentRequest | string | Array<string | Part>): Promise<GenerateContentResult>;
generateContentStream(request: GenerateContentRequest | string | Array<string | Part>): Promise<GenerateContentStreamResult>;
Expand All @@ -344,14 +410,25 @@ export class GenerativeModel extends VertexAIModel {
}

// @public
export function getGenerativeModel(vertexAI: VertexAI, modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;
export function getGenAI(app?: FirebaseApp, options?: GenAIOptions): GenAI;

// @public
export function getGenerativeModel(genAI: GenAI, modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;

// @beta
export function getImagenModel(vertexAI: VertexAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions): ImagenModel;
export function getImagenModel(genAI: GenAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions): ImagenModel;

// @public
export function getVertexAI(app?: FirebaseApp, options?: VertexAIOptions): VertexAI;

// @public
export type GoogleAIBackend = {
backendType: typeof BackendType.GOOGLE_AI;
};

// @public
export function googleAIBackend(): GoogleAIBackend;

// @public @deprecated (undocumented)
export interface GroundingAttribution {
// (undocumented)
Expand All @@ -374,7 +451,7 @@ export interface GroundingMetadata {
webSearchQueries?: string[];
}

// @public (undocumented)
// @public
export enum HarmBlockMethod {
PROBABILITY = "PROBABILITY",
SEVERITY = "SEVERITY"
Expand Down Expand Up @@ -413,7 +490,8 @@ export enum HarmSeverity {
HARM_SEVERITY_HIGH = "HARM_SEVERITY_HIGH",
HARM_SEVERITY_LOW = "HARM_SEVERITY_LOW",
HARM_SEVERITY_MEDIUM = "HARM_SEVERITY_MEDIUM",
HARM_SEVERITY_NEGLIGIBLE = "HARM_SEVERITY_NEGLIGIBLE"
HARM_SEVERITY_NEGLIGIBLE = "HARM_SEVERITY_NEGLIGIBLE",
HARM_SEVERITY_UNSUPPORTED = "HARM_SEVERITY_UNSUPPORTED"
}

// @beta
Expand Down Expand Up @@ -461,8 +539,8 @@ export interface ImagenInlineImage {
}

// @beta
export class ImagenModel extends VertexAIModel {
constructor(vertexAI: VertexAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions | undefined);
export class ImagenModel extends GenAIModel {
constructor(genAI: GenAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions | undefined);
generateImages(prompt: string): Promise<ImagenGenerationResponse<ImagenInlineImage>>;
// @internal
generateImagesGCS(prompt: string, gcsURI: string): Promise<ImagenGenerationResponse<ImagenGCSImage>>;
Expand Down Expand Up @@ -627,7 +705,6 @@ export interface SafetyRating {
export interface SafetySetting {
// (undocumented)
category: HarmCategory;
// (undocumented)
method?: HarmBlockMethod;
// (undocumented)
threshold: HarmBlockThreshold;
Expand Down Expand Up @@ -779,46 +856,22 @@ export interface UsageMetadata {
}

// @public
export interface VertexAI {
app: FirebaseApp;
// (undocumented)
export type VertexAI = GenAI;

// @public
export type VertexAIBackend = {
backendType: typeof BackendType.VERTEX_AI;
location: string;
}
};

// @public
export class VertexAIError extends FirebaseError {
constructor(code: VertexAIErrorCode, message: string, customErrorData?: CustomErrorData | undefined);
// (undocumented)
readonly code: VertexAIErrorCode;
// (undocumented)
readonly customErrorData?: CustomErrorData | undefined;
}
export function vertexAIBackend(location?: string): VertexAIBackend;

// @public
export const enum VertexAIErrorCode {
API_NOT_ENABLED = "api-not-enabled",
ERROR = "error",
FETCH_ERROR = "fetch-error",
INVALID_CONTENT = "invalid-content",
INVALID_SCHEMA = "invalid-schema",
NO_API_KEY = "no-api-key",
NO_APP_ID = "no-app-id",
NO_MODEL = "no-model",
NO_PROJECT_ID = "no-project-id",
PARSE_FAILED = "parse-failed",
REQUEST_ERROR = "request-error",
RESPONSE_ERROR = "response-error"
}
export const VertexAIError: typeof GenAIError;

// @public
export abstract class VertexAIModel {
// @internal
protected constructor(vertexAI: VertexAI, modelName: string);
// @internal (undocumented)
protected _apiSettings: ApiSettings;
readonly model: string;
static normalizeModelName(modelName: string): string;
}
export const VertexAIModel: typeof GenAIModel;

// @public
export interface VertexAIOptions {
Expand Down
14 changes: 8 additions & 6 deletions docs-devsite/_toc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -516,6 +516,14 @@ toc:
path: /docs/reference/js/vertexai.functionresponse.md
- title: FunctionResponsePart
path: /docs/reference/js/vertexai.functionresponsepart.md
- title: GenAI
path: /docs/reference/js/vertexai.genai.md
- title: GenAIError
path: /docs/reference/js/vertexai.genaierror.md
- title: GenAIModel
path: /docs/reference/js/vertexai.genaimodel.md
- title: GenAIOptions
path: /docs/reference/js/vertexai.genaioptions.md
- title: GenerateContentCandidate
path: /docs/reference/js/vertexai.generatecontentcandidate.md
- title: GenerateContentRequest
Expand Down Expand Up @@ -598,12 +606,6 @@ toc:
path: /docs/reference/js/vertexai.toolconfig.md
- title: UsageMetadata
path: /docs/reference/js/vertexai.usagemetadata.md
- title: VertexAI
path: /docs/reference/js/vertexai.vertexai.md
- title: VertexAIError
path: /docs/reference/js/vertexai.vertexaierror.md
- title: VertexAIModel
path: /docs/reference/js/vertexai.vertexaimodel.md
Comment on lines -601 to -606
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These types still exist (can't remove them until a breaking change), but our doc generation tool doesn't pick them up since they're just aliases to the new types (e.g. VertexAIError is just an alias to AIError.

- title: VertexAIOptions
path: /docs/reference/js/vertexai.vertexaioptions.md
- title: VideoMetadata
Expand Down
64 changes: 64 additions & 0 deletions docs-devsite/vertexai.genai.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
Project: /docs/reference/js/_project.yaml
Book: /docs/reference/_book.yaml
page_type: reference

{% comment %}
DO NOT EDIT THIS FILE!
This is generated by the JS SDK team, and any local changes will be
overwritten. Changes should be made in the source code at
https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# GenAI interface
An instance of the Firebase GenAI SDK.

Do not create this instance directly. Instead, use [getGenAI()](./vertexai.md#getgenai_65c48ee)<!-- -->.

<b>Signature:</b>

```typescript
export interface GenAI
```

## Properties

| Property | Type | Description |
| --- | --- | --- |
| [app](./vertexai.genai.md#genaiapp) | [FirebaseApp](./app.firebaseapp.md#firebaseapp_interface) | The [FirebaseApp](./app.firebaseapp.md#firebaseapp_interface) this [GenAI](./vertexai.genai.md#genai_interface) instance is associated with. |
| [backend](./vertexai.genai.md#genaibackend) | [Backend](./vertexai.md#backend) | A [Backend](./vertexai.md#backend) instance that specifies the backend configuration. |
| [location](./vertexai.genai.md#genailocation) | string | The location configured for this GenAI service instance, relevant for Vertex AI backends. |

## GenAI.app

The [FirebaseApp](./app.firebaseapp.md#firebaseapp_interface) this [GenAI](./vertexai.genai.md#genai_interface) instance is associated with.

<b>Signature:</b>

```typescript
app: FirebaseApp;
```

## GenAI.backend

A [Backend](./vertexai.md#backend) instance that specifies the backend configuration.

<b>Signature:</b>

```typescript
backend: Backend;
```

## GenAI.location

> Warning: This API is now obsolete.
>
> use `GenAI.backend.location` instead.
>

The location configured for this GenAI service instance, relevant for Vertex AI backends.

<b>Signature:</b>

```typescript
location: string;
```
Original file line number Diff line number Diff line change
Expand Up @@ -9,56 +9,56 @@ overwritten. Changes should be made in the source code at
https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# VertexAIError class
# GenAIError class
Error class for the Vertex AI in Firebase SDK.

<b>Signature:</b>

```typescript
export declare class VertexAIError extends FirebaseError
export declare class GenAIError extends FirebaseError
```
<b>Extends:</b> [FirebaseError](./util.firebaseerror.md#firebaseerror_class)

## Constructors

| Constructor | Modifiers | Description |
| --- | --- | --- |
| [(constructor)(code, message, customErrorData)](./vertexai.vertexaierror.md#vertexaierrorconstructor) | | Constructs a new instance of the <code>VertexAIError</code> class. |
| [(constructor)(code, message, customErrorData)](./vertexai.genaierror.md#genaierrorconstructor) | | Constructs a new instance of the <code>GenAIError</code> class. |

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [code](./vertexai.vertexaierror.md#vertexaierrorcode) | | [VertexAIErrorCode](./vertexai.md#vertexaierrorcode) | |
| [customErrorData](./vertexai.vertexaierror.md#vertexaierrorcustomerrordata) | | [CustomErrorData](./vertexai.customerrordata.md#customerrordata_interface) \| undefined | |
| [code](./vertexai.genaierror.md#genaierrorcode) | | [GenAIErrorCode](./vertexai.md#genaierrorcode) | |
| [customErrorData](./vertexai.genaierror.md#genaierrorcustomerrordata) | | [CustomErrorData](./vertexai.customerrordata.md#customerrordata_interface) \| undefined | |

## VertexAIError.(constructor)
## GenAIError.(constructor)

Constructs a new instance of the `VertexAIError` class.
Constructs a new instance of the `GenAIError` class.

<b>Signature:</b>

```typescript
constructor(code: VertexAIErrorCode, message: string, customErrorData?: CustomErrorData | undefined);
constructor(code: GenAIErrorCode, message: string, customErrorData?: CustomErrorData | undefined);
```

#### Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| code | [VertexAIErrorCode](./vertexai.md#vertexaierrorcode) | The error code from [VertexAIErrorCode](./vertexai.md#vertexaierrorcode)<!-- -->. |
| code | [GenAIErrorCode](./vertexai.md#genaierrorcode) | The error code from [GenAIErrorCode](./vertexai.md#genaierrorcode)<!-- -->. |
| message | string | A human-readable message describing the error. |
| customErrorData | [CustomErrorData](./vertexai.customerrordata.md#customerrordata_interface) \| undefined | Optional error data. |

## VertexAIError.code
## GenAIError.code

<b>Signature:</b>

```typescript
readonly code: VertexAIErrorCode;
readonly code: GenAIErrorCode;
```

## VertexAIError.customErrorData
## GenAIError.customErrorData

<b>Signature:</b>

Expand Down
Loading
Loading