Skip to content

feat(vertexai): Migrate to AI and add GoogleAI support #8931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
May 12, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c3ec037
feat(vertexai): Migrate to `GenAI` and add GoogleAI support
dlarocque Apr 15, 2025
4e831d1
Convert `GenAI` to new Firebase AI naming
dlarocque Apr 17, 2025
5a78b76
Convert backend types to classes
dlarocque Apr 22, 2025
02600d0
Cleanup
dlarocque Apr 22, 2025
ff62500
Move GoogleAI types to single file
dlarocque Apr 22, 2025
48fc75c
Format
dlarocque Apr 22, 2025
cee1fae
Encode/decode instance identifiers directly to/from backends
dlarocque Apr 22, 2025
475c81a
Update changeset
dlarocque Apr 23, 2025
730f460
Cleanup
dlarocque Apr 23, 2025
0e94110
fix(vertexai): pass `GenerativeModel`'s `BaseParams` to `ChatSession`
dlarocque Apr 24, 2025
15d9699
Fix generative model tests to use fakeAI
dlarocque Apr 28, 2025
6a1e02d
Add `packages/firebase/ai` directory for legacy resolvers
dlarocque Apr 29, 2025
dbdb762
Cleanup docs
dlarocque Apr 29, 2025
41b0385
[vertexai] Use json of unary-success-citations (#8981)
rlazo Apr 29, 2025
1d3b922
Use default location if location is empty string
dlarocque Apr 30, 2025
ee3e2a1
Merge branch 'main' into dl/genai
dlarocque Apr 30, 2025
44870ba
Merge branch 'dl/genai' of https://github.com/firebase/firebase-js-sd…
dlarocque Apr 30, 2025
4b6ab32
Replace 'vertexAI' error prefix with 'AI'
dlarocque May 1, 2025
1a41a92
Use Gemini Developer API and Gemini API in Vertex AI naming in docs
dlarocque May 2, 2025
cc1726d
update changeset
dlarocque May 2, 2025
6d8b119
Merge branch 'main' into dl/genai
dlarocque May 2, 2025
a516771
Update to Vertex AI Gemini API
dlarocque May 2, 2025
ea28656
Revert eslint change to fix firestore docs????
dlarocque May 6, 2025
c9f7113
Merge branch 'main' into dl/genai
dlarocque May 8, 2025
9529e08
docs fixes
dlarocque May 11, 2025
c4878c1
update toc
dlarocque May 11, 2025
07f68ad
docs review fixes
dlarocque May 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/tall-zoos-stare.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'firebase': minor
'@firebase/vertexai': minor
---

Add support for the Gemini Developer API, enabling usage in a free tier, and add new `AI` API to accomodate new product naming.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this isn't the final changelog that will be in the release notes. This will be changed to the final changelog entry we decide on when we write the release notes.

197 changes: 148 additions & 49 deletions common/api-review/vertexai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,62 @@ import { FirebaseApp } from '@firebase/app';
import { FirebaseAuthTokenData } from '@firebase/auth-interop-types';
import { FirebaseError } from '@firebase/util';

// @public
export interface AI {
app: FirebaseApp;
backend: Backend;
// @deprecated
location: string;
}

// @public
export class AIError extends FirebaseError {
constructor(code: AIErrorCode, message: string, customErrorData?: CustomErrorData | undefined);
// (undocumented)
readonly code: AIErrorCode;
// (undocumented)
readonly customErrorData?: CustomErrorData | undefined;
}

// @public
const enum AIErrorCode {
API_NOT_ENABLED = "api-not-enabled",
ERROR = "error",
FETCH_ERROR = "fetch-error",
INVALID_CONTENT = "invalid-content",
INVALID_SCHEMA = "invalid-schema",
NO_API_KEY = "no-api-key",
NO_APP_ID = "no-app-id",
NO_MODEL = "no-model",
NO_PROJECT_ID = "no-project-id",
PARSE_FAILED = "parse-failed",
REQUEST_ERROR = "request-error",
RESPONSE_ERROR = "response-error",
UNSUPPORTED = "unsupported"
}

export { AIErrorCode }

export { AIErrorCode as VertexAIErrorCode }

// @public
export abstract class AIModel {
// @internal
protected constructor(ai: AI, modelName: string);
// Warning: (ae-forgotten-export) The symbol "ApiSettings" needs to be exported by the entry point index.d.ts
//
// @internal (undocumented)
protected _apiSettings: ApiSettings;
readonly model: string;
// @internal
static normalizeModelName(modelName: string, backendType: BackendType): string;
}

// @public
export interface AIOptions {
backend: Backend;
}

// @public
export class ArraySchema extends Schema {
constructor(schemaParams: SchemaParams, items: TypedSchema);
Expand All @@ -18,6 +74,21 @@ export class ArraySchema extends Schema {
toJSON(): SchemaRequest;
}

// @public
export abstract class Backend {
protected constructor(type: BackendType);
readonly backendType: BackendType;
}

// @public
export const BackendType: {
readonly VERTEX_AI: "VERTEX_AI";
readonly GOOGLE_AI: "GOOGLE_AI";
};

// @public
export type BackendType = (typeof BackendType)[keyof typeof BackendType];

// @public
export interface BaseParams {
// (undocumented)
Expand All @@ -41,7 +112,6 @@ export class BooleanSchema extends Schema {

// @public
export class ChatSession {
// Warning: (ae-forgotten-export) The symbol "ApiSettings" needs to be exported by the entry point index.d.ts
constructor(apiSettings: ApiSettings, model: string, params?: StartChatParams | undefined, requestOptions?: RequestOptions | undefined);
getHistory(): Promise<Content[]>;
// (undocumented)
Expand All @@ -60,11 +130,9 @@ export interface Citation {
endIndex?: number;
// (undocumented)
license?: string;
// (undocumented)
publicationDate?: Date_2;
// (undocumented)
startIndex?: number;
// (undocumented)
title?: string;
// (undocumented)
uri?: string;
Expand Down Expand Up @@ -326,8 +394,8 @@ export interface GenerativeContentBlob {
}

// @public
export class GenerativeModel extends VertexAIModel {
constructor(vertexAI: VertexAI, modelParams: ModelParams, requestOptions?: RequestOptions);
export class GenerativeModel extends AIModel {
constructor(ai: AI, modelParams: ModelParams, requestOptions?: RequestOptions);
countTokens(request: CountTokensRequest | string | Array<string | Part>): Promise<CountTokensResponse>;
generateContent(request: GenerateContentRequest | string | Array<string | Part>): Promise<GenerateContentResult>;
generateContentStream(request: GenerateContentRequest | string | Array<string | Part>): Promise<GenerateContentStreamResult>;
Expand All @@ -347,14 +415,76 @@ export class GenerativeModel extends VertexAIModel {
}

// @public
export function getGenerativeModel(vertexAI: VertexAI, modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;
export function getAI(app?: FirebaseApp, options?: AIOptions): AI;

// @public
export function getGenerativeModel(ai: AI, modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;

// @beta
export function getImagenModel(vertexAI: VertexAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions): ImagenModel;
export function getImagenModel(ai: AI, modelParams: ImagenModelParams, requestOptions?: RequestOptions): ImagenModel;

// @public
export function getVertexAI(app?: FirebaseApp, options?: VertexAIOptions): VertexAI;

// @public
export class GoogleAIBackend extends Backend {
constructor();
}

// Warning: (ae-internal-missing-underscore) The name "GoogleAICitationMetadata" should be prefixed with an underscore because the declaration is marked as @internal
//
// @internal (undocumented)
export interface GoogleAICitationMetadata {
// (undocumented)
citationSources: Citation[];
}

// Warning: (ae-internal-missing-underscore) The name "GoogleAICountTokensRequest" should be prefixed with an underscore because the declaration is marked as @internal
//
// @internal (undocumented)
export interface GoogleAICountTokensRequest {
// (undocumented)
generateContentRequest: {
model: string;
contents: Content[];
systemInstruction?: string | Part | Content;
tools?: Tool[];
generationConfig?: GenerationConfig;
};
}

// Warning: (ae-internal-missing-underscore) The name "GoogleAIGenerateContentCandidate" should be prefixed with an underscore because the declaration is marked as @internal
//
// @internal (undocumented)
export interface GoogleAIGenerateContentCandidate {
// (undocumented)
citationMetadata?: GoogleAICitationMetadata;
// (undocumented)
content: Content;
// (undocumented)
finishMessage?: string;
// (undocumented)
finishReason?: FinishReason;
// (undocumented)
groundingMetadata?: GroundingMetadata;
// (undocumented)
index: number;
// (undocumented)
safetyRatings?: SafetyRating[];
}

// Warning: (ae-internal-missing-underscore) The name "GoogleAIGenerateContentResponse" should be prefixed with an underscore because the declaration is marked as @internal
//
// @internal (undocumented)
export interface GoogleAIGenerateContentResponse {
// (undocumented)
candidates?: GoogleAIGenerateContentCandidate[];
// (undocumented)
promptFeedback?: PromptFeedback;
// (undocumented)
usageMetadata?: UsageMetadata;
}

// @public @deprecated (undocumented)
export interface GroundingAttribution {
// (undocumented)
Expand All @@ -377,7 +507,7 @@ export interface GroundingMetadata {
webSearchQueries?: string[];
}

// @public (undocumented)
// @public
export enum HarmBlockMethod {
PROBABILITY = "PROBABILITY",
SEVERITY = "SEVERITY"
Expand Down Expand Up @@ -416,7 +546,8 @@ export enum HarmSeverity {
HARM_SEVERITY_HIGH = "HARM_SEVERITY_HIGH",
HARM_SEVERITY_LOW = "HARM_SEVERITY_LOW",
HARM_SEVERITY_MEDIUM = "HARM_SEVERITY_MEDIUM",
HARM_SEVERITY_NEGLIGIBLE = "HARM_SEVERITY_NEGLIGIBLE"
HARM_SEVERITY_NEGLIGIBLE = "HARM_SEVERITY_NEGLIGIBLE",
HARM_SEVERITY_UNSUPPORTED = "HARM_SEVERITY_UNSUPPORTED"
}

// @beta
Expand Down Expand Up @@ -464,8 +595,8 @@ export interface ImagenInlineImage {
}

// @beta
export class ImagenModel extends VertexAIModel {
constructor(vertexAI: VertexAI, modelParams: ImagenModelParams, requestOptions?: RequestOptions | undefined);
export class ImagenModel extends AIModel {
constructor(ai: AI, modelParams: ImagenModelParams, requestOptions?: RequestOptions | undefined);
generateImages(prompt: string): Promise<ImagenGenerationResponse<ImagenInlineImage>>;
// @internal
generateImagesGCS(prompt: string, gcsURI: string): Promise<ImagenGenerationResponse<ImagenGCSImage>>;
Expand Down Expand Up @@ -587,7 +718,6 @@ export const POSSIBLE_ROLES: readonly ["user", "model", "function", "system"];
export interface PromptFeedback {
// (undocumented)
blockReason?: BlockReason;
// (undocumented)
blockReasonMessage?: string;
// (undocumented)
safetyRatings: SafetyRating[];
Expand Down Expand Up @@ -627,19 +757,15 @@ export interface SafetyRating {
category: HarmCategory;
// (undocumented)
probability: HarmProbability;
// (undocumented)
probabilityScore: number;
// (undocumented)
severity: HarmSeverity;
// (undocumented)
severityScore: number;
}

// @public
export interface SafetySetting {
// (undocumented)
category: HarmCategory;
// (undocumented)
method?: HarmBlockMethod;
// (undocumented)
threshold: HarmBlockThreshold;
Expand Down Expand Up @@ -791,46 +917,19 @@ export interface UsageMetadata {
}

// @public
export interface VertexAI {
app: FirebaseApp;
// (undocumented)
location: string;
}
export type VertexAI = AI;

// @public
export class VertexAIError extends FirebaseError {
constructor(code: VertexAIErrorCode, message: string, customErrorData?: CustomErrorData | undefined);
// (undocumented)
readonly code: VertexAIErrorCode;
// (undocumented)
readonly customErrorData?: CustomErrorData | undefined;
export class VertexAIBackend extends Backend {
constructor(location?: string);
readonly location: string;
}

// @public
export const enum VertexAIErrorCode {
API_NOT_ENABLED = "api-not-enabled",
ERROR = "error",
FETCH_ERROR = "fetch-error",
INVALID_CONTENT = "invalid-content",
INVALID_SCHEMA = "invalid-schema",
NO_API_KEY = "no-api-key",
NO_APP_ID = "no-app-id",
NO_MODEL = "no-model",
NO_PROJECT_ID = "no-project-id",
PARSE_FAILED = "parse-failed",
REQUEST_ERROR = "request-error",
RESPONSE_ERROR = "response-error"
}
export const VertexAIError: typeof AIError;

// @public
export abstract class VertexAIModel {
// @internal
protected constructor(vertexAI: VertexAI, modelName: string);
// @internal (undocumented)
protected _apiSettings: ApiSettings;
readonly model: string;
static normalizeModelName(modelName: string): string;
}
export const VertexAIModel: typeof AIModel;

// @public
export interface VertexAIOptions {
Expand Down
20 changes: 14 additions & 6 deletions docs-devsite/_toc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -472,8 +472,18 @@ toc:
- title: vertexai
path: /docs/reference/js/vertexai.md
section:
- title: AI
path: /docs/reference/js/vertexai.ai.md
- title: AIError
path: /docs/reference/js/vertexai.aierror.md
- title: AIModel
path: /docs/reference/js/vertexai.aimodel.md
- title: AIOptions
path: /docs/reference/js/vertexai.aioptions.md
- title: ArraySchema
path: /docs/reference/js/vertexai.arrayschema.md
- title: Backend
path: /docs/reference/js/vertexai.backend.md
- title: BaseParams
path: /docs/reference/js/vertexai.baseparams.md
- title: BooleanSchema
Expand Down Expand Up @@ -532,6 +542,8 @@ toc:
path: /docs/reference/js/vertexai.generativecontentblob.md
- title: GenerativeModel
path: /docs/reference/js/vertexai.generativemodel.md
- title: GoogleAIBackend
path: /docs/reference/js/vertexai.googleaibackend.md
- title: GroundingAttribution
path: /docs/reference/js/vertexai.groundingattribution.md
- title: GroundingMetadata
Expand Down Expand Up @@ -598,12 +610,8 @@ toc:
path: /docs/reference/js/vertexai.toolconfig.md
- title: UsageMetadata
path: /docs/reference/js/vertexai.usagemetadata.md
- title: VertexAI
path: /docs/reference/js/vertexai.vertexai.md
- title: VertexAIError
path: /docs/reference/js/vertexai.vertexaierror.md
- title: VertexAIModel
path: /docs/reference/js/vertexai.vertexaimodel.md
Comment on lines -601 to -606
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These types still exist (can't remove them until a breaking change), but our doc generation tool doesn't pick them up since they're just aliases to the new types (e.g. VertexAIError is just an alias to AIError.

- title: VertexAIBackend
path: /docs/reference/js/vertexai.vertexaibackend.md
- title: VertexAIOptions
path: /docs/reference/js/vertexai.vertexaioptions.md
- title: VideoMetadata
Expand Down
2 changes: 1 addition & 1 deletion docs-devsite/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@ https://github.com/firebase/firebase-js-sdk
| [@firebase/performance](./performance.md#performance_package) | The Firebase Performance Monitoring Web SDK. This SDK does not work in a Node.js environment. |
| [@firebase/remote-config](./remote-config.md#remote-config_package) | The Firebase Remote Config Web SDK. This SDK does not work in a Node.js environment. |
| [@firebase/storage](./storage.md#storage_package) | Cloud Storage for Firebase |
| [@firebase/vertexai](./vertexai.md#vertexai_package) | The Vertex AI in Firebase Web SDK. |
| [@firebase/vertexai](./vertexai.md#vertexai_package) | The Firebase AI Web SDK. |

Loading
Loading