Skip to content

Firestore: Add ability to configure long polling timeout #7176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
May 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
0d7bff1
Firestore: add experimentalLongPollingTimeout setting (b/266868871, h…
dconeybe Mar 31, 2023
03d1dba
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 12, 2023
6ad3e0f
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 13, 2023
fd30d9e
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 13, 2023
bf5d409
experimentalLongPollingTimeout -> experimentalLongPollingOptions.idle…
dconeybe Apr 13, 2023
a7574d4
ExperimentalLongPollingOptions interface, part 1 [no ci]
dconeybe Apr 14, 2023
6ce1d01
ExperimentalLongPollingOptions interface, part 2 [no ci]
dconeybe Apr 14, 2023
57635d4
ExperimentalLongPollingOptions interface, part 3 [no ci]
dconeybe Apr 14, 2023
8e892ca
ExperimentalLongPollingOptions interface, part 4 (done)
dconeybe Apr 14, 2023
d8966ea
yarn docgen devsite
dconeybe Apr 14, 2023
9296b48
internal_helpers.ts: remove unused import: ExperimentalLongPollingOpt…
dconeybe Apr 14, 2023
9e51222
[no ci]
dconeybe Apr 14, 2023
d4ffeec
add unit tests for longPollingOptionsEqual() and cloneLongPollingOpti…
dconeybe Apr 14, 2023
376dde7
long_polling_options.ts: doc cleanup and minor code cleanup
dconeybe Apr 14, 2023
7cc521f
doc improvements
dconeybe Apr 14, 2023
23685ba
lite-api/settings.ts: move validation logic to a helper method, valid…
dconeybe Apr 14, 2023
061d11b
database.test.ts: ?. -> .
dconeybe Apr 14, 2023
1bebeda
yarn docgen devsite
dconeybe Apr 14, 2023
c3a0c77
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 14, 2023
7d5e283
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 20, 2023
b61f5d9
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 24, 2023
fea12e3
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 25, 2023
82ab9de
api council feedback - part 1
dconeybe Apr 25, 2023
d1d703f
api council feedback - part 2
dconeybe Apr 26, 2023
be90ef0
api council feedback - part 3
dconeybe Apr 26, 2023
4431c8e
yarn docgen devsite
dconeybe Apr 26, 2023
00c08e6
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 26, 2023
4dd243c
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 27, 2023
dc64cab
Replace internal link with b/266868871
dconeybe Apr 27, 2023
e5d37c9
Merge remote-tracking branch 'origin/master' into LongPollingTimeout
dconeybe Apr 28, 2023
3bd125d
yarn changeset
dconeybe Apr 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/witty-wasps-play.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@firebase/firestore': minor
'firebase': minor
---

Added the ability to configure the long-polling hanging get request timeout using the new `idleHttpRequestTimeoutSeconds` setting
6 changes: 6 additions & 0 deletions common/api-review/firestore.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,11 @@ export function endBefore(snapshot: DocumentSnapshot<unknown>): QueryEndAtConstr
// @public
export function endBefore(...fieldValues: unknown[]): QueryEndAtConstraint;

// @public
export interface ExperimentalLongPollingOptions {
timeoutSeconds?: number;
}

// @public
export class FieldPath {
constructor(...fieldNames: string[]);
Expand Down Expand Up @@ -227,6 +232,7 @@ export interface FirestoreSettings {
cacheSizeBytes?: number;
experimentalAutoDetectLongPolling?: boolean;
experimentalForceLongPolling?: boolean;
experimentalLongPollingOptions?: ExperimentalLongPollingOptions;
host?: string;
ignoreUndefinedProperties?: boolean;
localCache?: FirestoreLocalCache;
Expand Down
43 changes: 43 additions & 0 deletions docs-devsite/firestore_.experimentallongpollingoptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Project: /docs/reference/js/_project.yaml
Book: /docs/reference/_book.yaml
page_type: reference

{% comment %}
DO NOT EDIT THIS FILE!
This is generated by the JS SDK team, and any local changes will be
overwritten. Changes should be made in the source code at
https://github.com/firebase/firebase-js-sdk
{% endcomment %}

# ExperimentalLongPollingOptions interface
Options that configure the SDK’s underlying network transport (WebChannel) when long-polling is used.

Note: This interface is "experimental" and is subject to change.

See `FirestoreSettings.experimentalAutoDetectLongPolling`<!-- -->, `FirestoreSettings.experimentalForceLongPolling`<!-- -->, and `FirestoreSettings.experimentalLongPollingOptions`<!-- -->.

<b>Signature:</b>

```typescript
export declare interface ExperimentalLongPollingOptions
```

## Properties

| Property | Type | Description |
| --- | --- | --- |
| [timeoutSeconds](./firestore_.experimentallongpollingoptions.md#experimentallongpollingoptionstimeoutseconds) | number | The desired maximum timeout interval, in seconds, to complete a long-polling GET response. Valid values are between 5 and 30, inclusive. Floating point values are allowed and will be rounded to the nearest millisecond.<!-- -->By default, when long-polling is used the "hanging GET" request sent by the client times out after 30 seconds. To request a different timeout from the server, set this setting with the desired timeout.<!-- -->Changing the default timeout may be useful, for example, if the buffering proxy that necessitated enabling long-polling in the first place has a shorter timeout for hanging GET requests, in which case setting the long-polling timeout to a shorter value, such as 25 seconds, may fix prematurely-closed hanging GET requests. For example, see https://github.com/firebase/firebase-js-sdk/issues/6987. |

## ExperimentalLongPollingOptions.timeoutSeconds

The desired maximum timeout interval, in seconds, to complete a long-polling GET response. Valid values are between 5 and 30, inclusive. Floating point values are allowed and will be rounded to the nearest millisecond.

By default, when long-polling is used the "hanging GET" request sent by the client times out after 30 seconds. To request a different timeout from the server, set this setting with the desired timeout.

Changing the default timeout may be useful, for example, if the buffering proxy that necessitated enabling long-polling in the first place has a shorter timeout for hanging GET requests, in which case setting the long-polling timeout to a shorter value, such as 25 seconds, may fix prematurely-closed hanging GET requests. For example, see https://github.com/firebase/firebase-js-sdk/issues/6987.

<b>Signature:</b>

```typescript
timeoutSeconds?: number;
```
13 changes: 13 additions & 0 deletions docs-devsite/firestore_.firestoresettings.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ export declare interface FirestoreSettings
| [cacheSizeBytes](./firestore_.firestoresettings.md#firestoresettingscachesizebytes) | number | NOTE: This field will be deprecated in a future major release. Use <code>cache</code> field instead to specify cache size, and other cache configurations.<!-- -->An approximate cache size threshold for the on-disk data. If the cache grows beyond this size, Firestore will start removing data that hasn't been recently used. The size is not a guarantee that the cache will stay below that size, only that if the cache exceeds the given size, cleanup will be attempted.<!-- -->The default value is 40 MB. The threshold must be set to at least 1 MB, and can be set to <code>CACHE_SIZE_UNLIMITED</code> to disable garbage collection. |
| [experimentalAutoDetectLongPolling](./firestore_.firestoresettings.md#firestoresettingsexperimentalautodetectlongpolling) | boolean | Configures the SDK's underlying transport (WebChannel) to automatically detect if long-polling should be used. This is very similar to <code>experimentalForceLongPolling</code>, but only uses long-polling if required.<!-- -->This setting will likely be enabled by default in future releases and cannot be combined with <code>experimentalForceLongPolling</code>. |
| [experimentalForceLongPolling](./firestore_.firestoresettings.md#firestoresettingsexperimentalforcelongpolling) | boolean | Forces the SDK’s underlying network transport (WebChannel) to use long-polling. Each response from the backend will be closed immediately after the backend sends data (by default responses are kept open in case the backend has more data to send). This avoids incompatibility issues with certain proxies, antivirus software, etc. that incorrectly buffer traffic indefinitely. Use of this option will cause some performance degradation though.<!-- -->This setting cannot be used with <code>experimentalAutoDetectLongPolling</code> and may be removed in a future release. If you find yourself using it to work around a specific network reliability issue, please tell us about it in https://github.com/firebase/firebase-js-sdk/issues/1674. |
| [experimentalLongPollingOptions](./firestore_.firestoresettings.md#firestoresettingsexperimentallongpollingoptions) | [ExperimentalLongPollingOptions](./firestore_.experimentallongpollingoptions.md#experimentallongpollingoptions_interface) | Options that configure the SDK’s underlying network transport (WebChannel) when long-polling is used.<!-- -->These options are only used if <code>experimentalForceLongPolling</code> is true or if <code>experimentalAutoDetectLongPolling</code> is true and the auto-detection determined that long-polling was needed. Otherwise, these options have no effect. |
| [host](./firestore_.firestoresettings.md#firestoresettingshost) | string | The hostname to connect to. |
| [ignoreUndefinedProperties](./firestore_.firestoresettings.md#firestoresettingsignoreundefinedproperties) | boolean | Whether to skip nested properties that are set to <code>undefined</code> during object serialization. If set to <code>true</code>, these properties are skipped and not written to Firestore. If set to <code>false</code> or omitted, the SDK throws an exception when it encounters properties of type <code>undefined</code>. |
| [localCache](./firestore_.firestoresettings.md#firestoresettingslocalcache) | [FirestoreLocalCache](./firestore_.md#firestorelocalcache) | Specifies the cache used by the SDK. Available options are <code>MemoryLocalCache</code> and <code>IndexedDbLocalCache</code>, each with different configuration options.<!-- -->When unspecified, <code>MemoryLocalCache</code> will be used by default.<!-- -->NOTE: setting this field and <code>cacheSizeBytes</code> at the same time will throw exception during SDK initialization. Instead, using the configuration in the <code>FirestoreLocalCache</code> object to specify the cache size. |
Expand Down Expand Up @@ -68,6 +69,18 @@ This setting cannot be used with `experimentalAutoDetectLongPolling` and may be
experimentalForceLongPolling?: boolean;
```

## FirestoreSettings.experimentalLongPollingOptions

Options that configure the SDK’s underlying network transport (WebChannel) when long-polling is used.

These options are only used if `experimentalForceLongPolling` is true or if `experimentalAutoDetectLongPolling` is true and the auto-detection determined that long-polling was needed. Otherwise, these options have no effect.

<b>Signature:</b>

```typescript
experimentalLongPollingOptions?: ExperimentalLongPollingOptions;
```

## FirestoreSettings.host

The hostname to connect to.
Expand Down
1 change: 1 addition & 0 deletions docs-devsite/firestore_.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ https://github.com/firebase/firebase-js-sdk
| [AggregateSpec](./firestore_.aggregatespec.md#aggregatespec_interface) | Specifies a set of aggregations and their aliases. |
| [DocumentChange](./firestore_.documentchange.md#documentchange_interface) | A <code>DocumentChange</code> represents a change to the documents matching a query. It contains the document affected and the type of change that occurred. |
| [DocumentData](./firestore_.documentdata.md#documentdata_interface) | Document data (for use with [setDoc()](./firestore_lite.md#setdoc)<!-- -->) consists of fields mapped to values. |
| [ExperimentalLongPollingOptions](./firestore_.experimentallongpollingoptions.md#experimentallongpollingoptions_interface) | Options that configure the SDK’s underlying network transport (WebChannel) when long-polling is used.<!-- -->Note: This interface is "experimental" and is subject to change.<!-- -->See <code>FirestoreSettings.experimentalAutoDetectLongPolling</code>, <code>FirestoreSettings.experimentalForceLongPolling</code>, and <code>FirestoreSettings.experimentalLongPollingOptions</code>. |
| [FirestoreDataConverter](./firestore_.firestoredataconverter.md#firestoredataconverter_interface) | Converter used by <code>withConverter()</code> to transform user objects of type <code>T</code> into Firestore data.<!-- -->Using the converter allows you to specify generic type arguments when storing and retrieving objects from Firestore. |
| [FirestoreSettings](./firestore_.firestoresettings.md#firestoresettings_interface) | Specifies custom configurations for your Cloud Firestore instance. You must set these before invoking any other methods. |
| [Index](./firestore_.index.md#index_interface) | <b><i>(BETA)</i></b> The SDK definition of a Firestore index. |
Expand Down
1 change: 1 addition & 0 deletions packages/firestore/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ export {
} from './api/bundle';

export { FirestoreSettings, PersistenceSettings } from './api/settings';
export { ExperimentalLongPollingOptions } from './api/long_polling_options';

export {
DocumentChange,
Expand Down
73 changes: 73 additions & 0 deletions packages/firestore/src/api/long_polling_options.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/**
* @license
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/**
* Options that configure the SDK’s underlying network transport (WebChannel)
* when long-polling is used.
*
* Note: This interface is "experimental" and is subject to change.
*
* See `FirestoreSettings.experimentalAutoDetectLongPolling`,
* `FirestoreSettings.experimentalForceLongPolling`, and
* `FirestoreSettings.experimentalLongPollingOptions`.
*/
export interface ExperimentalLongPollingOptions {
/**
* The desired maximum timeout interval, in seconds, to complete a
* long-polling GET response. Valid values are between 5 and 30, inclusive.
* Floating point values are allowed and will be rounded to the nearest
* millisecond.
*
* By default, when long-polling is used the "hanging GET" request sent by
* the client times out after 30 seconds. To request a different timeout
* from the server, set this setting with the desired timeout.
*
* Changing the default timeout may be useful, for example, if the buffering
* proxy that necessitated enabling long-polling in the first place has a
* shorter timeout for hanging GET requests, in which case setting the
* long-polling timeout to a shorter value, such as 25 seconds, may fix
* prematurely-closed hanging GET requests.
* For example, see https://github.com/firebase/firebase-js-sdk/issues/6987.
*/
timeoutSeconds?: number;
}

/**
* Compares two `ExperimentalLongPollingOptions` objects for equality.
*/
export function longPollingOptionsEqual(
options1: ExperimentalLongPollingOptions,
options2: ExperimentalLongPollingOptions
): boolean {
return options1.timeoutSeconds === options2.timeoutSeconds;
}

/**
* Creates and returns a new `ExperimentalLongPollingOptions` with the same
* option values as the given instance.
*/
export function cloneLongPollingOptions(
options: ExperimentalLongPollingOptions
): ExperimentalLongPollingOptions {
const clone: ExperimentalLongPollingOptions = {};

if (options.timeoutSeconds !== undefined) {
clone.timeoutSeconds = options.timeoutSeconds;
}

return clone;
}
12 changes: 12 additions & 0 deletions packages/firestore/src/api/settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import { FirestoreSettings as LiteSettings } from '../lite-api/settings';

import { FirestoreLocalCache } from './cache_config';
import { ExperimentalLongPollingOptions } from './long_polling_options';

export { DEFAULT_HOST } from '../lite-api/settings';

Expand Down Expand Up @@ -92,4 +93,15 @@ export interface FirestoreSettings extends LiteSettings {
* cannot be combined with `experimentalForceLongPolling`.
*/
experimentalAutoDetectLongPolling?: boolean;

/**
* Options that configure the SDK’s underlying network transport (WebChannel)
* when long-polling is used.
*
* These options are only used if `experimentalForceLongPolling` is true or if
* `experimentalAutoDetectLongPolling` is true and the auto-detection
* determined that long-polling was needed. Otherwise, these options have no
* effect.
*/
experimentalLongPollingOptions?: ExperimentalLongPollingOptions;
}
3 changes: 3 additions & 0 deletions packages/firestore/src/core/database_info.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { FirebaseApp } from '@firebase/app';

import { ExperimentalLongPollingOptions } from '../api/long_polling_options';
import { Code, FirestoreError } from '../util/error';

/**
Expand Down Expand Up @@ -34,6 +35,7 @@ export class DatabaseInfo {
* when using WebChannel as the network transport.
* @param autoDetectLongPolling - Whether to use the detectBufferingProxy
* option when using WebChannel as the network transport.
* @param longPollingOptions Options that configure long-polling.
* @param useFetchStreams Whether to use the Fetch API instead of
* XMLHTTPRequest
*/
Expand All @@ -45,6 +47,7 @@ export class DatabaseInfo {
readonly ssl: boolean,
readonly forceLongPolling: boolean,
readonly autoDetectLongPolling: boolean,
readonly longPollingOptions: ExperimentalLongPollingOptions,
readonly useFetchStreams: boolean
) {}
}
Expand Down
2 changes: 2 additions & 0 deletions packages/firestore/src/lite-api/components.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import { _FirebaseService } from '@firebase/app';

import { CredentialsProvider } from '../api/credentials';
import { cloneLongPollingOptions } from '../api/long_polling_options';
import { User } from '../auth/user';
import { DatabaseId, DatabaseInfo } from '../core/database_info';
import { newConnection } from '../platform/connection';
Expand Down Expand Up @@ -117,6 +118,7 @@ export function makeDatabaseInfo(
settings.ssl,
settings.experimentalForceLongPolling,
settings.experimentalAutoDetectLongPolling,
cloneLongPollingOptions(settings.experimentalLongPollingOptions),
settings.useFetchStreams
);
}
57 changes: 57 additions & 0 deletions packages/firestore/src/lite-api/settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@

import { FirestoreLocalCache } from '../api/cache_config';
import { CredentialsSettings } from '../api/credentials';
import {
ExperimentalLongPollingOptions,
cloneLongPollingOptions,
longPollingOptionsEqual
} from '../api/long_polling_options';
import {
LRU_COLLECTION_DISABLED,
LRU_DEFAULT_CACHE_SIZE_BYTES
Expand All @@ -29,6 +34,18 @@ import { validateIsNotUsedTogether } from '../util/input_validation';
export const DEFAULT_HOST = 'firestore.googleapis.com';
export const DEFAULT_SSL = true;

// The minimum long-polling timeout is hardcoded on the server. The value here
// should be kept in sync with the value used by the server, as the server will
// silently ignore a value below the minimum and fall back to the default.
// Googlers see b/266868871 for relevant discussion.
const MIN_LONG_POLLING_TIMEOUT_SECONDS = 5;

// No maximum long-polling timeout is configured in the server, and defaults to
// 30 seconds, which is what Watch appears to use.
// Googlers see b/266868871 for relevant discussion.
const MAX_LONG_POLLING_TIMEOUT_SECONDS = 30;

// Whether long-polling auto-detected is enabled by default.
const DEFAULT_AUTO_DETECT_LONG_POLLING = false;

/**
Expand Down Expand Up @@ -58,6 +75,7 @@ export interface PrivateSettings extends FirestoreSettings {
cacheSizeBytes?: number;
experimentalForceLongPolling?: boolean;
experimentalAutoDetectLongPolling?: boolean;
experimentalLongPollingOptions?: ExperimentalLongPollingOptions;
useFetchStreams?: boolean;

localCache?: FirestoreLocalCache;
Expand All @@ -81,6 +99,8 @@ export class FirestoreSettingsImpl {

readonly experimentalAutoDetectLongPolling: boolean;

readonly experimentalLongPollingOptions: ExperimentalLongPollingOptions;

readonly ignoreUndefinedProperties: boolean;

readonly useFetchStreams: boolean;
Expand Down Expand Up @@ -146,6 +166,11 @@ export class FirestoreSettingsImpl {
!!settings.experimentalAutoDetectLongPolling;
}

this.experimentalLongPollingOptions = cloneLongPollingOptions(
settings.experimentalLongPollingOptions ?? {}
);
validateLongPollingOptions(this.experimentalLongPollingOptions);

this.useFetchStreams = !!settings.useFetchStreams;
}

Expand All @@ -159,8 +184,40 @@ export class FirestoreSettingsImpl {
other.experimentalForceLongPolling &&
this.experimentalAutoDetectLongPolling ===
other.experimentalAutoDetectLongPolling &&
longPollingOptionsEqual(
this.experimentalLongPollingOptions,
other.experimentalLongPollingOptions
) &&
this.ignoreUndefinedProperties === other.ignoreUndefinedProperties &&
this.useFetchStreams === other.useFetchStreams
);
}
}

function validateLongPollingOptions(
options: ExperimentalLongPollingOptions
): void {
if (options.timeoutSeconds !== undefined) {
if (isNaN(options.timeoutSeconds)) {
throw new FirestoreError(
Code.INVALID_ARGUMENT,
`invalid long polling timeout: ` +
`${options.timeoutSeconds} (must not be NaN)`
);
}
if (options.timeoutSeconds < MIN_LONG_POLLING_TIMEOUT_SECONDS) {
throw new FirestoreError(
Code.INVALID_ARGUMENT,
`invalid long polling timeout: ${options.timeoutSeconds} ` +
`(minimum allowed value is ${MIN_LONG_POLLING_TIMEOUT_SECONDS})`
);
}
if (options.timeoutSeconds > MAX_LONG_POLLING_TIMEOUT_SECONDS) {
throw new FirestoreError(
Code.INVALID_ARGUMENT,
`invalid long polling timeout: ${options.timeoutSeconds} ` +
`(maximum allowed value is ${MAX_LONG_POLLING_TIMEOUT_SECONDS})`
);
}
}
}
Loading