Pre-flight product check

docs/client-concepts/connection-pooling/exceptions/unexpected-exceptions.asciidoc

@@ -105,7 +105,7 @@ An unexpected hard exception on ping and sniff is something we *do* try to recov
 
 Here, pinging nodes on first use is enabled and the node on port 9200 throws an exception on ping; when this happens,
 we still fallover to retry the ping on node on port 9201, where it succeeds.
-Following this, the client call on 9201 throws a hard exception that we are not able to recover from
+Following this, the client call on 9201 throws a hard exception that we are not able to recover from.
 
 [source,csharp]
 ----

docs/client-concepts/connection-pooling/failover/falling-over.asciidoc

@@ -16,7 +16,7 @@ please modify the original csharp file found at the link and submit the PR with
 === Fail over
 
 When using a connection pool with more than one node, a request will be retried if
-the call to a node throws an exception or returns a 502, 503 or 504 response
+the call to a node throws an exception or returns a 502, 503 or 504 response.
 
 [source,csharp]
 ----
@@ -39,7 +39,7 @@ audit = await audit.TraceCall(
 [[bad-gateway]]
 ==== 502 Bad Gateway
 
-Will be treated as an error that requires retrying
+Will be treated as an error that requires retrying.
 
 [source,csharp]
 ----
@@ -62,7 +62,7 @@ audit = await audit.TraceCall(
 [[service-unavailable]]
 ==== 503 Service Unavailable
 
-Will be treated as an error that requires retrying
+Will be treated as an error that requires retrying.
 
 [source,csharp]
 ----
@@ -85,7 +85,7 @@ audit = await audit.TraceCall(
 [[gateway-timeout]]
 ==== 504 Gateway Timeout
 
-Will be treated as an error that requires retrying
+Will be treated as an error that requires retrying.
 
 [source,csharp]
 ----
@@ -108,7 +108,7 @@ audit = await audit.TraceCall(
 If a call returns a __valid__ HTTP status code other than 502 or 503, the request won't be retried.
 
 IMPORTANT: Different requests may have different status codes that are deemed __valid__. For example,
-a *404 Not Found* response is a __valid__ status code for an index exists request
+a *404 Not Found* response is a __valid__ status code for an index exists request.
 
 [source,csharp]
 ----

docs/client-concepts/connection-pooling/product-checking/product-check-at-startup.asciidoc

@@ -0,0 +1,114 @@
+:ref_current: https://www.elastic.co/guide/en/elasticsearch/reference/7.x
+
+:github: https://github.com/elastic/elasticsearch-net
+
+:nuget: https://www.nuget.org/packages
+
+////
+IMPORTANT NOTE
+==============
+This file has been generated from https://github.com/elastic/elasticsearch-net/tree/7.x/src/Tests/Tests/ClientConcepts/ConnectionPooling/ProductChecking/ProductCheckAtStartup.doc.cs. 
+If you wish to submit a PR for any spelling mistakes, typos or grammatical errors for this file,
+please modify the original csharp file found at the link and submit the PR with that change. Thanks!
+////
+
+[[product-check-at-startup]]
+== Product check at startup
+
+Since v7.14.0, the client performs a required product check before the first call.
+This pre-flight product check allows the client to establish the version of Elasticsearch that it is communicating with.
+
+The product check requires one additional HTTP request to be sent to the server as part of the request pipeline before
+the main API call is sent. In most cases, this will succeed during the very first API call that the client sends.
+Once the product check succeeds, no further product check HTTP requests are sent for subsequent API calls.
+
+[source,csharp]
+----
+var audit = new Auditor(() => VirtualClusterWith
+    .Nodes(1)
+    .ClientCalls(r => r.SucceedAlways())
+    .StaticConnectionPool()
+    .Settings(s => s.DisablePing())
+);
+
+audit = await audit.TraceCalls(skipProductCheck: false,
+    new ClientCall() {
+        { ProductCheckOnStartup },
+        { ProductCheckSuccess, 9200 }, <1>
+        { HealthyResponse, 9200 } <2>
+    },
+    new ClientCall() {
+        { HealthyResponse, 9200 } <3>
+    }
+);
+----
+<1> as this is the first call, the product check is executed
+<2> following the product check, the actual request is sent
+<3> subsequent calls no longer perform product check
+
+Here's an example with a single node cluster which fails for some reason during the first product check attempt. 
+
+[source,csharp]
+----
+var audit = new Auditor(() => VirtualClusterWith
+    .Nodes(1, productCheckAlwaysSucceeds: false)
+    .ProductCheck(r => r.Fails(TimesHelper.Once))
+    .ProductCheck(r => r.SucceedAlways())
+    .ClientCalls(r => r.SucceedAlways())
+    .StaticConnectionPool()
+    .Settings(s => s.DisablePing())
+);
+
+audit = await audit.TraceCalls(skipProductCheck: false,
+    new ClientCall() {
+        { ProductCheckOnStartup },
+        { ProductCheckFailure, 9200 }, <1>
+        { HealthyResponse, 9200 } <2>
+    },
+    new ClientCall() {
+        { ProductCheckOnStartup },
+        { ProductCheckSuccess, 9200 }, <3>
+        { HealthyResponse, 9200 }
+    },
+    new ClientCall() {
+        { HealthyResponse, 9200 } <4>
+    }
+);
+----
+<1> as this is the first call, the product check is executed, but fails
+<2> the actual request is still sent and succeeds
+<3> as the previous product check failed, it runs again on the second call
+<4> subsequent calls no longer perform product check
+
+Here's an example with a three node cluster which fails for some reason during the first and second product check attempts. 
+
+[source,csharp]
+----
+var audit = new Auditor(() => VirtualClusterWith
+    .Nodes(3, productCheckAlwaysSucceeds: false)
+    .ProductCheck(r => r.FailAlways())
+    .ProductCheck(r => r.OnPort(9202).SucceedAlways())
+    .ClientCalls(r => r.SucceedAlways())
+    .StaticConnectionPool()
+    .Settings(s => s.DisablePing())
+);
+
+audit = await audit.TraceCalls(skipProductCheck: false,
+    new ClientCall() {
+        { ProductCheckOnStartup },
+        { ProductCheckFailure, 9200 }, <1>
+        { ProductCheckFailure, 9201 }, <2>
+        { ProductCheckSuccess, 9202 }, <3>
+        { HealthyResponse, 9200 } <4>
+    },
+    new ClientCall() {
+        { HealthyResponse, 9201 } <5>
+    }
+);
+----
+<1> this is the first call, the product check is executed, but fails on this node
+<2> the next node is also tried and fails
+<3> the third node is tried, successfully responds and the product check succeeds
+<4> the actual request is sent and succeeds
+<5> subsequent calls no longer perform product check
+

docs/client-concepts/connection-pooling/request-overrides/disable-sniff-ping-per-request.asciidoc

@@ -36,7 +36,7 @@ var audit = new Auditor(() => VirtualClusterWith
 ----
 <1> sniff on startup
 
-Now We disable sniffing on the request so even though it's our first call,
+Now we disable sniffing on the request so even though it's our first call,
 we do not want to sniff on startup.
 
 Instead, the sniff on startup is deferred to the second call into the cluster that

docs/client-concepts/connection-pooling/request-overrides/request-timeouts-overrides.asciidoc

@@ -15,9 +15,9 @@ please modify the original csharp file found at the link and submit the PR with
 [[request-timeout]]
 === Request timeouts
 
-While you can specify Request time out globally you can override this per request too
+While you can specify Request time out globally, you can override this per request too.
 
-we set up a 10 node cluster with a global time out of 20 seconds.
+We set up a 10 node cluster with a global time out of 20 seconds.
 Each call on a node takes 10 seconds. So we can only try this call on 2 nodes
 before the max request time out kills the client call.
 
@@ -38,7 +38,7 @@ audit = await audit.TraceCalls(
         { MaxTimeoutReached }
     },
     /**
-    * On the second request we specify a request timeout override to 80 seconds
+    * On the second request we specify a request timeout override to 80 seconds.
     * We should now see more nodes being tried.
     */
     new ClientCall(r => r.RequestTimeout(TimeSpan.FromSeconds(80)))

docs/client-concepts/troubleshooting/audit-trail.asciidoc

@@ -20,7 +20,7 @@ occur when a request is made. This audit trail is available on the response as d
 following example.
 
 We'll use a Sniffing connection pool here since it sniffs on startup and pings before
-first usage, so we can get an audit trail with a few events out
+first usage, so we can get an audit trail with a few events out.
 
 [source,csharp]
 ----
@@ -33,7 +33,7 @@ var connectionSettings = new ConnectionSettings(pool)
 var client = new ElasticClient(connectionSettings);
 ----
 
-After issuing the following request
+After issuing the following request:
 
 [source,csharp]
 ----
@@ -48,10 +48,12 @@ readable fashion, similar to
 ....
 Valid NEST response built from a successful low level call on POST: /project/doc/_search
 # Audit trail of this API call:
- - [1] SniffOnStartup: Took: 00:00:00.0360264
- - [2] SniffSuccess: Node: http://localhost:9200/ Took: 00:00:00.0310228
- - [3] PingSuccess: Node: http://127.0.0.1:9200/ Took: 00:00:00.0115074
- - [4] HealthyResponse: Node: http://127.0.0.1:9200/ Took: 00:00:00.1477640
+ - [1] ProductCheckOnStartup: Took: 00:00:00.1014934
+ - [2] ProductCheckSuccess: Node: http://ipv4.fiddler:9200/ Took: 00:00:00.1002862
+ - [3] SniffOnStartup: Took: 00:00:00.0440780
+ - [4] SniffSuccess: Node: http://localhost:9200/ Took: 00:00:00.0257506
+ - [5] PingSuccess: Node: http://127.0.0.1:9200/ Took: 00:00:00.0115074
+ - [6] HealthyResponse: Node: http://127.0.0.1:9200/ Took: 00:00:00.1477640
 # Request:
 <Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
 # Response:
@@ -69,15 +71,17 @@ But can also be accessed manually:
 
 [source,csharp]
 ----
-response.ApiCall.AuditTrail.Count.Should().Be(4, "{0}", debug);
-response.ApiCall.AuditTrail[0].Event.Should().Be(AuditEvent.SniffOnStartup, "{0}", debug);
-response.ApiCall.AuditTrail[1].Event.Should().Be(AuditEvent.SniffSuccess, "{0}", debug);
-response.ApiCall.AuditTrail[2].Event.Should().Be(AuditEvent.PingSuccess, "{0}", debug);
-response.ApiCall.AuditTrail[3].Event.Should().Be(AuditEvent.HealthyResponse, "{0}", debug);
+response.ApiCall.AuditTrail.Count.Should().Be(6, "{0}", debug);
+response.ApiCall.AuditTrail[0].Event.Should().Be(AuditEvent.ProductCheckOnStartup, "{0}", debug);
+response.ApiCall.AuditTrail[1].Event.Should().Be(AuditEvent.ProductCheckSuccess, "{0}", debug);
+response.ApiCall.AuditTrail[2].Event.Should().Be(AuditEvent.SniffOnStartup, "{0}", debug);
+response.ApiCall.AuditTrail[3].Event.Should().Be(AuditEvent.SniffSuccess, "{0}", debug);
+response.ApiCall.AuditTrail[4].Event.Should().Be(AuditEvent.PingSuccess, "{0}", debug);
+response.ApiCall.AuditTrail[5].Event.Should().Be(AuditEvent.HealthyResponse, "{0}", debug);
 ----
 
 Each audit has a started and ended `DateTime` on it that will provide
-some understanding of how long it took
+some understanding of how long it took.
 
 [source,csharp]
 ----

docs/client-concepts/troubleshooting/logging-with-on-request-completed.asciidoc

@@ -51,7 +51,7 @@ var client = FixedResponseClient.Create( <1>
     500,
     connectionSettings => connectionSettings
         .ThrowExceptions() <2>
-        .OnRequestCompleted(r => counter++)
+        .OnRequestCompleted(r => counter++), productCheckSucceeds: false
 );
 
 Assert.Throws<ElasticsearchClientException>(() => client.RootNodeInfo()); <3>

docs/code-standards/requests.asciidoc

@@ -34,14 +34,14 @@ var searchResponse = client.Search<Project>(s => s.AllIndices());
 searchResponse.ApiCall.Uri.ToString().Should().Be("http://localhost:9200/_all/_search?typed_keys=true");
 
 var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200/elasticsearch"));
-var settings = new ConnectionSettings(pool, new InMemoryConnection());
+var settings = new ConnectionSettings(pool, new InMemoryConnection("elasticsearch"));
 var client = new ElasticClient(settings);
 var searchResponse = client.Search<Project>(s => s.AllIndices());
 
 searchResponse.ApiCall.Uri.ToString().Should().Be("http://localhost:9200/elasticsearch/_all/_search?typed_keys=true");
 
 var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200/elasticsearch/"));
-var settings = new ConnectionSettings(pool, new InMemoryConnection());
+var settings = new ConnectionSettings(pool, new InMemoryConnection("elasticsearch/"));
 var client = new ElasticClient(settings);
 var searchResponse = client.Search<Project>(s => s.AllIndices());
 

docs/conventions.asciidoc

@@ -35,6 +35,8 @@ Conventions that apply to both Elasticsearch.Net and NEST
 
 * <<transports-and-pipelines, Transports and pipelines>>
 
+* <<product-check-at-startup, Pre-flight product check at startup>>
+
 * <<sniffing-behaviour, Sniffing behaviour>>
 
 * <<pinging-behaviour, Pinging behaviour>>
@@ -97,6 +99,8 @@ include::{building-blocks}/date-time-providers.asciidoc[]
 
 include::{building-blocks}/keeping-track-of-nodes.asciidoc[]
 
+include::client-concepts/connection-pooling/product-checking/product-check-at-startup.asciidoc[]
+
 [[sniffing-behaviour]]
 == Sniffing behaviour