From ee6cf2c08c19fc5e8b9517242a2e95b48f7160d5 Mon Sep 17 00:00:00 2001 From: Steve Gordon Date: Wed, 23 Nov 2022 14:52:30 +0000 Subject: [PATCH] Add 8.0 release notes (#6998) * Initial release notes for 8.0 GA * Add release notes link * Cleanup release notes * Additional release notes work * Finalise release notes * Fix title * Remove DIagnosticSource docs --- .../release-notes-8.0.0.asciidoc | 512 +++++++++++++++++- docs/release-notes/release-notes.asciidoc | 2 + docs/troubleshooting.asciidoc | 12 - 3 files changed, 511 insertions(+), 15 deletions(-) diff --git a/docs/release-notes/release-notes-8.0.0.asciidoc b/docs/release-notes/release-notes-8.0.0.asciidoc index f69d8b57d94..31b4149f077 100644 --- a/docs/release-notes/release-notes-8.0.0.asciidoc +++ b/docs/release-notes/release-notes-8.0.0.asciidoc @@ -1,5 +1,511 @@ -[float] [[release-notes-8.0.0]] -== Release-Notes 8.0.0 +== Release notes v8.0.0 -Coming soon! +[TIP] +-- +Due to the extensive changes in the new {net-client}, we highly recommend +reviewing this documentation in full, before proceeding with migration to v8. +-- + +After many months of work, eleven alphas, six betas and two release candidates, +we are pleased to announce the GA release of the {net-client} v8.0.0. + +The overall themes of this release have been based around redesigning the client +for the future, standardizing serialization, performance improvements, codebase +simplification, and code-generation. + +The following release notes explain the current state of the client, missing +features, breaking changes and our rationale for some of the design choices we have introduced. + +[discrete] +=== Version 8 is a refresh + +[IMPORTANT] +-- +It is important to highlight that v8 of the {net-client} represents +a new start for the client design. It is important to review how this may affect +your code and usage. +-- + +Mature code becomes increasingly hard to maintain over time, and +our ability to make timely releases has diminished as code complexity has increased. +Major releases allow us to simplify and better align our language clients with +each other in terms of design. Here, it is crucial to find the right balance +between uniformity across programming languages and the idiomatic concerns of +each language. For .NET, we will typically compare and contrast with https://github.com/elastic/elasticsearch-java[Java] and https://github.com/elastic/go-elasticsearch[Go] +to make sure that our approach is equivalent for each of these. We also take +heavy inspiration from Microsoft framework design guidelines and the conventions +of the wider .NET community. + +[discrete] +==== New Elastic.Clients.Elasticsearch NuGet package + +We have shipped the new code-generated client as a +https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[new NuGet package] +with a new root namespace, `Elastic.Clients.Elasticsearch`. +The new v8 client is built upon the foundations of the v7 `NEST` client, but there +are changes. By shipping as a new package, the expectation is that migration can +be managed with a phased approach. + +While this is a new package, we have aligned the major version (v8.x.x) with the +supported {es} server version to clearly indicate the client/server compatibility. +The v8 client is designed to work with version 8 of {es}. + +The v7 `NEST` client continues to be supported but will not gain new features or +support for new {es} endpoints. It should be considered deprecated in favour of +the new client. + +[discrete] +==== Limited feature set + +[CAUTION] +-- +The 8.0.0 {net-client} does not have feature parity with the previous v7 `NEST` +high-level client. +-- + +For the initial 8.0.0 release we have limited the features we are shipping. +Over the course of the 8.x releases, we will reintroduce features. Therefore, +if something is missing, it may not be permanently omitted. We will endeavour to communicate our plans as and when they become available. + +If a feature you depend on is missing (and not explicitly documented below as a +feature that we do not plan to reintroduce), please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] +or comment on a relevant existing issue to highlight your need to us. This will +help us prioritise our roadmap. + +[discrete] +=== Code generation + +Given the size of the Elasticsearch API surface today, it is no longer practical +to maintain thousands of types (requests, responses, queries, aggregations, etc.) +by hand. To ensure consistent, accurate and timely alignment between language +clients and {es}, the 8.x clients, and many of the associated types are now +automatically code-generated from a https://github.com/elastic/elasticsearch-specification[shared specification]. This is a common solution to maintaining alignment between +client and server among SDKs and libraries, such as those for Azure, AWS and the +Google Cloud Platform. + +code-generation from a specification has inevitably led to some differences +between the existing v7 `NEST` types and those available in the new v7 {net-client}. +For the 8.0.0 release, we generate strictly from the specification, special +casing a few areas to improve usability or to align with language idioms. + +The base type hierarchy for concepts such as `Properties`, `Aggregations` and +`Queries` is no longer present in generated code, as these arbitrary groupings do +not align with concrete concepts of the public server API. These considerations +do not preclude adding syntactic sugar and usability enhancements to types in future +releases on a case-by-case basis. + +[discrete] +=== Elastic.Transport + +The .NET client includes a transport layer responsible for abstracting HTTP +concepts and to provide functionality such as our request pipeline. This +supports round-robin load-balancing of requests to nodes, pinging failed +nodes and sniffing the cluster for node roles. + +In v7, this layer shipped as `Elasticsearch.Net` and was considered our low-level +client which could be used to send and receive raw JSON bytes between the client +and server. + +As part of the work for 8.0.0, we have moved the transport layer out into +a https://www.nuget.org/packages/Elastic.Transport[new dedicated package] and +https://github.com/elastic/elastic-transport-net[repository], named +`Elastic.Transport`. This supports reuse across future clients and allows +consumers with extremely high-performance requirements to build upon this foundation. + +[discrete] +=== System.Text.Json for serialization + +The v7 `NEST` high-level client used an internalized and modified version of +https://github.com/neuecc/Utf8Json[Utf8Json] for request and response +serialization. This was introduced for its performance improvements +over https://www.newtonsoft.com/json[Json.NET], the more common JSON framework at +the time. + +While Utf8Json provides good value, we have identified minor bugs and +performance issues that have required maintenance over time. Some of these +are hard to change without more significant effort. This library is no longer +maintained, and any such changes cannot easily be contributed back to the +original project. + +With .NET Core 3.0, Microsoft shipped new https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis[System.Text.Json APIs] +that are included in-the-box with current versions of .NET. We have adopted +`System.Text.Json` for all serialization. Consumers can still define and register +their own `Serializer` implementation for their document types should they prefer +to use a different serialization library. + +By adopting `System.Text.Json`, we now depend on a well-maintained and supported +library from Microsoft. `System.Text.Json` is designed from the ground up to support +the latest performance optimizations in .NET and, as a result, provides both fast and low-allocation serialization. + +[discrete] +=== Mockability of ElasticsearchClient + +Testing code is an important part of software development. We recommend +that consumers prefer introducing an abstraction for their use of the {net-client} +as the prefered way to decouple consuming code from client types and support unit +testing. + +In order to support user testing scenarios, we have unsealed the `ElasticsearchClient` +type and made its methods virtual. This supports mocking the type directly for unit +testing. This is an improvement over the original `IElasticClient` interface from +`NEST` (v7) which only supported mocking of top-level client methods. + +We have also introduced a `TestableResponseFactory` in `Elastic.Transport` to +make it easier to create response instances with specific status codes and validity +that can be used during unit testing. + +These changes are in addition to our existing support for testing with an +`InMemoryConnection`, virtualized clusters and with our +https://github.com/elastic/elasticsearch-net-abstractions/blob/master/src/Elastic.Elasticsearch.Managed[`Elastic.Elasticsearch.Managed`] library for integration +testing against real {es} instances. We will introduce more documentation on testing methodologies in the near future. + +[discrete] +=== Migrating to Elastic.Clients.Elasticsearch + +[WARNING] +-- +The 8.0.0 release does not currently have full-feature parity with `NEST`. The +client primary use case is for application developers communitating with {es}. +-- + +The 8.0.0 release focuses on core endpoints, more specifically for common CRUD +scenarios. We intend to reduce the feature gap in subsequent versions. We anticipate +that this initial release will best suit new applications and may not yet be migration-ready for all existing consumers. We recommend reviewing this documentation carefully to learn about the missing features and reduced API surface details before migrating from the v7 `NEST` client. + +The choice to code-generate a new evolution of the {net-client} introduces some +significant breaking changes. We consciously took the opportunity to refactor +and reconsider historic design choices as part of this major release, intending +to limit future breaking changes. + +The v8 client is shipped as a new https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[NuGet package] +which can be installed alongside v7 `NEST`. We +anticipate that some consumers may prefer a phased migration with both +packages side-by-side for a short period of time to manage complex migrations. In addition, `NEST` 7.17.x can continue to be used in +https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[compatibility mode] +with {es} 8.x servers until the v8 {net-client} features +align with application requirements. + +We will continue to prioritize the feature roadmap and code-generation work +based on https://github.com/elastic/elasticsearch-net/issues[feedback] from consumers who may rely on features that are initially unavailable. + +[discrete] +=== Breaking Changes + +[WARNING] +-- +As a result of code-generating a majority of the client types, this version of +the client includes multiple breaking changes. +-- + +We have strived to keep the core foundation reasonably similar, but types emitted +through code-generation are subject to change between `NEST` (v7) and the new +`Elastic.Clients.Elasticsearch` (v8) package. + +[discrete] +==== Namespaces + +We have renamed the package and top-level namespace for the v8 client to +`Elastic.Clients.Elasticsearch`. All types belong to this namespace. When +necessary, to avoid potential conflicts, types are generated into suitable +sub-namespaces based on the https://github.com/elastic/elasticsearch-specification[{es} specification]. Additional `using` directives may be required to access such types +when using the {net-client}. + +Transport layer concepts have moved to the new `Elastic.Transport` NuGet package +and related types are defined under its namespace. Some configuration and low-level transport functionality may require a `using` directive for the `Elastic.Transport` +namespace. + +[discrete] +==== Type names + +Type names may have changed from previous versions. We are not listing these +explicitly due to the potentially vast number of subtle differences. +Type names will now more closely align to those used in the JSON and as documented +in the {es} documentation. + +[discrete] +==== Class members + +Types may include renamed properties based on the {es} specification, +which differ from the original `NEST` property names. The types used for properties +may also have changed due to code-generation. If you identify missing or +incorrectly-typed properties, please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] to alert us. + +[discrete] +==== Sealing classes + +Opinions on "sealing by default" within the .NET ecosystem tend to be quite +polarized. Microsoft seal all internal types for potential performance gains +and we see a benefit in starting with that approach for the {net-client}, +even for our public API surface. + +While it prevents inheritance and, therefore, may inhibit a few consumer scenarios, +sealing by default is intended to avoid the unexpected or invalid +extension of types that could inadvertently be broken in the future. That said, +sealing is not necessarily a final choice for all types; but it is clearly +easier for a future release to unseal a previously-sealed class than +vice versa. We can therefore choose to unseal as valid scenarios arise, +should we determine that doing so is the best solution for those scenarios, such +as with mockability of the `ElasticsearchClient`. This goes back to our clean-slate concept for this new client. + +[discrete] +==== Removed features + +As part of the clean-slate redesign of the new client, we have opted to remove +certain features from the v8.0 client. These are listed below: + +[discrete] +===== Attribute mappings + +In previous versions of the `NEST` client, attributes could be used to configure +the mapping behaviour and inference for user types. We have removed support for +these attributes and recommend that mapping be completed via the fluent API when +configuring client instances. `System.Text.Json` attributes may be used to rename +and ignore properties during source serialization. + +[discrete] +===== CAT APIs + +The https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html[CAT APIs] +of {es} are intended for human-readable usage and will no longer be supported +via the v8 {net-client}. + +[discrete] +===== Interface removal + +We have removed several interfaces that previously shipped as part of `NEST`. This +is a design decision to simplify the library and avoid interfaces where only a +single implementation of that interface is expected to exist, such as +`IElasticClient` in `NEST`. We have also switched to prefer abstract base classes +over interfaces across the library, as this allows us to add enhancements more +easily without introducing breaking changes for derived types. + +[discrete] +==== Missing features + +While not an exhaustive list, the following are some of the main features which +have not been re-implemented for this initial 8.0.0 GA release. +These remain on our roadmap and will be reviewed and prioritized for inclusion in +future releases. + +* Query DSL operators for combining queries. +* Scroll Helper. +* Fluent API for union types. +* `AutoMap` for field datatype inference. +* Visitor pattern support for types such as `Properties`. +* Support for `JoinField` which affects `ChildrenAggregation`. +* Conditionless queries. +* DiagnosticSources have been removed in `Elastic.Transport` to provide a clean-slate +for an improved diagnostics story. The {net-client} emits https://opentelemetry.io/[OpenTelemetry] compatible `Activity` spans which can be consumed by APM agents such as the https://www.elastic.co/guide/en/apm/agent/dotnet/current/index.html[Elastic APM Agent for .NET]. +* Documentation is a work in progress, and we will expand on the documented scenarios +in future releases. + +[discrete] +=== Reduced API surface + +In this first release of the code-generated .NET client, we have specifically +focused on supporting commonly used endpoints. We have also skipped specific +queries and aggregations which need further work to generate code correctly. +Before migrating, please refer to the lists below for the endpoints, +queries and aggregations currently generated and available +in the 8.0.0 GA release to ensure that the features you are using are currently +supported. + +[discrete] +==== Supported {es} endpoints + +The following are {es} endpoints currently generated and available +in the 8.0.0 {net-client}. + +* AsyncSearch.Delete +* AsyncSearch.Get +* AsyncSearch.Status +* AsyncSearch.Submit +* Bulk +* ClearScroll +* ClosePointInTime +* Cluster.Health +* Count +* Create +* Delete +* DeleteByQuery +* DeleteByQueryRethrottle +* DeleteScript +* EQL.Delete +* EQL.Get +* EQL.Search +* EQL.Status +* Exists +* ExistsSource +* Explain +* FieldCaps +* Get +* GetScript +* GetScriptContext +* GetScriptLanguages +* GetSource +* Index +* Indices.Clone +* Indices.Close +* Indices.Create +* Indices.CreateDataStream +* Indices.Delete +* Indices.DeleteAlias +* Indices.DeleteDataStream +* Indices.DeleteIndexTemplate +* Indices.DeleteTemplate +* Indices.Exists +* Indices.ExistsIndexTemplate +* Indices.ExistsTemplate +* Indices.Flush +* Indices.ForceMerge +* Indices.Get +* Indices.GetAlias +* Indices.GetDataStream +* Indices.GetFieldMapping +* Indices.GetIndexTemplate +* Indices.GetMapping +* Indices.GetTemplate +* Indices.Indices.SimulateTemplate +* Indices.MigrateToDataStream +* Indices.Open +* Indices.PutAlias +* Indices.PutIndexTemplate +* Indices.PutMapping +* Indices.PutTemplate +* Indices.Refresh +* Indices.Rollover +* Indices.Shrink +* Indices.SimulateIndexTemplate +* Indices.Split +* Indices.Unfreeze +* Info +* MGet +* MSearch +* MSearchTemplate +* OpenPointInTime +* Ping +* PutScript +* RankEval +* Reindex +* ReindexRethrottle +* Scroll +* Search +* SearchShards +* SQL.ClearCursor +* SQL.DeleteAsync +* SQL.GetAsync +* SQL.GetAsyncStatus +* SQL.Query +* TermsEnum +* Update +* UpdateByQuery +* UpdateByQueryRethrottle + +[discrete] +==== Supported queries + +The following are query types currently generated and available +in the 8.0.0 {net-client}. + +* Bool +* Boosting +* CombinedFields +* ConstantScore +* DisMax +* Exists +* FunctionScore +* Fuzzy +* HasChild +* HasParent +* Ids +* Intervals +* Match +* MatchAll +* MatchBool +* MatchNone +* MatchPhrase +* MatchPhrasePrefix +* MoreLikeThis +* MultiMatch +* Nested +* ParentId +* Percolate +* Prefix +* QueryString +* RankFeature +* Regexp +* Script +* ScriptScore +* Shape +* SimpleQueryString +* SpanContaining +* SpanFirst +* SpanMultiTerm +* SpanNear +* SpanNot +* SpanOr +* SpanTerm +* SpanWithin +* Term +* Terms +* TermsSet +* Wildcard +* Wrapper + +[discrete] +==== Supported aggregations + +The following are aggregation types currently generated and available +in the 8.0.0 {net-client}. + +* AdjacencyMatrix +* AutoDateHistogram +* Avg +* Boxplot +* Cardinality +* Children +* Composite +* CumulativeCardinality +* DateHistogram +* DateRange +* Derivative +* ExtendedStats +* Filters +* Global +* Histogram +* Inference +* IpRange +* MatrixStats +* Max +* MedianAbsoluteDeviation +* Min +* Missing +* MultiTerms +* Nested +* Parent +* PercentilesBucket +* Range +* Rate +* ReverseNested +* Sampler +* ScriptedMetric +* Stats +* StatsBucket +* StringStats +* Sum +* Terms +* TopHits +* TopMetrics +* TTest +* ValueCount +* VariableWidthHistogram +* WeightedAvg + +[discrete] +=== In closing + +Please give the new `Elastic.Clients.Elasticsearch` client a try in your .NET +applications. If you run into any problems, please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] to raise those +with us. Please let us know how you get on and if you have any questions, +reach out on the https://discuss.elastic.co[Discuss forums]. \ No newline at end of file diff --git a/docs/release-notes/release-notes.asciidoc b/docs/release-notes/release-notes.asciidoc index dcad8ac40c3..80402ce60b6 100644 --- a/docs/release-notes/release-notes.asciidoc +++ b/docs/release-notes/release-notes.asciidoc @@ -2,5 +2,7 @@ = Release notes * <> +* <> include::breaking-change-policy.asciidoc[] +include::release-notes-8.0.0.asciidoc[] diff --git a/docs/troubleshooting.asciidoc b/docs/troubleshooting.asciidoc index e92271c8fbe..ae27725923f 100644 --- a/docs/troubleshooting.asciidoc +++ b/docs/troubleshooting.asciidoc @@ -32,22 +32,10 @@ There are a couple of popular ways of capturing this information * <> -More detailed diagnostic information can be captured with - -* <> - -In addition to logging requests and responses, Elasticsearch 5.0+ also -<> in the response, -to notify you if you are using a feature that is marked as deprecated. - include::client-concepts/troubleshooting/logging-with-on-request-completed.asciidoc[] -include::client-concepts/troubleshooting/diagnostic-source.asciidoc[] - include::client-concepts/troubleshooting/logging-with-fiddler.asciidoc[] -include::client-concepts/troubleshooting/deprecation-logging.asciidoc[] - [[debugging]] == Debugging