Skip to content

Commit 2c59c05

Browse files
github-actions[bot]flobernd
github-actions[bot]
and
flobernd
authored
Add basic NEST -> v8 migration guide (#7893) (#7900)
Co-authored-by: Florian Bernd <[email protected]>
1 parent 6ddbc19 commit 2c59c05

File tree

2 files changed

+306
-0
lines changed

2 files changed

+306
-0
lines changed

‎docs/index.asciidoc

+2
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ include::client-concepts/client-concepts.asciidoc[]
2222

2323
include::usage/index.asciidoc[]
2424

25+
include::migration-guide.asciidoc[]
26+
2527
include::troubleshooting.asciidoc[]
2628

2729
include::redirects.asciidoc[]

‎docs/migration-guide.asciidoc

+304
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
[[migration-guide]]
2+
== Migration guide: From NEST v7 to .NET Client v8
3+
4+
The following migration guide explains the current state of the client, missing
5+
features, breaking changes and our rationale for some of the design choices we have introduced.
6+
7+
[discrete]
8+
=== Version 8 is a refresh
9+
10+
[IMPORTANT]
11+
--
12+
It is important to highlight that v8 of the {net-client} represents
13+
a new start for the client design. It is important to review how this may affect
14+
your code and usage.
15+
--
16+
17+
Mature code becomes increasingly hard to maintain over time.
18+
Major releases allow us to simplify and better align our language clients with
19+
each other in terms of design. It is crucial to find the right balance
20+
between uniformity across programming languages and the idiomatic concerns of
21+
each language. For .NET, we typically compare and contrast with https://github.com/elastic/elasticsearch-java[Java] and https://github.com/elastic/go-elasticsearch[Go]
22+
to make sure that our approach is equivalent for each of these. We also take
23+
heavy inspiration from Microsoft framework design guidelines and the conventions
24+
of the wider .NET community.
25+
26+
[discrete]
27+
==== New Elastic.Clients.Elasticsearch NuGet package
28+
29+
We have shipped the new code-generated client as a
30+
https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[NuGet package]
31+
with a new root namespace, `Elastic.Clients.Elasticsearch`.
32+
The v8 client is built upon the foundations of the v7 `NEST` client, but there
33+
are changes. By shipping as a new package, the expectation is that migration can
34+
be managed with a phased approach.
35+
36+
While this is a new package, we have aligned the major version (v8.x.x) with the
37+
supported {es} server version to clearly indicate the client/server compatibility.
38+
The v8 client is designed to work with version 8 of {es}.
39+
40+
The v7 `NEST` client continues to be supported but will not gain new features or
41+
support for new {es} endpoints. It should be considered deprecated in favour of
42+
the new client.
43+
44+
[discrete]
45+
==== Limited feature set
46+
47+
[CAUTION]
48+
--
49+
The version 8 {net-client} does not have feature parity with the previous v7 `NEST`
50+
high-level client.
51+
--
52+
53+
If a feature you depend on is missing (and not explicitly documented below as a
54+
feature that we do not plan to reintroduce), open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue]
55+
or comment on a relevant existing issue to highlight your need to us. This will
56+
help us prioritise our roadmap.
57+
58+
[discrete]
59+
=== Code generation
60+
61+
Given the size of the {es} API surface today, it is no longer practical
62+
to maintain thousands of types (requests, responses, queries, aggregations, etc.)
63+
by hand. To ensure consistent, accurate, and timely alignment between language
64+
clients and {es}, the 8.x clients, and many of the associated types are now
65+
automatically code-generated from a https://github.com/elastic/elasticsearch-specification[shared specification]. This is a common solution to maintaining alignment between
66+
client and server among SDKs and libraries, such as those for Azure, AWS and the
67+
Google Cloud Platform.
68+
69+
Code-generation from a specification has inevitably led to some differences
70+
between the existing v7 `NEST` types and those available in the new v7 {net-client}.
71+
For version 8, we generate strictly from the specification, special
72+
casing a few areas to improve usability or to align with language idioms.
73+
74+
The base type hierarchy for concepts such as `Properties`, `Aggregations` and
75+
`Queries` is no longer present in generated code, as these arbitrary groupings do
76+
not align with concrete concepts of the public server API. These considerations
77+
do not preclude adding syntactic sugar and usability enhancements to types in future
78+
releases on a case-by-case basis.
79+
80+
[discrete]
81+
=== Elastic.Transport
82+
83+
The .NET client includes a transport layer responsible for abstracting HTTP
84+
concepts and to provide functionality such as our request pipeline. This
85+
supports round-robin load-balancing of requests to nodes, pinging failed
86+
nodes and sniffing the cluster for node roles.
87+
88+
In v7, this layer shipped as `Elasticsearch.Net` and was considered our low-level
89+
client which could be used to send and receive raw JSON bytes between the client
90+
and server.
91+
92+
As part of the work for 8.0.0, we have moved the transport layer out into
93+
a https://www.nuget.org/packages/Elastic.Transport[new dedicated package] and
94+
https://github.com/elastic/elastic-transport-net[repository], named
95+
`Elastic.Transport`. This supports reuse across future clients and allows
96+
consumers with extremely high-performance requirements to build upon this foundation.
97+
98+
[discrete]
99+
=== System.Text.Json for serialization
100+
101+
The v7 `NEST` high-level client used an internalized and modified version of
102+
https://github.com/neuecc/Utf8Json[Utf8Json] for request and response
103+
serialization. This was introduced for its performance improvements
104+
over https://www.newtonsoft.com/json[Json.NET], the more common JSON framework at
105+
the time.
106+
107+
While Utf8Json provides good value, we have identified minor bugs and
108+
performance issues that have required maintenance over time. Some of these
109+
are hard to change without more significant effort. This library is no longer
110+
maintained, and any such changes cannot easily be contributed back to the
111+
original project.
112+
113+
With .NET Core 3.0, Microsoft shipped new https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis[System.Text.Json APIs]
114+
that are included in-the-box with current versions of .NET. We have adopted
115+
`System.Text.Json` for all serialization. Consumers can still define and register
116+
their own `Serializer` implementation for their document types should they prefer
117+
to use a different serialization library.
118+
119+
By adopting `System.Text.Json`, we now depend on a well-maintained and supported
120+
library from Microsoft. `System.Text.Json` is designed from the ground up to support
121+
the latest performance optimizations in .NET and, as a result, provides both fast and low-allocation serialization.
122+
123+
[discrete]
124+
=== Mockability of ElasticsearchClient
125+
126+
Testing code is an important part of software development. We recommend
127+
that consumers prefer introducing an abstraction for their use of the {net-client}
128+
as the prefered way to decouple consuming code from client types and support unit
129+
testing.
130+
131+
To support user testing scenarios, we have unsealed the `ElasticsearchClient`
132+
type and made its methods virtual. This supports mocking the type directly for unit
133+
testing. This is an improvement over the original `IElasticClient` interface from
134+
`NEST` (v7) which only supported mocking of top-level client methods.
135+
136+
We have also introduced a `TestableResponseFactory` in `Elastic.Transport` to
137+
make it easier to create response instances with specific status codes and validity
138+
that can be used during unit testing.
139+
140+
These changes are in addition to our existing support for testing with an
141+
`InMemoryConnection`, virtualized clusters and with our
142+
https://github.com/elastic/elasticsearch-net-abstractions/blob/master/src/Elastic.Elasticsearch.Managed[`Elastic.Elasticsearch.Managed`] library for integration
143+
testing against real {es} instances.
144+
145+
[discrete]
146+
=== Migrating to Elastic.Clients.Elasticsearch
147+
148+
[WARNING]
149+
--
150+
The version 8 client does not currently have full-feature parity with `NEST`. The
151+
client primary use case is for application developers communicating with {es}.
152+
--
153+
154+
The version 8 client focuses on core endpoints, more specifically for common CRUD
155+
scenarios. The intention is to reduce the feature gap in subsequent versions. Review this documentation carefully to learn about the missing features and reduced API surface details before migrating from the v7 `NEST` client!
156+
157+
The choice to code-generate a new evolution of the {net-client} introduces some
158+
significant breaking changes.
159+
160+
The v8 client is shipped as a new https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[NuGet package]
161+
which can be installed alongside v7 `NEST`. Some consumers may prefer a phased migration with both
162+
packages side-by-side for a short period of time to manage complex migrations. In addition, `NEST` 7.17.x can continue to be used in
163+
https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[compatibility mode]
164+
with {es} 8.x servers until the v8 {net-client} features
165+
align with application requirements.
166+
167+
[discrete]
168+
=== Breaking Changes
169+
170+
[WARNING]
171+
--
172+
As a result of code-generating a majority of the client types, version 8 of
173+
the client includes multiple breaking changes.
174+
--
175+
176+
We have strived to keep the core foundation reasonably similar, but types emitted
177+
through code-generation are subject to change between `NEST` (v7) and the new
178+
`Elastic.Clients.Elasticsearch` (v8) package.
179+
180+
[discrete]
181+
==== Namespaces
182+
183+
The package and top-level namespace for the v8 client have been renamed to
184+
`Elastic.Clients.Elasticsearch`. All types belong to this namespace. When
185+
necessary, to avoid potential conflicts, types are generated into suitable
186+
sub-namespaces based on the https://github.com/elastic/elasticsearch-specification[{es} specification]. Additional `using` directives may be required to access such types
187+
when using the {net-client}.
188+
189+
Transport layer concepts have moved to the new `Elastic.Transport` NuGet package
190+
and related types are defined under its namespace. Some configuration and low-level transport functionality may require a `using` directive for the `Elastic.Transport`
191+
namespace.
192+
193+
[discrete]
194+
==== Type names
195+
196+
Type names may have changed from previous versions. These are not listed explicitly due to the potentially vast number of subtle differences.
197+
Type names will now more closely align to those used in the JSON and as documented
198+
in the {es} documentation.
199+
200+
[discrete]
201+
==== Class members
202+
203+
Types may include renamed properties based on the {es} specification,
204+
which differ from the original `NEST` property names. The types used for properties
205+
may also have changed due to code-generation. If you identify missing or
206+
incorrectly-typed properties, please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] to alert us.
207+
208+
[discrete]
209+
==== Sealing classes
210+
211+
Opinions on "sealing by default" within the .NET ecosystem tend to be quite
212+
polarized. Microsoft seal all internal types for potential performance gains
213+
and we see a benefit in starting with that approach for the {net-client},
214+
even for our public API surface.
215+
216+
While it prevents inheritance and, therefore, may inhibit a few consumer scenarios,
217+
sealing by default is intended to avoid the unexpected or invalid
218+
extension of types that could inadvertently be broken in the future.
219+
220+
[discrete]
221+
==== Removed features
222+
223+
As part of the clean-slate redesign of the new client,
224+
certain features are removed from the v8.0 client. These are listed below:
225+
226+
[discrete]
227+
===== Attribute mappings
228+
229+
In previous versions of the `NEST` client, attributes could be used to configure
230+
the mapping behaviour and inference for user types. It is recommended that
231+
mapping be completed via the fluent API when configuring client instances.
232+
`System.Text.Json` attributes may be used to rename
233+
and ignore properties during source serialization.
234+
235+
[discrete]
236+
===== CAT APIs
237+
238+
The https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html[CAT APIs]
239+
of {es} are intended for human-readable usage and will no longer be supported
240+
via the v8 {net-client}.
241+
242+
[discrete]
243+
===== Interface removal
244+
245+
Several interfaces are removed to simplify the library and avoid interfaces where only a
246+
single implementation of that interface is expected to exist, such as
247+
`IElasticClient` in `NEST`. Abstract base classes are preferred
248+
over interfaces across the library, as this makes it easier to add enhancements
249+
without introducing breaking changes for derived types.
250+
251+
[discrete]
252+
==== Missing features
253+
254+
The following are some of the main features which
255+
have not been re-implemented for the v8 client.
256+
These might be reviewed and prioritized for inclusion in
257+
future releases.
258+
259+
* Query DSL operators for combining queries.
260+
* Scroll Helper.
261+
* Fluent API for union types.
262+
* `AutoMap` for field datatype inference.
263+
* Visitor pattern support for types such as `Properties`.
264+
* Support for `JoinField` which affects `ChildrenAggregation`.
265+
* Conditionless queries.
266+
* DiagnosticSources have been removed in `Elastic.Transport` to provide a clean-slate
267+
for an improved diagnostics story. The {net-client} emits https://opentelemetry.io/[OpenTelemetry] compatible `Activity` spans which can be consumed by APM agents such as the https://www.elastic.co/guide/en/apm/agent/dotnet/current/index.html[Elastic APM Agent for .NET].
268+
* Documentation is a work in progress, and we will expand on the documented scenarios
269+
in future releases.
270+
271+
[discrete]
272+
=== Reduced API surface
273+
274+
In the current versions of the code-generated .NET client, supporting commonly used
275+
endpoints is critical. Some specific queries and aggregations need further work to generate code correctly,
276+
hence they are not included yet.
277+
Ensure that the features you are using are currently supported before migrating.
278+
279+
An up to date list of all supported and unsupported endpoints can be found on https://github.com/elastic/elasticsearch-net/issues/7890[GitHub].
280+
281+
[discrete]
282+
=== Workarounds for missing features
283+
284+
If you encounter a missing feature with the v8 client, there are several ways to temporarily work around this issue until we officially reintroduce the feature.
285+
286+
`NEST` 7.17.x can continue to be used in
287+
https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[compatibility mode]
288+
with {es} 8.x servers until the v8 {net-client} features
289+
align with application requirements.
290+
291+
As a last resort, the low-level client `Elastic.Transport` can be used to create any desired request by hand:
292+
293+
[source,csharp]
294+
----
295+
var body = """
296+
{
297+
"name": "my-api-key",
298+
"expiration": "1d",
299+
"...": "..."
300+
}
301+
""";
302+
303+
var response = await client.Transport.RequestAsync<StringResponse>(HttpMethod.POST, "/_security/api_key", PostData.String(body));
304+
----

0 commit comments

Comments
 (0)