Adding specification for the new data stream settings APIs

[masseyke]

This adds specifications for the new data stream settings APIs.

[JoshMock]

This is the only URL in this whole file that links to a docs v2 page. @lcawl @georgewallace Does this need to be changed?

[masseyke]

I had just put in the placeholder we had used on the elasticsearch side. I can change it to something else. Looking now to see what fits best.

[lcawl]

I've added a lot of minor suggestions, but the important one is related to the @doc_tag. It would also be nice to have request and response examples per https://github.com/elastic/elasticsearch-specification/blob/main/docs/add-new-api.md#add-endpoint-request-and-response-examples

[lcawl]

settings: IndexSettings

The example includes effective_settings. If that's valid, shouldn't it appear here too?

[masseyke]

Oh wow yes good catch.

[lcawl]

body: IndexSettings

This currently seems to result in a kind of recursive situation in the OpenAPI output:

  "requestBody": {
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/indices._types.IndexSettings"
              },
...
      "indices._types.IndexSettings": {
        "type": "object",
        "properties": {
          "index": {
            "$ref": "#/components/schemas/indices._types.IndexSettings"
          },
          "mode": {
            "type": "string"
          },
...

We don't have that level of detail in our old docs (e.g. https://www.elastic.co/guide/en/elasticsearch/reference/8.18/indices-update-settings.html or https://www.elastic.co/guide/en/elasticsearch/reference/8.18/index-modules.html#index-modules-settings), so I want to call it out to ensure it's accurate.

[masseyke]

I don't think I understand. I see that I haven't made any changes to output/openapi/elasticsearch-openapi.json. Did I miss a step? And body:IndexSettings is just copied directly from https://github.com/elastic/elasticsearch-specification/blob/main/specification/indices/put_settings/IndicesPutSettingsRequest.ts#L124. Is it generating different OpenAPI output for some reason?

[masseyke]

Oh I just realized that make generate does not run make transform-to-openapi. Looking into this now.

[masseyke]

I still don't think I see the recursion you're talking about, even after running make transform-to-openapi. Could you show me where you're seeing this? Thanks.

[masseyke]

For some reason, make transform-to-openapi does not pull in the new APIs I have added.

[masseyke]

OK, I realized that make transform-to-openapi was not generating anything because the API was behind a feature flag. I removed the feature flag (just merged that into elasticsearch this morning) so now I am generating openapi information. But I don't understand what you mean by recursion, or how it is different from what we've done for IndicesPutSettingsRequest. I am not sure how to generate the docs to take a look.

[lcawl]

Sorry for the confusion, yes I was testing it with feature flag turned off so I could see the output. You're right that this is not different than the index settings APIs. I'm just seeing potentially problematic output in the docs so I'll pursue as a separate issue.

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
indices.add_block	🟢	2/2	2/2
indices.analyze	🟢	246/246	246/246
indices.cancel_migrate_reindex	🟢	2/2	2/2
indices.clear_cache	🟢	4/4	4/4
indices.clone	🟢	5/5	5/5
indices.close	🟢	49/49	49/49
indices.create_data_stream	🟢	31/31	31/31
indices.create_from	🟢	4/4	4/4
indices.create	🔴	1179/1225	1225/1225
indices.data_streams_stats	🟢	4/4	4/4
indices.delete_alias	🔴	12/12	5/12
indices.delete_data_lifecycle	⚪	Missing test	Missing test
indices.delete_data_stream_options	🟠	Missing type	Missing type
indices.delete_data_stream	🟢	28/28	28/28
indices.delete_index_template	🟢	1/1	1/1
indices.delete_template	🟢	9/9	9/9
indices.delete	🟢	105/105	105/105
indices.disk_usage	🟢	6/6	6/6
indices.downsample	🟢	4/4	4/4
indices.exists_alias	🟢	37/37	37/37
indices.exists_index_template	⚪	Missing test	Missing test
indices.exists_template	🟢	15/15	15/15
indices.exists	🟢	39/39	39/39
indices.explain_data_lifecycle	⚪	Missing test	Missing test
indices.field_usage_stats	🟢	5/5	5/5
indices.flush	🟢	41/41	41/41
indices.forcemerge	🟢	12/12	12/12
indices.get_alias	🟢	68/68	68/68
indices.get_data_lifecycle_stats	⚪	Missing test	Missing test
indices.get_data_lifecycle	⚪	Missing test	Missing test
indices.get_data_stream_options	🟠	Missing type	Missing type
indices.get_data_stream_settings	⚪	Missing test	Missing test
indices.get_data_stream	🟢	11/11	11/11
indices.get_field_mapping	🟢	15/15	15/15
indices.get_index_template	🟢	21/21	21/21
indices.get_mapping	🟢	142/142	142/142
indices.get_migrate_reindex_status	🟢	1/1	1/1
indices.get_settings	🔴	76/76	47/76
indices.get_template	🟢	32/32	32/32
indices.get	🟢	55/55	55/55
indices.migrate_reindex	🟢	2/2	2/2
indices.migrate_to_data_stream	⚪	Missing test	Missing test
indices.modify_data_stream	⚪	Missing test	Missing test
indices.open	🟢	20/20	20/20
indices.promote_data_stream	⚪	Missing test	Missing test
indices.put_alias	🔴	58/58	20/58
indices.put_data_lifecycle	⚪	Missing test	Missing test
indices.put_data_stream_options	🟠	Missing type	Missing type
indices.put_data_stream_settings	⚪	Missing test	Missing test
indices.put_index_template	🟢	68/68	68/68
indices.put_mapping	🔴	124/125	128/128
indices.put_settings	🔴	56/58	58/58
indices.put_template	🔴	45/46	46/46
indices.recovery	🟢	9/9	9/9
indices.refresh	🟢	283/283	283/283
indices.reload_search_analyzers	🟢	2/2	2/2
indices.resolve_cluster	🟢	13/13	13/13
indices.resolve_index	🟢	6/6	6/6
indices.rollover	🔴	27/27	2/27
indices.segments	🔴	6/6	5/6
indices.shard_stores	🔴	5/5	4/5
indices.shrink	🟢	5/5	5/5
indices.simulate_index_template	🔴	6/7	5/7
indices.simulate_template	🔴	5/5	4/5
indices.split	🟢	11/11	11/11
indices.stats	🔴	123/123	31/122
indices.update_aliases	🔴	26/26	7/26
indices.validate_query	🟢	12/12	12/12

You can validate these APIs yourself by using the make validate target.