Skip to content

Add function to list all indexes #1646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jbauerrfid opened this issue Jan 15, 2021 · 3 comments · Fixed by #1693
Closed

Add function to list all indexes #1646

jbauerrfid opened this issue Jan 15, 2021 · 3 comments · Fixed by #1693
Labels
in: core Issues in core support type: enhancement A general enhancement

Comments

@jbauerrfid
Copy link

jbauerrfid commented Jan 15, 2021

Please add a function to list all indexes on the Elasticsearch host to ReactiveElasticsearchClient.indices().

This should do the same as running GET /_all on the ES host, and return a Flux of index descriptions.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Jan 15, 2021
@sothawo sothawo added in: core Issues in core support type: enhancement A general enhancement labels Jan 15, 2021
@jbauerrfid
Copy link
Author

jbauerrfid commented Jan 15, 2021

as a work around I build a query on top of .execute to fetch the index list:

esClient.execute(cb -> cb.get()
                .uri("/" + indexName)
                .accept(MediaType.APPLICATION_JSON)
                .exchange())
                .flatMap(clientResponse -> {
                    if (clientResponse.statusCode() == HttpStatus.OK) {
                        return clientResponse.bodyToMono(ObjectNode.class);
                    }
                    return Mono.empty();
                })
                .flatMapIterable(results  -> {
                   // convert results and collect them into a Flux...
                }

it does its job, but it feels a bit weird to do a lot of special logic for a pretty basic task

@sothawo sothawo removed the status: waiting-for-triage An issue we've not yet triaged label Jan 15, 2021
@gpopides
Copy link
Contributor

I tried to look at the code in order to grasp whats going on.

spring-data-elasticsearch uses response classes from elasticsearch library, so there is not a response which can represent the response of "/_all" endpoint.

I tried to map it my self but i have troubles reading the response body from

RawActionResponse

Any hint would be welcome, and then i could open a PR

@sothawo
Copy link
Collaborator

sothawo commented Feb 13, 2021

_all is just a special case of the get index API.

We should have a new function in IndexOperations like getInformation(IndexCoordinates) (and a corresponding function in the reactive part). The parameter could then contain a single index name, multiple names or a name like _all or *. The return value could be a list of some new IndexInformation class that contains the information for one specific index.

To get the result for this call it is necessary to use a GetIndexRequest with the RestHighlevelClient with the corresponding index names. The RestHighlevelClientthen returns a org.elasticsearch.client.indices.GetIndexResponse object which contains (taken from the code):

    private Map<String, MappingMetadata> mappings;
    private Map<String, List<AliasMetadata>> aliases;
    private Map<String, Settings> settings;
    private Map<String, Settings> defaultSettings;
    private Map<String, String> dataStreams;
    private String[] indices;

and the needed getter functions to retrieve the information for all the indices. This information should be used to build the IndexInformation instances.

Building GetIndexRequest instances is already present in code, the mapping from a GetIndexResponse to the new classes not.

And  - as Elasticsearch 7 still does support the TransportClient - this must be added to Rest, Reactive and Transport implementations.

gpopides added a commit to gpopides/spring-data-elasticsearch that referenced this issue Feb 14, 2021
gpopides added a commit to gpopides/spring-data-elasticsearch that referenced this issue Feb 14, 2021
@sothawo sothawo linked a pull request Feb 15, 2021 that will close this issue
1 task
@sothawo sothawo changed the title Feature request: add function to list all indexes Add function to list all indexes Feb 15, 2021
sothawo pushed a commit that referenced this issue Feb 20, 2021
Original Pull Request #1693
Closes #1646
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: core Issues in core support type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants