Skip to content

Commit 8b10611

Browse files
safwanrahmansafwanrahman
authored
Merge pull request #136 from safwanrahman/major_fix
[Fix #115] Support elasticsearch-dsl > 6.1.0
2 parents 9cc6022 + 489200d commit 8b10611

20 files changed

+322
-378
lines changed

‎.travis.yml

-52
Original file line numberDiff line numberDiff line change
@@ -7,68 +7,24 @@ dist: trusty # default "precise" distro doesn't include Java 8 for Elasticsearch
77

88
matrix:
99
include:
10-
- env: TOX_ENV=py36-django-110-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
11-
python: 3.6
12-
- env: TOX_ENV=py36-django-110-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
13-
python: 3.6
1410
- env: TOX_ENV=py36-django-110-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
1511
python: 3.6
16-
- env: TOX_ENV=py37-django-110-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
17-
python: 3.7
18-
sudo: true
19-
dist: xenial
20-
- env: TOX_ENV=py37-django-110-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
21-
python: 3.7
22-
sudo: true
23-
dist: xenial
2412
- env: TOX_ENV=py37-django-110-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
2513
python: 3.7
2614
sudo: true
2715
dist: xenial
28-
- env: TOX_ENV=py27-django-110-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
29-
python: 2.7
30-
- env: TOX_ENV=py27-django-110-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
31-
python: 2.7
3216
- env: TOX_ENV=py27-django-110-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
3317
python: 2.7
34-
- env: TOX_ENV=py36-django-111-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
35-
python: 3.6
36-
- env: TOX_ENV=py36-django-111-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
37-
python: 3.6
3818
- env: TOX_ENV=py36-django-111-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
3919
python: 3.6
40-
- env: TOX_ENV=py37-django-111-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
41-
python: 3.7
42-
sudo: true
43-
dist: xenial
44-
- env: TOX_ENV=py37-django-111-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
45-
python: 3.7
46-
sudo: true
47-
dist: xenial
4820
- env: TOX_ENV=py37-django-111-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
4921
python: 3.7
5022
sudo: true
5123
dist: xenial
52-
- env: TOX_ENV=py27-django-111-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
53-
python: 2.7
54-
- env: TOX_ENV=py27-django-111-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
55-
python: 2.7
5624
- env: TOX_ENV=py27-django-111-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
5725
python: 2.7
58-
- env: TOX_ENV=py36-django-2-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
59-
python: 3.6
60-
- env: TOX_ENV=py36-django-2-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
61-
python: 3.6
6226
- env: TOX_ENV=py36-django-2-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
6327
python: 3.6
64-
- env: TOX_ENV=py37-django-2-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
65-
python: 3.7
66-
sudo: true
67-
dist: xenial
68-
- env: TOX_ENV=py37-django-2-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
69-
python: 3.7
70-
sudo: true
71-
dist: xenial
7228
- env: TOX_ENV=py37-django-2-es6 ES_APT_URL=https://artifacts.elastic.co/packages/6.x/apt
7329
python: 3.7
7430
sudo: true
@@ -77,14 +33,6 @@ matrix:
7733
python: 3.7
7834
dist: xenial
7935
sudo: true
80-
- env: TOX_ENV=py37-django-21-es5 ES_APT_URL=https://artifacts.elastic.co/packages/5.x/apt
81-
python: 3.7
82-
dist: xenial
83-
sudo: true
84-
- env: TOX_ENV=py37-django-21-es2 ES_APT_URL=https://packages.elastic.co/elasticsearch/2.x/debian
85-
python: 3.7
86-
dist: xenial
87-
sudo: true
8836

8937
cache: pip
9038

‎README.rst

+58-65
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Features
2727

2828
- Django >= 1.10
2929
- Python 2.7, 3.5, 3.6, 3.7
30-
- Elasticsearch >= 2.0 < 7.0
30+
- Elasticsearch >= 6.0 < 7.0
3131

3232
.. _Search: http://elasticsearch-dsl.readthedocs.io/en/stable/search_dsl.html
3333

@@ -39,13 +39,8 @@ Install Django Elasticsearch DSL::
3939
pip install django-elasticsearch-dsl
4040

4141
# Elasticsearch 6.x
42-
pip install 'elasticsearch-dsl>=6.0,<6.2'
42+
pip install 'elasticsearch-dsl>=6.3.0,<7.0'
4343

44-
# Elasticsearch 5.x
45-
pip install 'elasticsearch-dsl>=5.0,<6.0'
46-
47-
# Elasticsearch 2.x
48-
pip install 'elasticsearch-dsl>=2.1,<3.0'
4944

5045
Then add ``django_elasticsearch_dsl`` to the INSTALLED_APPS
5146

@@ -81,30 +76,31 @@ Then for a model:
8176
(4, "SUV"),
8277
])
8378
84-
To make this model work with Elasticsearch, create a subclass of ``django_elasticsearch_dsl.DocType``
85-
and create a ``django_elasticsearch_dsl.Index`` to define your Elasticsearch indices, names, and settings. This classes must be
86-
defined in a ``documents.py`` file.
79+
To make this model work with Elasticsearch, create a subclass of ``django_elasticsearch_dsl.Document``,
80+
create a ``class Index`` inside the ``Document`` class
81+
to define your Elasticsearch indices, names, settings etc and at last register the class using
82+
``registry.register_document`` decorator.
8783

8884
.. code-block:: python
8985
9086
# documents.py
9187
92-
from django_elasticsearch_dsl import DocType, Index
88+
from django_elasticsearch_dsl import Document
89+
from django_elasticsearch_dsl.registries import registry
9390
from .models import Car
9491
95-
# Name of the Elasticsearch index
96-
car = Index('cars')
97-
# See Elasticsearch Indices API reference for available settings
98-
car.settings(
99-
number_of_shards=1,
100-
number_of_replicas=0
101-
)
10292
93+
@registry.register_document
94+
class CarDocument(Document):
95+
class Index:
96+
# Name of the Elasticsearch index
97+
name = 'cars'
98+
# See Elasticsearch Indices API reference for available settings
99+
settings = {'number_of_shards': 1,
100+
'number_of_replicas': 0}
103101
104-
@car.doc_type
105-
class CarDocument(DocType):
106-
class Meta:
107-
model = Car # The model associated with this DocType
102+
class Django:
103+
model = Car # The model associated with this Document
108104
109105
# The fields of the model you want to be indexed in Elasticsearch
110106
fields = [
@@ -205,25 +201,25 @@ the model to a string, so we'll just add a method for it:
205201
else:
206202
return "SUV"
207203
208-
Now we need to tell our ``DocType`` subclass to use that method instead of just
204+
Now we need to tell our ``Document`` subclass to use that method instead of just
209205
accessing the ``type`` field on the model directly. Change the CarDocument to look
210206
like this:
211207

212208
.. code-block:: python
213209
214210
# documents.py
215211
216-
from django_elasticsearch_dsl import DocType, fields
212+
from django_elasticsearch_dsl import Document, fields
217213
218214
# ... #
219215
220-
@car.doc_type
221-
class CarDocument(DocType):
216+
@registry.register_document
217+
class CarDocument(Document):
222218
# add a string field to the Elasticsearch mapping called type, the
223219
# value of which is derived from the model's type_to_string attribute
224220
type = fields.TextField(attr="type_to_string")
225221
226-
class Meta:
222+
class Django:
227223
model = Car
228224
# we removed the type field from here
229225
fields = [
@@ -240,7 +236,7 @@ Using prepare_field
240236
~~~~~~~~~~~~~~~~~~~
241237

242238
Sometimes, you need to do some extra prepping before a field should be saved to
243-
Elasticsearch. You can add a ``prepare_foo(self, instance)`` method to a DocType
239+
Elasticsearch. You can add a ``prepare_foo(self, instance)`` method to a Document
244240
(where foo is the name of the field), and that will be called when the field
245241
needs to be saved.
246242

@@ -250,7 +246,7 @@ needs to be saved.
250246
251247
# ... #
252248
253-
class CarDocument(DocType):
249+
class CarDocument(Document):
254250
# ... #
255251
256252
foo = TextField()
@@ -292,18 +288,11 @@ You can use an ObjectField or a NestedField.
292288
293289
# documents.py
294290
295-
from django_elasticsearch_dsl import DocType, Index, fields
291+
from django_elasticsearch_dsl import Document, fields
296292
from .models import Car, Manufacturer, Ad
297293
298-
car = Index('cars')
299-
car.settings(
300-
number_of_shards=1,
301-
number_of_replicas=0
302-
)
303-
304-
305-
@car.doc_type
306-
class CarDocument(DocType):
294+
@registry.register_document
295+
class CarDocument(Document):
307296
manufacturer = fields.ObjectField(properties={
308297
'name': fields.TextField(),
309298
'country_code': fields.TextField(),
@@ -314,7 +303,10 @@ You can use an ObjectField or a NestedField.
314303
'pk': fields.IntegerField(),
315304
})
316305
317-
class Meta:
306+
class Index:
307+
name = 'cars'
308+
309+
class Django:
318310
model = Car
319311
fields = [
320312
'name',
@@ -367,14 +359,14 @@ So for example you can use a custom analyzer_:
367359
char_filter=["html_strip"]
368360
)
369361
370-
@car.doc_type
371-
class CarDocument(DocType):
362+
@registry.register_document
363+
class CarDocument(Document):
372364
description = fields.TextField(
373365
analyzer=html_strip,
374366
fields={'raw': fields.KeywordField()}
375367
)
376368
377-
class Meta:
369+
class Django:
378370
model = Car
379371
fields = [
380372
'name',
@@ -417,20 +409,19 @@ instance.
417409

418410
Index
419411
-----
420-
421-
To define an Elasticsearch index you must instantiate a ``django_elasticsearch_dsl.Index`` class and set the name
422-
and settings of the index. This class inherits from elasticsearch-dsl-py Index_.
423-
After you instantiate your class, you need to associate it with the DocType you
424-
want to put in this Elasticsearch index.
412+
In typical scenario using `class Index` on a `Document` class is sufficient to perform any action.
413+
In a few cases though it can be useful to manipulate an Index object directly.
414+
To define an Elasticsearch index you must instantiate a ``elasticsearch_dsl.Index`` class and set the name
415+
and settings of the index.
416+
After you instantiate your class, you need to associate it with the Document you
417+
want to put in this Elasticsearch index and also add the `registry.register_document` decorator.
425418

426419

427-
.. _Index: http://elasticsearch-dsl.readthedocs.io/en/stable/persistence.html#index
428-
429420
.. code-block:: python
430421
431422
# documents.py
432-
433-
from django_elasticsearch_dsl import DocType, Index
423+
from elasticsearch_dsl import Index
424+
from django_elasticsearch_dsl import Document
434425
from .models import Car, Manufacturer
435426
436427
# The name of your index
@@ -441,33 +432,35 @@ want to put in this Elasticsearch index.
441432
number_of_replicas=0
442433
)
443434
444-
445-
@car.doc_type
446-
class CarDocument(DocType):
447-
class Meta:
435+
@registry.register_document
436+
@car.document
437+
class CarDocument(Document):
438+
class Django:
448439
model = Car
449440
fields = [
450441
'name',
451442
'color',
452443
]
453444
454-
@car.doc_type
455-
class ManufacturerDocument(DocType):
456-
class Meta:
445+
@registry.register_document
446+
class ManufacturerDocument(Document):
447+
class Index:
448+
name = 'manufacture'
449+
settings = {'number_of_shards': 1,
450+
'number_of_replicas': 0}
451+
452+
class Django:
457453
model = Car
458454
fields = [
459-
'name', # If a field as the same name in multiple DocType of
460-
# the same Index, the field type must be identical
461-
# (here fields.TextField)
455+
'name',
462456
'country_code',
463457
]
464458
465459
When you execute the command::
466460

467461
$ ./manage.py search_index --rebuild
468462

469-
This will create an index named ``cars`` in Elasticsearch with two mappings:
470-
``manufacturer_document`` and ``car_document``.
463+
This will create two index named ``cars`` and ``manufacture`` in Elasticsearch with appropriate mapping.
471464

472465

473466
Management Commands
@@ -565,6 +558,6 @@ TODO
565558
- Add support for --using (use another Elasticsearch cluster) in management commands.
566559
- Add management commands for mapping level operations (like update_mapping....).
567560
- Dedicated documentation.
568-
- Generate ObjectField/NestField properties from a DocType class.
561+
- Generate ObjectField/NestField properties from a Document class.
569562
- More examples.
570563
- Better ``ESTestCase`` and documentation for testing

0 commit comments

Comments
 (0)