diff --git a/052_Mapping_Analysis/25_Data_type_differences.asciidoc b/052_Mapping_Analysis/25_Data_type_differences.asciidoc index 7f3c69444..af061869b 100644 --- a/052_Mapping_Analysis/25_Data_type_differences.asciidoc +++ b/052_Mapping_Analysis/25_Data_type_differences.asciidoc @@ -1,10 +1,7 @@ [[mapping-analysis]] -== Mapping and Analysis +== 映射和分析 -While playing around with the data in our index, we notice something odd. -Something seems to be broken: we have 12 tweets in our indices, and only one -of them contains the date `2014-09-15`, but have a look at the `total` hits -for the following queries: +当摆弄索引里面的数据时，我们发现一些奇怪的事情。一些事情看起来被打乱了：在我们的索引中有12条推文，其中只有一条包含日期 `2014-09-15` ，但是看一看下面查询命中的 `总数` （total）： [source,js] -------------------------------------------------- @@ -15,16 +12,9 @@ GET /_search?q=date:2014 # 0 results ! -------------------------------------------------- // SENSE: 052_Mapping_Analysis/25_Data_type_differences.json -Why does querying the <> for the full date -return all tweets, and querying the `date` field for just the year return no -results? Why do our results differ when searching within the `_all` field or -the `date` field? +为什么在 <> 字段查询日期返回所有推文，而在 `date` 字段只查询年份却没有返回结果？为什么我们在 `_all` 字段和 `date` 字段的查询结果有差别？ -Presumably, it is because the way our data has been indexed in the `_all` -field is different from how it has been indexed in the `date` field. -So let's take a look at how Elasticsearch has interpreted our document -structure, by requesting((("mapping (types)"))) the _mapping_ (or schema definition) -for the `tweet` type in the `gb` index: +推测起来，这是因为数据在 `_all` 字段与 `data` 字段的索引方式不同。所以，通过请求 `gb` 索引中 `tweet` 类型的((("mapping (types)")))_映射_（或模式定义），让我们看一看 Elasticsearch 是如何解释我们文档结构的： [source,js] -------------------------------------------------- @@ -33,7 +23,7 @@ GET /gb/_mapping/tweet // SENSE: 052_Mapping_Analysis/25_Data_type_differences.json -This gives us the following: +这将得到如下结果： [source,js] -------------------------------------------------- @@ -63,20 +53,8 @@ This gives us the following: -------------------------------------------------- -Elasticsearch has dynamically generated a mapping for us, based on what it -could guess about our field types. The response shows us that the `date` field -has been recognized as a field of type `date`. ((("_all field", sortas="all field")))The `_all` field isn't -mentioned because it is a default field, but we know that the `_all` field is -of type `string`.((("string fields"))) +基于对字段类型的猜测， Elasticsearch 动态为我们产生了一个映射。这个响应告诉我们 `date` 字段被认为是 `date` 类型的。由于((("_all field", sortas="all field"))) `_all` 是默认字段，所以没有提及它。但是我们知道 `_all` 字段是 `string` 类型的。((("string fields"))) -So fields of type `date` and fields of type `string` are((("indexing", "differences in, for different core types"))) indexed differently, -and can thus be searched differently. That's not entirely surprising. -You might expect that each of the ((("data types", "core, different indexing of")))core data types--strings, numbers, Booleans, -and dates--might be indexed slightly differently. And this is true: -there are slight differences. - -But by far the biggest difference is between fields((("exact values", "fields representing")))((("full text", "fields representing"))) that represent -_exact values_ (which can include `string` fields) and fields that -represent _full text_. This distinction is really important--it's the thing -that separates a search engine from all other databases. +所以 `date` 字段和 `string` 字段((("indexing", "differences in, for different core types")))索引方式不同，因此搜索结果也不一样。这完全不令人吃惊。你可能会认为((("data types", "core, different indexing of")))核心数据类型 strings、numbers、Booleans 和 dates 的索引方式有稍许不同。没错，他们确实稍有不同。 +但是，到目前为止，最大的差异在于((("exact values", "fields representing")))((("full text", "fields representing")))代表 _精确值_ （它包括 `string` 字段）的字段和代表 _全文_ 的字段。这个区别非常重要——它将搜索引擎和所有其他数据库区别开来。 diff --git a/130_Partial_Matching/05_Postcodes.asciidoc b/130_Partial_Matching/05_Postcodes.asciidoc index 5aea8d103..d3a47a907 100644 --- a/130_Partial_Matching/05_Postcodes.asciidoc +++ b/130_Partial_Matching/05_Postcodes.asciidoc @@ -7,7 +7,7 @@ postcode `W1V 3DG` can((("postcodes (UK), partial matching with"))) be broken do * `W1V`: This outer part identifies the postal area and district: ** `W` indicates the area (one or two letters) -** `1V` indicates the district (one or two numbers, possibly followed by a letter +** `1V` indicates the district (one or two numbers, possibly followed by a letter) * `3DG`: This inner part identifies a street or building: diff --git a/402_Nested/30_Nested_objects.asciidoc b/402_Nested/30_Nested_objects.asciidoc index fc90de870..be4392d8b 100644 --- a/402_Nested/30_Nested_objects.asciidoc +++ b/402_Nested/30_Nested_objects.asciidoc @@ -79,7 +79,7 @@ The correlation between `Alice` and `31`, or between `John` and `2014-09-01`, ha from a search point of view, for storing an array of objects. This is the problem that _nested objects_ are designed to solve. By mapping -the `commments` field as type `nested` instead of type `object`, each nested +the `comments` field as type `nested` instead of type `object`, each nested object is indexed as a _hidden separate document_, something like this: [source,json] diff --git a/510_Deployment/20_hardware.asciidoc b/510_Deployment/20_hardware.asciidoc index acc588466..d7feccb06 100644 --- a/510_Deployment/20_hardware.asciidoc +++ b/510_Deployment/20_hardware.asciidoc @@ -1,117 +1,69 @@ [[hardware]] -=== Hardware - -If you've been following the normal development path, you've probably been playing((("deployment", "hardware")))((("hardware"))) -with Elasticsearch on your laptop or on a small cluster of machines laying around. -But when it comes time to deploy Elasticsearch to production, there are a few -recommendations that you should consider. Nothing is a hard-and-fast rule; -Elasticsearch is used for a wide range of tasks and on a bewildering array of -machines. But these recommendations provide good starting points based on our experience with -production clusters. - -==== Memory - -If there is one resource that you will run out of first, it will likely be memory.((("hardware", "memory")))((("memory"))) -Sorting and aggregations can both be memory hungry, so enough heap space to -accommodate these is important.((("heap"))) Even when the heap is comparatively small, -extra memory can be given to the OS filesystem cache. Because many data structures -used by Lucene are disk-based formats, Elasticsearch leverages the OS cache to -great effect. - -A machine with 64 GB of RAM is the ideal sweet spot, but 32 GB and 16 GB machines -are also common. Less than 8 GB tends to be counterproductive (you end up -needing many, many small machines), and greater than 64 GB has problems that we will -discuss in <>. +=== 硬件 + +按照正常的流程，((("deployment", "hardware")))((("hardware")))你可能已经在自己的笔记本电脑或集群上使用了 Elasticsearch。 +但是当要部署 Elasticsearch 到生产环境时，有一些建议是你需要考虑的。这里没有什么必须要遵守的准则，Elasticsearch 被用于在众多的机器上处理各种任务。基于我们在生产环境使用 Elasticsearch 集群的经验，这些建议可以为你提供一个好的起点。 + +==== 内存 + +如果有一种资源是最先被耗尽的，它可能是内存。((("hardware", "memory")))((("memory")))排序和聚合都很耗内存，所以有足够的堆空间来应付它们是很重要的。((("heap")))即使堆空间是比较小的时候， +也能为操作系统文件缓存提供额外的内存。因为 Lucene 使用的许多数据结构是基于磁盘的格式，Elasticsearch 利用操作系统缓存能产生很大效果。 + +64 GB 内存的机器是非常理想的， 但是32 GB 和16 GB 机器也是很常见的。少于8 GB 会适得其反（你最终需要很多很多的小机器），大于64 GB 的机器也会有问题， +我们将在 <> 中讨论。 ==== CPUs -Most Elasticsearch deployments tend to be rather light on CPU requirements. As -such,((("CPUs (central processing units)")))((("hardware", "CPUs"))) the exact processor setup matters less than the other resources. You should -choose a modern processor with multiple cores. Common clusters utilize two to eight -core machines. +大多数 Elasticsearch 部署往往对 CPU 要求不高。因此，((("CPUs (central processing units)")))((("hardware", "CPUs")))相对其它资源，具体配置多少个（CPU）不是那么关键。你应该选择具有多个内核的现代处理器，常见的集群使用两到八个核的机器。 -If you need to choose between faster CPUs or more cores, choose more cores. The -extra concurrency that multiple cores offers will far outweigh a slightly faster -clock speed. +如果你要在更快的 CPUs 和更多的核心之间选择，选择更多的核心更好。多个内核提供的额外并发远胜过稍微快一点点的时钟频率。 -==== Disks +==== 硬盘 -Disks are important for all clusters,((("disks")))((("hardware", "disks"))) and doubly so for indexing-heavy clusters -(such as those that ingest log data). Disks are the slowest subsystem in a server, -which means that write-heavy clusters can easily saturate their disks, which in -turn become the bottleneck of the cluster. +硬盘对所有的集群都很重要，((("disks")))((("hardware", "disks")))对大量写入的集群更是加倍重要（例如那些存储日志数据的）。硬盘是服务器上最慢的子系统，这意味着那些写入量很大的集群很容易让硬盘饱和，使得它成为集群的瓶颈。 -If you can afford SSDs, they are by far superior to any spinning media. SSD-backed -nodes see boosts in both query and indexing performance. If you can afford it, -SSDs are the way to go. +如果你负担得起 SSD，它将远远超出任何旋转介质（注：机械硬盘，磁带等）。 基于 SSD 的节点，查询和索引性能都有提升。如果你负担得起，SSD 是一个好的选择。 -.Check Your I/O Scheduler -**** -If you are using SSDs, make sure your OS I/O scheduler is((("I/O scheduler"))) configured correctly. -When you write data to disk, the I/O scheduler decides when that data is -_actually_ sent to the disk. The default under most *nix distributions is a -scheduler called `cfq` (Completely Fair Queuing). - -This scheduler allocates _time slices_ to each process, and then optimizes the -delivery of these various queues to the disk. It is optimized for spinning media: -the nature of rotating platters means it is more efficient to write data to disk -based on physical layout. - -This is inefficient for SSD, however, since there are no spinning platters -involved. Instead, `deadline` or `noop` should be used instead. The deadline -scheduler optimizes based on how long writes have been pending, while `noop` -is just a simple FIFO queue. - -This simple change can have dramatic impacts. We've seen a 500-fold improvement -to write throughput just by using the correct scheduler. +.检查你的 I/O 调度程序 **** +如果你正在使用 SSDs，确保你的系统 I/O 调度程序是((("I/O scheduler")))配置正确的。 +当你向硬盘写数据，I/O 调度程序决定何时把数据实际发送到硬盘。 +大多数默认 *nix 发行版下的调度程序都叫做 `cfq`（完全公平队列）。 + +调度程序分配 _时间片_ 到每个进程。并且优化这些到硬盘的众多队列的传递。但它是为旋转介质优化的： +机械硬盘的固有特性意味着它写入数据到基于物理布局的硬盘会更高效。 -If you use spinning media, try to obtain the fastest disks possible (high-performance server disks, 15k RPM drives). +这对 SSD 来说是低效的，尽管这里没有涉及到机械硬盘。但是，`deadline` 或者 `noop` 应该被使用。`deadline` 调度程序基于写入等待时间进行优化， +`noop` 只是一个简单的 FIFO 队列。 -Using RAID 0 is an effective way to increase disk speed, for both spinning disks -and SSD. There is no need to use mirroring or parity variants of RAID, since -high availability is built into Elasticsearch via replicas. +这个简单的更改可以带来显著的影响。仅仅是使用正确的调度程序，我们看到了500倍的写入能力提升。 +**** -Finally, avoid network-attached storage (NAS). People routinely claim their -NAS solution is faster and more reliable than local drives. Despite these claims, -we have never seen NAS live up to its hype. NAS is often slower, displays -larger latencies with a wider deviation in average latency, and is a single -point of failure. +如果你使用旋转介质，尝试获取尽可能快的硬盘（高性能服务器硬盘，15k RPM 驱动器）。 -==== Network +使用 RAID 0 是提高硬盘速度的有效途径，对机械硬盘和 SSD 来说都是如此。没有必要使用镜像或其它 RAID 变体，因为高可用已经通过 replicas 内建于 Elasticsearch 之中。 -A fast and reliable network is obviously important to performance in a distributed((("hardware", "network")))((("network"))) -system. Low latency helps ensure that nodes can communicate easily, while -high bandwidth helps shard movement and recovery. Modern data-center networking -(1 GbE, 10 GbE) is sufficient for the vast majority of clusters. +最后，避免使用网络附加存储（NAS）。人们常声称他们的 NAS 解决方案比本地驱动器更快更可靠。除却这些声称， +我们从没看到 NAS 能配得上它的大肆宣传。NAS 常常很慢，显露出更大的延时和更宽的平均延时方差，而且它是单点故障的。 -Avoid clusters that span multiple data centers, even if the data centers are -colocated in close proximity. Definitely avoid clusters that span large geographic -distances. +==== 网络 -Elasticsearch clusters assume that all nodes are equal--not that half the nodes -are actually 150ms distant in another data center. Larger latencies tend to -exacerbate problems in distributed systems and make debugging and resolution -more difficult. +快速可靠的网络显然对分布式系统的性能是很重要的((("hardware", "network")))((("network")))。 +低延时能帮助确保节点间能容易的通讯，大带宽能帮助分片移动和恢复。现代数据中心网络（1 GbE, 10 GbE）对绝大多数集群都是足够的。 -Similar to the NAS argument, everyone claims that their pipe between data centers is -robust and low latency. This is true--until it isn't (a network failure will -happen eventually; you can count on it). From our experience, the hassle of -managing cross–data center clusters is simply not worth the cost. +即使数据中心们近在咫尺，也要避免集群跨越多个数据中心。绝对要避免集群跨越大的地理距离。 -==== General Considerations +Elasticsearch 假定所有节点都是平等的--并不会因为有一半的节点在150ms 外的另一数据中心而有所不同。更大的延时会加重分布式系统中的问题而且使得调试和排错更困难。 -It is possible nowadays to obtain truly enormous machines:((("hardware", "general considerations"))) hundreds of gigabytes -of RAM with dozens of CPU cores. Conversely, it is also possible to spin up -thousands of small virtual machines in cloud platforms such as EC2. Which -approach is best? +和 NAS 的争论类似，每个人都声称他们的数据中心间的线路都是健壮和低延时的。这是真的--直到它不是时（网络失败终究是会发生的，你可以相信它）。 +从我们的经验来看，处理跨数据中心集群的麻烦事是根本不值得的。 -In general, it is better to prefer medium-to-large boxes. Avoid small machines, -because you don't want to manage a cluster with a thousand nodes, and the overhead -of simply running Elasticsearch is more apparent on such small boxes. +==== 总则 -At the same time, avoid the truly enormous machines. They often lead to imbalanced -resource usage (for example, all the memory is being used, but none of the CPU) and can -add logistical complexity if you have to run multiple nodes per machine. +获取真正的高配机器在今天是可能的：((("hardware", "general considerations")))成百 GB 的 RAM 和几十个 CPU 核心。 +反之，在云平台上串联起成千的小虚拟机也是可能的，例如 EC2。哪种方式是最好的? +通常，选择中配或者高配机器更好。避免使用低配机器， +因为你不会希望去管理拥有上千个节点的集群，而且在这些低配机器上运行 Elasticsearch 的开销也是显著的。 +与此同时，避免使用真正的高配机器。它们通常会导致资源使用不均衡（例如，所有的内存都被使用，但 CPU 却没有）而且在单机上运行多个节点时，会增加逻辑复杂度。 diff --git a/520_Post_Deployment/60_restore.asciidoc b/520_Post_Deployment/60_restore.asciidoc index d4f383eb6..f0ef88b6d 100644 --- a/520_Post_Deployment/60_restore.asciidoc +++ b/520_Post_Deployment/60_restore.asciidoc @@ -78,7 +78,7 @@ GET /_recovery/ ---- The output will look similar to this (and note, it can become very verbose -depending on the activity of your clsuter!): +depending on the activity of your cluster!): [source,js] ----