Skip to content

Elasticsearch search

ElasticsearchSearchExtract

Bases: ExtractionMethod

Search Elasticsearch.

Method name: elasticsearch_search

Example Configuration

.. code-block:: yaml

- method: elasticsearch
  inputs:
    index: ceda-index
    client_kwargs:
      hosts: ['host1:9200','host2:9200']
    search_kwargs:
      timeout: 100s
    body:
      query:
        regex: $regex_value
      _source:
        - path
Source code in extraction_methods/plugins/elasticsearch_search.py 
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
class ElasticsearchSearchExtract(ExtractionMethod):
    """
    Search Elasticsearch.

    **Method name:** ``elasticsearch_search``

    Example Configuration:
        .. code-block:: yaml

            - method: elasticsearch
              inputs:
                index: ceda-index
                client_kwargs:
                  hosts: ['host1:9200','host2:9200']
                search_kwargs:
                  timeout: 100s
                body:
                  query:
                    regex: $regex_value
                  _source:
                    - path
    """

    input_class = ElasticsearchSearchInput

    @update_input
    def run(self, body: dict[str, Any]) -> dict[str, Any]:

        es = Elasticsearch_client(**self.input.client_kwargs)

        # Run search
        result = es.search(
            index=self.input.index, body=self.input.body, **self.input.search_kwargs
        )

        body[self.input.output_key] = result["hits"]["hits"]

        return body

ElasticsearchSearchInput

Bases: Input

Model for Elasticsearch Assets Backend Input.

Parameters:

Name Type Description Default
index str

Index to search on.

 required
client_kwargs dict[str, Any]

Client kwargs.

 {}
search_kwargs dict[str, Any]

Search kwargs.

 {'timeout': '60s'}
body dict[str, Any]

Body of search request.

 required
output_key str

key to output to.

 'es_result'
Source code in extraction_methods/plugins/elasticsearch_search.py 
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
class ElasticsearchSearchInput(Input):
    """
    Model for Elasticsearch Assets Backend Input.
    """

    index: str = Field(
        description="Index to search on.",
    )
    client_kwargs: dict[str, Any] = Field(
        default={},
        description="Client kwargs.",
    )
    search_kwargs: dict[str, Any] = Field(
        default={"timeout": "60s"},
        description="Search kwargs.",
    )
    body: dict[str, Any] = Field(
        description="Body of search request.",
    )
    output_key: str = Field(
        default="es_result",
        description="key to output to.",
    )