Tuesday, 12 May 2015

Elasticsearch curator - delete commands

If you wish to delete some old elasticsearch indices, curator is the de-facto standard. The program provides so many useful options and I appreciate the versatility. Although, curator has a wiki and elasticsearch page, things were not that clear to me in one go. So, here are my notes on deleting elasticsearch indices:

[admin@mgr ~]$ curator --version
curator, version 3.0.0

[admin@mgr ~]$ /usr/bin/curator --host 10.4.0.46 delete --help
Usage: curator delete [OPTIONS] COMMAND [ARGS]...

  Delete indices or snapshots

Options:
  --disk-space FLOAT  Delete indices beyond DISK_SPACE gigabytes.
  --reverse BOOLEAN   Only valid with --disk-space. Affects sort order of the
                      indices.  True means reverse-alphabetical (if dates are
                      involved, older is deleted first).  [default: True]
  --help              Show this message and exit.

Commands:
  indices    Index selection.
  snapshots  Snapshot selection.


[admin@mgr ~]$ /usr/bin/curator --host 10.4.0.46 delete indices --help
Usage: curator delete indices [OPTIONS]

  Get a list of indices to act on from the provided arguments, then perform
  the command [alias, allocation, bloom, close, delete, etc.] on the
  resulting list.

Options:
  --newer-than INTEGER            Include only indices newer than n time_units
  --older-than INTEGER            Include only indices older than n time_units
  --prefix TEXT                   Include only indices beginning with prefix.
  --suffix TEXT                   Include only indices ending with suffix.
  --time-unit [hours|days|weeks|months]
                                  Unit of time to reckon by
  --timestring TEXT               Python strftime string to match your index
                                  definition, e.g. 2014.07.15 would be
                                  %Y.%m.%d
  --regex TEXT                    Provide your own regex, e.g
                                  '^prefix-.*-suffix$'
  --exclude TEXT                  Exclude matching indices. Can be invoked
                                  multiple times.
  --index TEXT                    Include the provided index in the list. Can
                                  be invoked multiple times.
  --all-indices                   Do not filter indices.  Act on all indices.
  --help                          Show this message and exit.

If you wish to use dry-run feature in curator:

[admin@mgr ~]$ /usr/bin/curator --host 10.4.0.46 --dry-run --debug delete indices --time-unit days --timestring "\%Y.\%m.\%d" --older-than 7 --prefix logstash- --all-indices

You should always specify logfile and loglevel (CRITICAL = 50, ERROR=40, WARNING=30, INFO=20, DEBUG=10, NOTSET=0) to capture curator program output for ease of debugging and specify sufficient timeout period. The timeout period becomes important when you are taking snapshots of indices. So, always  give sufficient timeout period:

[admin@mgr ~]$ /usr/bin/curator --host 10.4.0.47 --timeout 300 --logfile curator_log.txt --loglevel 10 --debug delete indices --time-unit days --timestring "\%Y.\%m.\%d" --older-than 7 --prefix logstash- --all-indices


[admin@mgr ~]$ /usr/bin/curator --host 10.4.0.46 --timeout 300 --logfile curator_log.txt --loglevel 10 --debug delete indices --time-unit days --timestring "\%Y.\%m.\%d" --older-than 7 --prefix logstash-apache --all-indices

If you wish to delete the indices on the basis of hard disk space,use the following command:

[admin@mgr ~]$ /usr/bin/curator --host 10.4.0.46 --timeout 300 --logfile ttt.txt --loglevel 10 --debug delete --disk-space 40 indices --all-indices




No comments:

Post a Comment