After adding the new elasticsearch node, initially, I was struggling with error :
WARN: org.elasticsearch.discovery: [logstash-id.xxx.in-25379-6424] waited for 30s and no initial state was set by the discovery
and I corrected the situation by adding iptables rules.
Thereafter, things appeared to be smooth but could not find new elasticsearch index using Elasticsearch head plugin even after few minutes. So, I started searching through debug logs and spotted an error :
log4j, [2015-05-12T11:00:02.003] DEBUG: org.elasticsearch.discovery.zen: [logstash-id.xxx.in-26328-4264] filtered ping responses: (filter_client[true], filter_data[false]) {none}
Exception in thread ">output" org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
log4j, [2015-05-12T11:00:06.507] DEBUG: org.elasticsearch.discovery.zen: [logstash-id.xxx.in-26328-4264] filtered ping responses: (filter_client[true], filter_data[false]) {none}
^CInterrupt received. Shutting down the pipeline. {:level=>:warn, :file=>"logstash/agent.rb", :line=>"119"}
Since I was using latest version of logstash and elasticsearch, I was bit puzzled as all the google solutions(references) were pointing to the old versions of them:
[root@id admin]# /opt/logstash/bin/logstash --version
logstash 1.4.2-modified
[root@es2 ~]# rpm -qa|grep elastic -i
elasticsearch-1.4.4-1.noarch
Finally, after reading good documentation of logstash, I decided to add 'protocol' option and vola! - it worked!!
output {
#stdout { codec => rubydebug }
if [type] == "netflow" {
elasticsearch {
cluster => "elk-cluster"
index => "netflow-%{+YYYY.MM.dd}"
host => "10.4.0.47"
protocol => "http"
workers => 2
}
}
WARN: org.elasticsearch.discovery: [logstash-id.xxx.in-25379-6424] waited for 30s and no initial state was set by the discovery
and I corrected the situation by adding iptables rules.
Thereafter, things appeared to be smooth but could not find new elasticsearch index using Elasticsearch head plugin even after few minutes. So, I started searching through debug logs and spotted an error :
log4j, [2015-05-12T11:00:02.003] DEBUG: org.elasticsearch.discovery.zen: [logstash-id.xxx.in-26328-4264] filtered ping responses: (filter_client[true], filter_data[false]) {none}
Exception in thread ">output" org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
log4j, [2015-05-12T11:00:06.507] DEBUG: org.elasticsearch.discovery.zen: [logstash-id.xxx.in-26328-4264] filtered ping responses: (filter_client[true], filter_data[false]) {none}
^CInterrupt received. Shutting down the pipeline. {:level=>:warn, :file=>"logstash/agent.rb", :line=>"119"}
Since I was using latest version of logstash and elasticsearch, I was bit puzzled as all the google solutions(references) were pointing to the old versions of them:
[root@id admin]# /opt/logstash/bin/logstash --version
logstash 1.4.2-modified
[root@es2 ~]# rpm -qa|grep elastic -i
elasticsearch-1.4.4-1.noarch
Finally, after reading good documentation of logstash, I decided to add 'protocol' option and vola! - it worked!!
output {
#stdout { codec => rubydebug }
if [type] == "netflow" {
elasticsearch {
cluster => "elk-cluster"
index => "netflow-%{+YYYY.MM.dd}"
host => "10.4.0.47"
protocol => "http"
workers => 2
}
}
No comments:
Post a Comment