logging - start harvest logs from specific day -
i installed logstash, elasticsearch, kibana, on ubuntu 14.04, use cca 40 logstash-forwarders (ubuntu 12.04 & 14.04), able receive cca 300-400logs per second, start harvest logs since "today", not older logs.. ran logstash few weeks ago , still harvested older logs, , harvest logs more in real-time.
i tried set filters, work curator, tried google it, still dont have solution.
unfortunately, rotate old logs not possible in case, because have store older logs security audit.
if logs aren't in separate files (where might able come glob pattern limit gets indexed), suggest calculating "lag" (the difference between event's timestamp , current timestamp). if lag higher threshold, drop{} event.
Comments
Post a Comment