Performance Discussion

[poy]

I am curious about improving performance for splunk ingestion. We have had many customers use this product but run into problems with splunk putting back pressure on the nozzle. We have deduced this to be the cause by observing the nozzle's diode dropping data.

Has the option to write into splunk via batching been explored? In most systems (and therefore I would assume splunk as well), this is the first step to making ingestion faster.

[shinji62]

No back pressure on nozzle, as it will throw away logs if the buffer reach a certain limit.
It's just normal behaviour as Splunk could not ingest logs fast enough.

Same as doppler / Loggregator not having back pressure implementing the same diode concept/code.

Not I did'nt check yet the options of using batch ingestion.