Typical event rate · category

Application log rate: 100 events/s per device

An application source typically produces 100 events per second per device, bursting to 1,000: web access logs high, service audit trails low. The rate is an informed estimate, a starting point for a sizing, not a measurement; the event sizes below are measured.

100
events per second, typical
1,000
at the burst
400 B
median event size, 7 sources
3.5 GB
per device per day

Measured sources in this category

Average bytes per event, measured, and one device's volume at the typical 100 events per second.

SourceBytes per eventA dayA year
Apache Flink2,00017.3 GB6.3 TB
Apache Kafka2,00017.3 GB6.3 TB
Office 3651,95816.9 GB6.2 TB
Google Audit4003.5 GB1.3 TB
Apache HTTP Server3002.6 GB946 GB
Twitter message1401.2 GB442 GB
Open SSH111959 MB350 GB

At the median size, a day from one device, 3.5 GB, is about the size of an hour of Netflix in HD.

Size a fleet

1,000
events per second
34.6 GB
per day
3.1 TB
at retention

Raw bytes, before compression, indexing overhead and replication.

Measure a sample of your own

The figure above is an average; yours will differ, and for application logs what moves it most is this: web access logs high, service audit trails low. The profiler reads a sample from application logs in your browser, line by line, one event per line, and reports the bytes per line to set against it. Nothing is uploaded.

Logcaliper, free on iOS, has these sizes and rates built in and sizes a whole list of sources at retention: storage, cluster capacity, nodes.