Measured event size · Distributed system
Open Stack log size: 297 bytes per event
The average Open Stack event is 297 bytes, measured from samples of its logs. A distributed system source typically produces 100 events per second per device, bursting to 1,000: per node at INFO; log level is the lever. The rate is an informed estimate; the size is measured.
What that comes to
One device, raw bytes, at the typical rate for a distributed system source.
| Period | At 100 events/s | At the 1,000 events/s burst |
|---|---|---|
| An hour | 107 MB | 1.1 GB |
| A day | 2.6 GB | 25.7 GB |
| 90 days | 231 GB | 2.3 TB |
| A year | 937 GB | 9.4 TB |
A day of Open Stack logs from one device at the category's typical rate, 2.6 GB, is about the size of an hour of Netflix in HD.
Size your own
Raw bytes, before compression, indexing overhead and replication.
Measure a sample of your own
The figure above is an average; yours will differ, and for Open Stack what moves it most is this: per node at INFO; log level is the lever. The profiler reads a sample from Open Stack in your browser, line by line, one event per line, and reports the bytes per line to set against it. Nothing is uploaded.
Other distributed system sources
- Apache Hadoop192 B
- Apache Zookeeper139 B
- Apache Spark97 B
The distributed system category: its typical rate and every source in it →
Logcaliper, free on iOS, has these sizes and rates built in and sizes a whole list of sources at retention: storage, cluster capacity, nodes.