An incredibly powerful free tool that has been around a long time is Mikrotik's 'The Dude'. Source: about 1 year ago

If you have a powerful enough router in the network, one of your options is to use The DUDE (Documentation). Note that DUDE requires IP access to WinBox from the management network. For some more features, you should also enable SNMP on your routers. DUDE server can also host a syslog server, so you can have centralized log ingestion. The downside of this solution is that it's unfortunately not a fully-featured... Source: over 1 year ago

“The Dude” is what the tool is called (yeah I don’t know why they called it that either). It’s a client-server application and the server nowadays only runs on a Mikrotik router as a separate package. You could use any Mikrotik router but since it’s heavy on logging I use a Mikrotik CHR as a VM just to host the dude. The free CHR license gives you 1Mbps throughput which for SNMP monitoring is plenty. Source: about 2 years ago

MikroTik has a software called The Dude, that runs on some sort of computer acting as a server (an always on computer to receive, monitor, and store the data collected from the statistics). This would be the best way to achieve what you want to do (collecting user usage statistics and storing it in one place). Source: about 2 years ago

Https://mikrotik.com/thedude Supports SNMP, ICMP, DNS and TCP monitoring for devices that support it. Source: over 2 years ago

Pig, a platform/programming language for authoring parallelizable jobs. - Source: dev.to / over 1 year ago

In the early days of the Big Data era when K8s hasn't even been born yet, the common open source go-to solution was the Hadoop stack. We have written several old-fashioned Map-Reduce jobs, scripts using Pig until we came across Spark. Since then Spark has became one of the most popular data processing engines. It is very easy to start using Lighter on YARN deployments. Just run a docker with proper configuration... - Source: dev.to / over 2 years ago