Data routing, transformation, and system mediation in Big Data & IoT scenarios with Apache NiFi

Holistic Security and Technology

So a few months ago I published a serie of post explaining how to capture WIFI traffic and process it near to real time by using WSO2 BAM, CEP Siddhi, Apache Cassandra, Apache Thrift, Kismet running on a Raspberry Pi and Docker.


Now, after several Big Data and Security projects, I can add to previous solution, fresh air and improve the technological approach.

Using Elasticsearch, Logstash and Kibana

Well, the first approach I considered was starting with ELK stack (Elasticsearch, Logstash and Kibana), that is the natural way to follow.


But, there are still some issues to face:

  • Deal with the resilience.
    • Several times Logstash stops because it was processing a malformed incoming message.
  • Portability.
    • Logstash uses Java, Ruby and should be compiled and tuned for ARM architectures (Raspberry Pi). Yes, there are some instructions to do that, but I don’t want to spent time to do that…

View original post 631 more words


Thank for your comments

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.