Data routing, transformation, and system mediation in Big Data & IoT scenarios with Apache NiFi


Holistic Security and Technology

So a few months ago I published a serie of post explaining how to capture WIFI traffic and process it near to real time by using WSO2 BAM, CEP Siddhi, Apache Cassandra, Apache Thrift, Kismet running on a Raspberry Pi and Docker.

01-wifi-traffic-capture-wso2-bam

Now, after several Big Data and Security projects, I can add to previous solution, fresh air and improve the technological approach.

Using Elasticsearch, Logstash and Kibana

Well, the first approach I considered was starting with ELK stack (Elasticsearch, Logstash and Kibana), that is the natural way to follow.

02-wifi-traffic-capture-elasticsearch-logstash-kibana

But, there are still some issues to face:

  • Deal with the resilience.
    • Several times Logstash stops because it was processing a malformed incoming message.
  • Portability.
    • Logstash uses Java, Ruby and should be compiled and tuned for ARM architectures (Raspberry Pi). Yes, there are some instructions to do that, but I don’t want to spent time to do that…

View original post 631 more words

Advertisements

Thank for your comments

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s