Another Nifi Experiment: Hashing Files – Part 3

In the first two installments (1, 2) of this series on Nifi, I demonstrated how I used Nifi to generate MD5 hashes for all the files on my hard drive and save them to a MongoDB for quick reference. I also demonstrated how I checked the database for the existence of the hash value before … Continue reading Another Nifi Experiment: Hashing Files – Part 3 →

Another Nifi Experiment: Hashing Files – Part 2

In part 1 of this blog I explained how I used Nifi to read files from my hard drive, create MD5 hashes for them, and check a database to determine if they were duplicates. I also reviewed how I used Wait/Notify processors as a gate to ensure only one file at a time was processed … Continue reading Another Nifi Experiment: Hashing Files – Part 2 →

Another Nifi Experiment: Hashing Files – Part 1

I had some more fun experimenting with Nifi this week. I set out to create a database containing MD5 hash values of all the files on my hard drive. I wanted to use only Nifi out-of-the-box components and was encouraged to find processors already designed to read files from a file system, create hashes, and … Continue reading Another Nifi Experiment: Hashing Files – Part 1 →

YOLO and Nifi: You Only LOOK Once – Part 2

In part 1 of this blog I described how I integrated a YOLO (You Only Look Once) image classification process with Nifi. The first part of the process sent images to YOLO for classification and retrieved the results. The second part, and the subject of this blog, will aggregate the results of each image into … Continue reading YOLO and Nifi: You Only LOOK Once – Part 2 →

YOLO and Nifi: You Only LOOK Once – Part I

YOLO, You Only Look Once, is a real-time object detection system. You can read the research paper or watch the videos on the website to get a feel for what this system offers. Best of all, the source code is completely free! This blog will not get into retraining the system or implementing it for … Continue reading YOLO and Nifi: You Only LOOK Once – Part I →

Apache Nifi – Part 1

For the past few days I've been experimenting with Apache Nifi. Nifi is a scalable, visual programming tool for developing and running data migration, transformation, and processing dataflows within and among systems. Dataflows are built by dropping pre-built processors on a canvas, configuring them, and connecting their inputs and outputs as necessary. Each processor is … Continue reading Apache Nifi – Part 1 →

Apache Nifi – Part 2

In my last blog about Nifi, I demonstrated a simple dataflow to obtain current weather data from Open Weather Map and write it to a JSON file. Recall the overview of the dataflow shown in the figure below. Nifi dataflow to query http://www.openweathermap.org for local weather, save data as JSON, and display results in web … Continue reading Apache Nifi – Part 2 →