IoT based insect monitoring concept

By Kuldeep Singh

- 8 minutes read - 1593 words

Insects are the most diverse species of animals found on earth. While every species is required to maintain the balance of nature, excessive accumulation of any one type can have devastating effects on humans and environment.

Insect management relies on the early detection and accuracy of insect population monitoring techniquessensors-12-15801. Integrated data gathering about insect population together with the ecological factors like temperature, humidity, light etc., makes it possible to execute appropriate pest control at the right time in the right place.

A well-known technique to perform pest control monitoring is based on the use of insect traps conveniently spread over the specified control area. Human operators typically performs periodical surveys of the traps disseminated through the field. This is a labour, time and cost-consuming activity. This paper proposes an affordable, autonomous monitoring system that uses image processing to detect insect population at regular intervals and sends that data to a web server via an IOT protocol (MQTT in this case). A web dashboard visualises the insect density at different time instances, with a warning system if the density crosses a set threshold value.

Solution Concept

Due to the rapid development of digital technology, there is an opportunity for image processing technology to be used for the automation of insect detection. This paper illustrates a system based on a low cost, low power imaging and processing device operated through a wireless sensor network that is able to automatically acquire images of the trapping area and process them to count the insect density. The data is then sent to a web server where it gets stored in a database. The web server hosts a web dashboard through which human operators can remotely monitor the insect traps.

The insect monitoring system is developed using open source software on a Linux based machine. The app runs on a small device like Raspberry Pi 2 with the help of Raspbian Jessie OS. The work flow is as such that the application running on the raspberry pi 2 captures few seconds worth of video footage at regular intervals. The frames captured are then processed using openCV image processing library to calculate the insect density during that time frame. Once the data is available, it is then to a web server using the IoT protocol MQTT (Message Queuing Telemetry Transport). The web server receives the data sent and stores it in the database. We have a web dashboard hosted on the web server. It retrieves the data from the database and visualises the insect density at different time instances, with a warning system if the density crosses a set threshold value.

Here are the key components of the system :

  • OpenCv Image Processing - OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision and image processing. We use the C/C++ interface to capture images from the webcam and process those images to calculate the insect count.

  • Mosquitto MQTT Broker - Mosquitto is an open source message broker that implements the MQTT protocol for carrying out messages using publisher/subscriber method. We send out the json messages from our application to the web server.

  • Paho MQTT client for python - Paho MQTT python client library implements the MQTT protocol and provides easy to use API which enables out python script to connect to the MQTT broker and receive messages from the raspberry pi application.

  • SQLite Database - SQLite provides a lightweight disk-based database that doesn’t require a separate server process and allows accessing the database using a nonstandard variant of the SQL query language. We use the sqlite database to store the data received by the python script.

  • Apache web server for the User Web Dashboard - We have used apache2 as our web server to host the web dashboard. The dashboard is written as a python script which we execute as a CGI program in the apache web server.

Implementation Strategy

Our webcam will be located inside the trap, fixed at the top so that it faces the bottom of the trap. It will be connected to the raspberry pi (located somewhere outside the trap) with a USB cable.

The C++ application deployed in the raspberry pi accesses the webcam to capture the images of the trap and process them using opencv library to calculate the insect count. When the program is first initialised, it captures initial video frames of the empty insect trap to be used as a reference. After that it captures the change in the scene at regular intervals. The image frames captured are then processed using the opencv functions. In our system, we use their background subtraction algorithm to generate the foreground masks that clearly show the insects trapped. Following figure shows the image captured by the webcam and the foreground mask generated.

The foreground masks generated are then processed to calculate the number of insects trapped. A JSON string is created with current timestamp and the insect count. This string is then sent to the web server by publishing it using the Mosquitto broker and its corresponding mosquito_pub client. A python script is executing on the web server that receives this json message we have sent. It uses the paho mqtt client to subscribe to the same topic as our c++ application. The json data received is parsed and stored in the sqlite database on the server. The SQLite database contains only a single table called insect_density. It contains two columns.

  1. date time: It contains the datetime data and stores the timestamp received in the json string. It also acts as a primary key.
  2. density: It contains integer data and stores the number of insects at time instance stored in the date_time column.

Finally, the data stored in the database is consumed by the web dashboard that visualises the data into a line chart format. The dashboard is written in a python script that is executed as a CGI program on the apache server and can be accessed using any browser. The dashboard also shows a warning message if the insect density exceeds a set threshold limit.

Benefits

Monitoring of insect infestation relies on manpower, however automatic monitoring has been advancing in order to minimize human efforts and errors. The aim of this paper is to establish a fully computerized system that overcome traditional and classical methods of monitoring and is environmentally and user friendly. The main advantages of our proposal are:

  • A more accurate and reliable system for monitoring insects’ activity as it mitigates human made errors
  • Reduces human effort hence lowering time and cost efforts
  • Image analysis provides a realistic opportunity for the automation of insect pest detection
  • high degree of accuracy at least when tried under controlled environments.
  • effective use of pest surveillance and monitoring system which may result to efficient timing of interventions
  • The user interface and the software are friendly and easy to follow; they are understandable by normal users.
  • Higher scalability, being able to deploy in small monitoring areas (greenhouses) as in large plantation extensions
  • Trap monitoring data may be available in real time through an Internet connection

Constraints and Challenges

Currently the Insect Monitoring system have following limitations that can be worked upon in coming future:

  • Presently, the configuration for the server IP address, buffer message limit and the density threshold value have to be set manually. We plan to create a configuration page for the user to remotely configure the application running on the raspberry pi.
  • We had planned to send the image of the trap along with the timestamp and the insect count in the json string via mqtt. We ran into problems while serialising the image and it became difficult to do so within the given time frame.

Conclusion

The Insect Monitoring System displays lot of potential and can be extended to implement advance features and functionalities. The advance feature list includes but not limited to the following:

  • Higher scalability: The system can be scaled to a large number of devices over large areas.
  • Stream Analytics: We can port the system over to the cloud and apply stream analytics and/or machine learning algorithms to get accurate predictions and patterns of insect activity according to ecological factors.
  • Advanced Image Processing: In the future, other image processing techniques may be used to enable the detection and extraction of insects more efficient and accurate. As of now, we are only processing the count of insects without identifying what kind of insect it is. Identification of different type of insects can be done.
  • Show Live Feed: We can incorporate the option to show live feed from the webcam. Web dashboard can contain an option which will enable the user to select a particular device and show the live feed of the insect trap from the webcam.

References

This concept was supported by my colleague Ricktam K at IoT Center of Excellence of Nagarro

＃insect monitoring ＃IOT ＃raspberry pi ＃concept ＃c++ ＃opencv ＃python ＃MQTT ＃nagarro ＃technology