Industry 4.0, or the Industrial Internet of Things (IIoT), is about industrial transformation using new digital technology that makes it possible to gather and analyze data across machines and business systems, enabling faster, more flexible, and more efficient processes to produce higher-quality individualized goods at lower cost by improving resource productivity and efficiency, driving agility and responsiveness, speed to market, and enabling customization to meet customer needs.
Industry 4.0 has catapulted the usage of drones as flying sensors. Drones are now being used extensively for aerial photography, surveillance and 3D mapping, shipping, remote sensing etc. It has found a place in almost all kinds of industrial ecosystems. The increasing adaption of drones can be attributed to the easy availability and customization of drones in the market, minimal change to infrastructure when adding them into an existing workplace and the possibility of reducing manual labor.
Drones for Smart Warehouse Management
We at SAP have been adapting to the changing trend and have included drones as part of our solution for smart warehouse management. Our current solution uses drones for performing inventory checks in a warehouse. They are used to count the number of packages, scan these packages and pass these info into the database where these numbers can be cross checked with the inventory. For the demo, we have used a single shelf of cardboard boxes. The drone takes off, follows a specific route to reach the shelf and takes a picture of the shelf and passes it to the backend and lands in the same position from where it took off. At the back end, on receiving the image, the number of boxes is estimated and stored in a database. The data from the database is polled and displayed on a dashboard. A mock dashboard is shown in Figure1. As shown in the figure, the dashboard provides a good picture of the current status of the warehouse.
Drones can be of several flavors, i.e. drones with cameras, GPUs, pickers, weight lifters etc. Depending on the use case, we have to select an appropriate drone. In order to perform inventory checks we just need a drone that has a camera tagged to it and can pass frames on the go for further processing. Since this drone is going to be flying within a warehouse that has a specific layout, we can preprogram the drone to follow a certain path. If we want the drone to find its way around the warehouse, we will need a drone fitted with a camera as well as a GPU. For our demo, we have preprogrammed the drone to follow a simple path and hence we have used a normal Tello EDU drone.
A dashboard has been created using Angular and the back-end is written in python. The Tello drone comes with an SDK that lets us program the drone by sending instructions to it over WiFi using UDP. The dashboard enables the end user to start the drone while the back-end code sends out requests to the drone to route the drone to a specific point of location and to capture an image and send it to the back-end. This captured image is then fed into a deep neural network that perform object detection and the total count of the objects are sent back to the dashboard along the captured images that has bounding boxes drawn around our region of interest ( i.e. packages) along with the confidence percentage of the object detected.
The neural network used for the current task is YOLO, You Only Look Once. The sole reason behind choosing this algorithm over faster RCNN and Single shot detector(SSD) is that the inference time is very less for Yolo as compared to others. Additionally, the model has been trained on images that were captured from the same shelf and hence the accuracy of the model is quite good. The data was collected by running the drone around the shelf at various distances from the shelf and at different angles. This data was augmented by doing some flip operations and using some image enhancement techniques to change the brightness, intensity and contrast of the images.
Innovating our way forward one step at a time, SAP.
Dashboard conceptualisation & design – Sai Prakash