Implementation of Tensorflow in the CCTV-Based People Counter Application at PT Matahari Department Store, Tbk

Counting the number of visitors (people counter) is an activity carried out in many retail stores. This information, although very simple and basic, can be used as one of the main bases for making decisions. In the past, PT Matahari Department Store, Tbk (MDS) has calculated the number of visitors manually with an accuracy of around 90%. The purpose of this research is to design an automated people counter system using the existing CCTV video networks in MDS stores. The research methodology used is data collection methods in the form of literature studies and interviews, and systems development methods in the form of prototyping. The system uses Tensorflow to recognize visitors who enter and leave the store. This system is a prototype and is used as a proof of concept for MDS to make decisions regarding the application of appropriate technology in carrying out the people counter process along with the things that need to be prepared if this system is to be implemented thoroughly in all MDS stores in Indonesia. Keywords​: people counter, TensorFlow, artificial intelligence, retail shop, computer vision Copyright © 2020 IAIC All rights reserved.


Introduction
People counter aims to estimate the number of people indoors and outdoors. There are many advantages that will be obtained in using a people counter system. For example, by knowing the number of customers who leave and enter the store, we can find out information on the time of the store that is most visited by customers. Not only for shops, but people counters can also be used for other things such as calculating the estimated people who can pass through the evacuation hallway. Therefore, people counters can be useful in various fields. PT Matahari Department Store, Tbk (MDS) has also used a people counter. These people's counter data can be used by MDS to compare with purchase data so that it is known how many customers entered the store at that time and made transactions. Another advantage is that MDS can find out the paths in the store that are traversed by many people so that it is known which shop areas are most interested in customers. There are many other advantages that come with this data [1].
People counters currently running on MDS use infrared devices found on pedestals. Data on the number of customers can be viewed on a computer in a shop and analyzed manually by MDS. One of the weaknesses of this tool is that it cannot distinguish whether a person is detected or not. MDS also wants to develop a people counter system using a piece of hardware, where the device will be attached along the entrance of the MDS. The disadvantage of this device is that the camera range is not wide, so it is needed in large numbers. Therefore, there is one solution that can make it easier to develop this people counter system. By utilizing the CCTV that has been installed in each shop, a people counter system can be developed that will automatically count customers who enter and leave the shop, as well as customers in the store in a certain area [2]. By using a system like this, MDS does not need to buy special tools to implement the people counter information system. The company also does not need to spend money and effort to install the equipment one by one for their shops and does not have to pay any more to maintain the equipment. The goal of MDS to develop a people counter system was achieved and from a cost perspective, it could be more efficient. The research methodology used in system development is data collection methods in the form of literature studies and interviews, and systems development methods in the form of prototyping. Interviews were conducted to find out the requirements needed in the system to be developed. A literature study was conducted to find out how to develop a CCTV-based people counter information system. One of the things done in the literature study is to review previous research that is related to the planning and development of CCTV-based people counter information systems. These studies include: Anuaruddin Ahmadon, Shingo Yamaguchi in 2020. The development method is based on an analytical algorithm using OpenCV, Python, Pi camera, and Raspberry Pi 3 with the Raspbian operating system. The system can count people in real-time in the captured video and send statistics and data to ThingSpeak. • Integration of People Counter Systems to Decision Support Systems in Clothing Retail Industry: A Preliminary Analysis and Possible Applications by Ersin Karaman in 2015. The amount of visitor or customer data can be used to produce analyzes and reports that support the decision-making process.
To improve the quality of the analysis, data must be integrated with other data sources. Information can be used for store comparisons and evaluations for logistics, manager performance, location decision making, and so on. Integrating various technologies such as gender detection, queue management, store heat analysis can support to make more accurate decisions [3].

Research Method 2.1 TensorFlow
TensorFlow is an open-source software library for the computation of numbers for data flow graphs that can help machine learning experts perform more data-intensive computations, developed in 2011 by the Google Brain Team. The main goal of TensorFlow is to conduct research on machine learning, deep neural networks, and various other areas. In 2015, Google made Tensorflow and all of its implementation sources open-source. Google also allows people to access TensorFlow by uploading it on GitHub under the Apache 2.0 license. Because of this, TensorFlow has been adopted widely, from academic to industrial research [4]. The main features of TensorFlow are: • Define, optimize, and compute efficient mathematical expressions involving multi-dimensional arrays (tensors). • Programming support of deep neural networks and machine learning techniques • The use of transparent GPU computing, automation management, and optimization of memory and data used. • High computational scalability across machines and very large data sets.
Currently, the latest version of TensorFlow is TensorFlow 2.0. However, this study uses TensorFlow version 1.0 because CUDA support in TensorFlow version 2.0 for system development is still not perfect. What's more, TensorFlow version 2.0 is not backward-compatible, which will complicate future development. In general, the operations in TensorFlow can be seen in the following figure: ISSN: 2528-2417 ■ 40

Figure 1 . Diagram of the basic operation of TensorFlow
In the image above, the tensor calculation operation requires a model so that TensorFlow can recognize images of people. For this research, from the experiments that have been carried out, the most suitable model to use is ssd_mobilenet_v1_coco_2018_01_28 which is obtained from the TensorFlow website. COCO (Common Objects In Context) is a dataset commonly used for large-scale object detection, segmentation, and text data rendering. The features of the COCO dataset include object segmentation, object recognition in context, and superpixel segmentation. The COCO dataset contains 91 different types of images that can be easily recognized by a 4-year-old child. There are 2.5 million labeled instances in 328 thousand images, making it easier to detect and localize objects with the competitive performance [5].

System Design
At this time, MDS already has a system for counting visitors or the people counter. Based on the existing system, this study explores alternative problem solutions to increase the effectiveness and efficiency of the existing system. This section describes existing systems and new systems that use TensorFlow [6].

Current System
The people counter system currently used in MDS generally uses two methods: manual counting by assigning several people who are at the entrance and counting everyone who enters/enters the store. The second is to use an infrared sensor that is placed on the shop door. Both of these methods have disadvantages including: • In manual counting, errors can most often occur due to human factors or in a busy shop, causing misperceptions. • In calculations using infrared sensors, the sensor cannot detect whether customers (humans) or goods pass through the entrance/exit. Calculation errors generally occur when people who enter carry goods or use trolleys/trains.
According to MDS, the accuracy of using the two methods above is only around 90%.

Proposed System Using Tensorflow
In this study, the people counter system is divided into two main modules: front-end and back-end. Starting from the front-end, the implementation of the proposed system using CCTV is already available in all MDS stores throughout Indonesia. The main input comes from the CCTV which is located at all entrances to the shop. CCTV data is not processed immediately but is sent periodically after ISSN: 2528-2417 ■ 41 the shop closes. The process of sending video data originating from CCTV will be carried out by a special server in each shop which is directly connected to the main server at MDS head office. Due to limited hardware, there are various types of CCTV in each shop the system is designed to send data after the shop is closed [7]. After the data is received by the central server, the data that has been labeled in the database will be processed by the back-end module. This back-end module runs TensorFlow to analyze videos and count the number of people who leave or enter the store. Therefore, the number of people counter statistics in this system has a period of 1 day to get the latest data updates. The proposed system diagram can be seen in Figure 2.

Centroid Tracking Algorithm for People Counter using TensorFlow
The people counter program was developed using the centroid tracking algorithm. The first step is to define the bounding boxes of the object and compute the midpoints of the bounding boxes. As can be seen in Figure 3, it can be seen that there are two objects to track [8]. The second stage, computing the Euclidean distance between the new centroid (yellow) and the existing centroid (purple), which can be seen in Figure 3. The Centroid Tracking algorithm assumes that the centroid pairs with the closest Euclidean distance have the same object ID [9]. Example. In Figure 3, there are two centroids that belong to an existing object (purple), and three centroids owned by a new object (yellow). This means that new objects are detected because the number of new centroids is more than the old centroid. The arrow in figure 4 shows the Euclidean distance between the old centroid (purple) and the new centroid (yellow).
After getting the Euclidean distance, we try to give ID to the object in the third stage, which can be seen in Figure 5. In Figure 5, it can be seen that the closest Euclidean distance between old and new centroids has been selected [10]. However, when there is a centroid that is not related to anything, the fourth step is to register a new object which can be seen in Figure 6. After each object is assigned an ID number, the system will store the position of each object and compare the positions over time. When posisi tersebut berubah dari sisi atas ke bawah dari video dan batasan yang telah ditentukan, maka jumlah orang yang masuk bertambah 1.

Results and Analysis
The people counter system that has been made was tested using CCTV video obtained from Matahari Department Store Supermall branch, Lippo Karawaci. An example of recognition and calculation by the system can be seen in the image below: Figure 7. Visitor counting at MDS Supermall Karawaci Figure 7 shows that the system is functioning properly. However, the experimental results found several problems, including: • The inhomogeneous CCTV specifications resulted in differences in the accuracy of visitor recognition. Some CCTVs are more than 10 years old and have poor image quality or frames per second that are too low to be able to analyze visitor movement. • Sme CCTVs have good specifications, but the image results are affected by external factors such as electric and electromagnetic currents, causing shaky image quality. The experiments performed showed image recognition failures. Examples of CCTV results with unfavorable images can be seen in the image below: Figure 8. An example of a CCTV image that failed to be recognized due to poor quality • From Figure 8, it can also be identified the constraints where the CCTV placement requires a lot of considerations such as angle, lighting, width, and shape of the entrance (some shops have a circular entrance). • In many cases, there is an identification error where the shop employees on duty are also counted by the system because the system cannot distinguish between visitors and employees.

Conclusion
The people counter system that has been created is a proof of concept and will be further developed. The problems that have been identified from the experimental results have been discussed with MDS and several solutions were found as follows: • For the problem of CCTV specifications that are not homogeneous and of poor quality, it will be resolved when MDS does a CCTV update in problem shops. • Based on the discussion and suggestions from this research, some CCTVs will be added and/or relocated so that they can get the right angle. The lack of lighting will also be improved by adding lights. • Based on the discussion and suggestions from this research, some CCTVs will be added and/or relocated so that they can get the right angle. The lack of lighting will also be improved by adding lights. • Mistakes in counting visitors due to misidentification of shop employees have not found a good solution. Future research will seek to solve this problem.
With the completion of this research phase, further research will aim to improve the accuracy of visitor recognition by establishing a TensorFlow data model using the obtained CCTV data. Then, the next research will specifically analyze visitor data so that it can be a supplement to other data such as visitor/transaction ratios, promotion effectiveness, and in-store visitor behavior analysis.