Algorithm Implementation for Detection and Tracking of Ships Using FMCW Radar

1. INTRODUCTION

According to the increasing maritime trades and development of marine transportation, marine surveillance must come into action for prevention of accident and guarantee of safety and security sailing. As a part of surveillance, tracking system may integrate with heterogeneous data in order to overcome the limitations of each sensor. In the maritime domain, sensor assets may include Automatic Identification System (AIS) tracks, contacts from coastal radar, video, IR, and Synthetic Aperture Radar (SAR) (Guerriero et al. [2008]). In the coastal water, satellite and radar are mostly used for object detection and tracking (EMSA [2011]). But each of the equipment has limited performance. In case of SAR, it covers a large area with fine spatial resolution under all weather conditions (Chaturvedi et al. [2012]) but limited temporal coverage, while radar has a limited spatial coverage, resolution decrease according to the distance and long warm-up time. Therefore, if we use both satellite and radar to marine surveillance, it will be useful for real time monitoring of the wide ocean. In this study, we obtain information of targets position and track before integrating satellite and radar data. FMCW radar has a wide variety advantage: simple solid-state transmitters, resistance to interception, good range resolution (Stove [1992]). It can improve maritime safety and security as well as possibility to expand compacter surveillance coverage system from cost-cutting. There are many types of ship-mounted FMCW radar for smaller ships and ship detection algorithm has been developed.

Currently, object detection and tracking technology in a complex environment has been studied: e.g. human, aircraft and ship tracking for surveillance, medical image processing, hand, face and eye tracking for human-computer interaction etc. (Yang [2008]). There are many studies that developed detection and tracking algorithm using radar for maritime surveillance system. Mecocci et al. [1995] studied a new experimental an automatic target recognition (ATR) system for ship traffic control in the access area of a sea-port using real aperture X-band radar. Each of targets is extracted from three steps that are segmentation, contour extraction, and feature extraction and then tracked. Lee [2008] used ARPA radar for consolidation of AIS and radar positions by comparing the AIS and radar information for the tracked ship targets using PC-based ECDIS in Korea. Khan et al. [1994] showed Doppler processing for the horizon detection and tracking of a variety of targets including vessels, aircraft, and icebergs using shore-based ground real time frequency modulated interrupted continuous wave radar system. In Korea, ship detection and tracking system stays on the level of individual development and depends on overseas. For this reason, this study focuses on algorithm design using FMCW radar for the horizon detection and tracking of ships. It is intended to develop easy and effective ships detection and tracking algorithm using X-band FMCW radar. The ships are detection through the image processing and detected ships are tracked by easy and effective methods using searchwindow.

In section 2, experimental design such as site and radar specification, and conversion method from data to an image are described. In section 3, object detection and tracking algorithm is introduced. Section 4 shows the experimental detection and tracking results using the proposed algorithm. The tracked results of ships which equipped AIS are compared with AIS information. Through the comparison, the detection and tracking rates are calculated. Finally, conclusions are presented in section 5.

2. DATA ACQUISITION

2.1 Experiment Design

To detecting and tracking ships, system was installed with main equipment which is radar and AIS. Figure 1 shows configuration of image acquisition system. The radar provides raw data (spoke data) containing the sampling of a particular angle. It contains all the information for reconstructing the exact position where data were sampled. Also we obtained ship’s position information using AIS.

The generator provides power (12V) to radar interface box, AIS receiver, heading sensor and SimNet converter. Then the device is working and sending information into PC simultaneously. In proposed system, image sequences formed by marine radar (Broadband 3G radar) whose main characteristics are summarized in Table 1.

We installed FMCW radar and AIS in the Pyeong-Taek port where is suitable to monitor incoming and outgoing ships (36.9764°N, 126.7788°E) as shown in Figure 2. Environment of the experiment are summarized in Table 2. The wind blew slightly and wave was calm. Radar range was configured as 3/4 nm (≒1,389 m) but the actual measured distances of radar is 2,496 m. So we use actual measured distance for exact distance data. When we set this range, radar covers the ship’s route because the distance that between radar and opposite land is 2,252 m. Rotation was 24 rpm but interval of image is 5 sec because it creates an image by twice rotation of radar antenna.

Ship’s dimensions which have been observed during the experiment time are summarized in Table 3. These were result of the experiment during ten minutes from 10:05 to 10:15. The five ships were detected, three ships of total ships are equipped AIS and the rest two ships are fishing ships. Therefore, AIS equipped ships are able to get information of ships name, type and dimensions. The three ships which equipped AIS (Hwang Ryong 202, Jule, and In Hwang No.8) are tracked and then they are compared with position information of AIS. In general, fishing boat length is less than 15 m, breadth is less than 5 m, and material is Fiber Reinforced Plastics (FRP).

Fig. 1.

Configuration of image acquisition system using FMCW radar and AIS. The raw data (spoke data) from radar can be delivered to PC through radar interface box. Ship’s position information can be obtained through the AIS receiver.

Table 1.

Main characteristics of the FMCW radar

Table 2.

Experimental conditions

Fig. 2.

Research area and installation site (36.9764°N, 126.7788°E). The Port of Pyeong-Taek is closest to the east China in Korean ports. Pyeong-Taek port is suitable to monitor incoming and outgoing ships.

Table 3.

List of ships monitored during the field experiment

3. METHODS

Our approach is to estimate the motion of target. In order to do so, we should select the target under the irregularly appeared errors unlike video camera that extracts moving objects from the fixed background. The most important thing in this algorithm is self-identification between the targets and errors before tracking the targets. First, targets are extracted through the image filtering. Image filtering process was focused to eliminate the device noise in order to distinguish the ships. A noise was removed using morphology and difference image that is congruent with that used for image filtering processing. And then the selected target will be tracked through the proposed methods.

3.1. Image Filtering for Detection

Target can be extracted after filtering and noise suppression process of the received radar image. In radar image, no signal that corresponds to sample value of 0 appears as dark area, while object and noise is bright region (Figure 3). Image received from the radar has useless noise information. Thus, irregular noise of image data should be removed because it affects to the result in the processing steps.

Figure 3 shows the steps of image filtering to remove the noise. First of all, we can remove the land area. It is possible to know with the naked eye where the land is, because we already have information about experiment area. And then ocean part is marked by creating the boundaries of the ocean and land (b). Second, the objects region and holes are filled (c). In this process, only values over the threshold are extracted and fill the voids between extracted values. Third, morphological transformations such as dilating and eroding are used for noise reduction, component combined or separation (d)~(e). Both long axis is much longer than short axis and small area are filtered for eliminate the errors caused in the morphology processing (f). And then finally, only duplicated objects are extracted by compare to next image. This process plays an important role to delete mechanical error.

Once a ship is detected, all of position information in pixel are converted to a geographic coordinate. The center point of all the values is calculated after converted into the coordinate. That calculated value is used for tracking ships.

Fig. 3.

Sample of the filtering steps: (a) initial image of 3th frame at 10:05, (b) separation of ocean and land, (c)~(e) morphology processing, (f) axis & area filtering, (g) difference image.

3.2. Tracking Methods

For estimate the motion between two consecutive frames, search-window algorithm was used. Tracking method was developed and Figure 4 shows the example. This simple algorithm has an advantage of being easy implementation. And it is good at real-time tracking because it does not need a large amount of computation. In this experiment, the interval time is very short as 5 seconds, so this simple method is possible to use to track. It estimates the motion by creating a search-window that is ship’s expected rectangle area in the next frame. The r1 and r2 are 1.5 times of the value that was calculated considering expected maximum speed (19 kts) and interval time (5 sec) in experiment area. The frame in the time t_n configures the search-window and the target in next frame at t_n+1 are compared. If the target which at t_n+1 exists in search-window which made at t_n, that recognized such as the same target at t_n.

Fig. 4.

The example of a search-window tracking. The goal here is to find the same target on the next frame (tn+1) as the target on the current frame (tn). We define a search-window size as r1 and r2 from a center of target in the current frame. The target which is entering a search-window in the next frame is recognized as the matched target.

In the first frame, tracking begins after all of center points are recognized as initial targets. So when the new ships enter the radar range, tracking is impossible. Therefore, it should be considered about ships as an instance. In case of this radar system, a ship cannot suddenly appear in the inside of radar network. A ship was detected newly around the maximum radar range. In this way, the tracking method for the new ship was applied automatically. When this tracking method applied to the ship when it moves at a constant speed with a directional, targets are simply tracked.

Fig. 5.

The display of tracking performance results. Buoys were drawn as a triangle symbol. White dots and 5 different circles represent the detected and tracked ships for each target, respectively. This data was collected for a ten-minute from 10:05:00 to 10:15:55 on February 14, 2012.

Table 4

Detection and tracking performance for ships and bu

4. RESULTS

The following section details results of horizontal radar image detection and tracks. First of all, the ships and buoys were detected and tracked during the ten-minutes using the proposed algorithm. Figure 5 shows the display of tracking performance results. In this figure, ships and buoys which shown as a white dot obtained through the detection algorithm. The five different circles represent the tracked ships for each target and all of ships tracked.

To validate result of the ship tracking, trajectory of ships was compared with position information of AIS data as shown in Figure 6. There are three ships (Hwang Ryong 202, Jule, and In Hwang No.8) which equipped AIS during the experiment time. Each of ships displayed by the square symbols while position information of AIS displayed by solid line. AIS operation time was ten-minute (10:05-10:15) and solid line is a trajectory. The data of Hwang Ryong 202, Jule, and In Hwang No.8 were obtained during 6 minute and 35 seconds (81 frames), 30 seconds (30 frames), and 4 minute and 40 seconds (56 frames) respectively. The reason of the difference between AIS and tracking results is that a position of tracked ship represents the ship centroid, while AIS means the AIS position. Information represent the center of ships, while AIS position information do not shows the center of ships.

Table 4 shows detection and tracking rate which are calculated after making sure tracking results was stable. The frame number was defined as the number of frames that each ship exists. The detection rate was defined as the number of appeared objects over the total number of frames. And the tracking rate was defined as the number of tracked objects over the number of detected objects. In case of Hwang Ryong 202, Jule, and In Hwang No.8 were detected and tracked perfectly. These three ships are large vessels which equipped AIS, and they never overlap each other, so it was easier to use this algorithm. Ship 3 and 4 are small fishing boat. In case of ship 3, initial tracking (~77_th frame) was very well but failed nearing the light-house because it did not separated between ship and light-house. So the detection rate was 73.84% and the tracking rate was 80.20%. The low detection rate for fishing vessels mainly occurred when the ships are close to a mooring buoy. Other reasons are that the ship is made of FRP and its height is low. In case of ship 4, each of rates is 96.92%, 95.38%. Light-house and buoys are also detected. But the nature of this radar, buoy behind the other buoy detection rate was lower because foremost buoy impede a transmission of radio waves. So the detection rate of behind buoy (buoy 1) is low. However all of ships tracked very accurately. It can be seen that the fishing boat can also be tracked very well, except separation between boat and light-house.

Fig. 6.

Comparison of AIS data and trajectory of ships which equipped AIS. Solid line and square represent the ship’s trajectory of AIS and tracking result, respectively. Hwang Ryong 202 is obtained during 6 minute and 35 seconds (81 frames), Jule for 2 minute and 30 seconds (30 frames), and In Hwang No.8 for 4 minute and 40 seconds (56 frames).

5. CONCLUSIONS

We presented target detection and tracking algorithm applied to FMCW radar image. This simple algorithm is developed for image filtering and tracking. The objects are extracted through clearing the error using proposed filtering algorithm. This algorithm can efficiently extract objects. The moving objects are first identified on the first frame. For estimate the motion between two consecutive frames, search-window algorithm was used. If the target at the next frame exists in the search-window, the target are recognized such as the same target with previous target. To validate the ship tracking algorithm, trajectory of ships was compared with position information of AIS data.

The proposed detection and tracking algorithm can be processed simply and quickly. Our tracking method is most useful when the targets do not overlap each other. AIS and tracking results are slightly different. Because AIS means the AIS position, while tracked ship represents the ship centroid. So position of ships is not perfect but all of ships tracked. So it can be usefully applied in other research areas.

Acknowledgments

This work was supported in part by a grant from the Project “Development of Korea Operational Oceanographic System(PM57041)” funded by the Ministry of Land, Transport and Maritime Affairs of Korean Government and the Research Projects(PE98911, PG47770 and PNS184C) of the Korea Institute Ocean Science and Technology.

References

• B Benfold, I Reid, “Stable Multi-Target Tracking in Real-Time Surveillance Video”, Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, (2011, June), p3457-3464. [https://doi.org/10.1109/CVPR.2011.5995667]
• S.K Chaturvedi, C.S Yang, K Ouchi, P Shanmugam, “Ship Recognition by Integration of SAR and AIS”, The Journal of Navigation, (2012, April), 65(2), p323-337. [https://doi.org/10.1017/S0373463311000749]
• S Corda, G Bell, “NavRadar NRP Class Description”, Technical Paper of Navico Auckland Limited, (2009, December).
• S Corda, “PPI Class Description”, Technical Paper of Navico Auckland Limited, (2009, December).
• EMSA, EMSA Annual Report, (2011).
• M Guerriero, P Willett, S Coraluppi, C Carthel, “Radar/AIS Data Fusion and SAR Tasking for Maritime Surveillance”,, Information Fusionk, 2008 11th International Conference on, (2008, July), p1-5.
• R Hoseinnezhad, B.N Vo, B.T Vo, D Suter, “Visual Tracking of Numerous Targets via Multi-Bernoulli Filtering of Image Data”, Pattern Recognition, (2012, October), 45(10), p3625-3635. [https://doi.org/10.1016/j.patcog.2012.04.004]
• S.K Hwang, IT EXPERT Programing of Image Processing by Visual C++, Hanbit media, Inc, (2007), p778-779.
• F.R Inacio, A Raybaud, “Multispectral Target Detection and Tracking for Seaport Video Surveillance”, Proceedings of the Image and Vision Computing New Zealand, (2007, December), p169-174.
• R Khan, B Gamberg, D Power, J Walsh, B Dawe, W Pearson, D Millan, “Target Detection and Tracking with a High Frequency Ground Wave Radar”, Oceanic Engineering, IEEE Journal of, (1994, October), 17(4), p540-548. [https://doi.org/10.1109/48.338390]
• K.H Kim, K.E Ko, J.S Kang, K.B Sim, “Specified Object Tracking in an Environment of Multiple Moving Object using Particle Filter”, Journal of Korean Institute of Intelligent Systems, (2011, February), 21, p106-111. [https://doi.org/10.5391/JKIIS.2011.21.1.106]
• D.J Lee, “Real-time Position Tracking of Traffic Ships by ARPA Radar and AIS in Busan Harbor, Korea”, The Korean Society of Fisheries Technology, (2008,, August), 44(3), p229-238. [https://doi.org/10.3796/KSFT.2008.44.3.229]
• A Mecocci, G Benelli, A Garzelli, S Bottalico, “Radar Image Processing for Ship-Traffic Control”, Image and Vision Computing, (1995, March), 13(2), p119-128. [https://doi.org/10.1016/0262-8856(95)93153-J]
• S.M Mun, J.Y Son, “Current Activities of Navigation & Communication Equipment Industry and R&D”, Journal of the Korean society of marine engineering, (2011, May), 35(4), p512-518. [https://doi.org/10.5916/jkosme.2011.35.4.512]
• L Meng, J.P Kerekes, “Object Tracking Using High Resolution Satellite Imagery”, Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of, (2012, February), 5(1), p146-152. [https://doi.org/10.1109/JSTARS.2011.2179639]
• M Seibert, B.J Rhodes, N.A Bomberger, P.O Beane, J.J Srokal, “SeeCoast Port Surveillance”, (2006, April), Proceedings of SPIE Vol. 6204: Photonics for Port and Harbor Security II. [https://doi.org/10.1117/12.666980]
• A.G Stove, “Linear FMCW Radar Technique”, Radar and Signal Processing IEE PROCEEDINGS-F, (1992, October), 139(5), p343-350. [https://doi.org/10.1049/ip-f-2.1992.0048]
• M Yang, Context-aware and Attentional Visual Object Tracking, Ph.D. Dissertation, Northwestern University, (2008).