Muswas | Infrastructureless unmanned aerial vehicle localization

journal articles

1. RAS

   

   LSVL: Large-scale season-invariant visual localization for UAVs

   Jouko Kinnari, Riccardo Renzulli, Francesco Verdoja, and Ville Kyrki

   Robotics and Autonomous Systems, Oct 2023

   Abs arXiv Bib PDF Video

   Localization of autonomous unmanned aerial vehicles (UAVs) relies heavily on Global Navigation Satellite Systems (GNSS), which are susceptible to interference. Especially in security applications, robust localization algorithms independent of GNSS are needed to provide dependable operations of autonomous UAVs also in interfered conditions. Typical non-GNSS visual localization approaches rely on known starting pose, work only on a small-sized map, or require known flight paths before a mission starts. We consider the problem of localization with no information on initial pose or planned flight path. We propose a solution for global visual localization on a map at scale up to 100 km2, based on matching orthoprojected UAV images to satellite imagery using learned season-invariant descriptors. We show that the method is able to determine heading, latitude and longitude of the UAV at 12.6-18.7 m lateral translation error in as few as 23.2-44.4 updates from an uninformed initialization, also in situations of significant seasonal appearance difference (winter-summer) between the UAV image and the map. We evaluate the characteristics of multiple neural network architectures for generating the descriptors, and likelihood estimation methods that are able to provide fast convergence and low localization error. We also evaluate the operation of the algorithm using real UAV data and evaluate running time on a real-time embedded platform. We believe this is the first work that is able to recover the pose of an UAV at this scale and rate of convergence, while allowing significant seasonal difference between camera observations and map.

   @article{202310_kinnari_lsvl,
     title = {{LSVL}: {Large}-scale season-invariant visual localization for 
                      {UAVs}},
     volume = {168},
     doi = {10.1016/j.robot.2023.104497},
     journal = {Robotics and Autonomous Systems},
     author = {Kinnari, Jouko and Renzulli, Riccardo and Verdoja, Francesco and Kyrki, Ville},
     month = oct,
     year = {2023},
   }

2. RA-L

   

   Season-Invariant GNSS-Denied Visual Localization for UAVs

   Jouko Kinnari, Francesco Verdoja, and Ville Kyrki

   IEEE Robotics and Automation Letters, Oct 2022

   2023 Best Paper Award Abs arXiv Bib PDF Code

   Awarded by the IEEE Finland CSS/RAS/SMCS Joint Chapter

   Localization without Global Navigation Satellite Systems (GNSS) is a critical functionality in autonomous operations of unmanned aerial vehicles (UAVs). Vision-based localization on a known map can be an effective solution, but it is burdened by two main problems: places have different appearance depending on weather and season, and the perspective discrepancy between the UAV camera image and the map make matching hard. In this work, we propose a localization solution relying on matching of UAV camera images to georeferenced orthophotos with a trained convolutional neural network model that is invariant to significant seasonal appearance difference (winter-summer) between the camera image and map. We compare the convergence speed and localization accuracy of our solution to six reference methods. The results show major improvements with respect to reference methods, especially under high seasonal variation. We finally demonstrate the ability of the method to successfully localize a real UAV, showing that the proposed method is robust to perspective changes.

   @article{202210_kinnari_season-invariant,
     title = {Season-{Invariant} {GNSS}-{Denied} {Visual} {Localization} for 
                      {UAVs}},
     volume = {7},
     doi = {10.1109/LRA.2022.3191038},
     number = {4},
     journal = {IEEE Robotics and Automation Letters},
     author = {Kinnari, Jouko and Verdoja, Francesco and Kyrki, Ville},
     month = oct,
     year = {2022},
     pages = {10232--10239},
   }

conference articles

1. ICAR

   

   GNSS-denied geolocalization of UAVs by visual matching of onboard camera images with orthophotos

   Jouko Kinnari, Francesco Verdoja, and Ville Kyrki

   In 2021 IEEE Int. Conf. on Advanced Robotics (ICAR), Dec 2021

   Abs arXiv Bib PDF

   Localization of low-cost Unmanned Aerial Vehicles (UAVs) often relies on Global Navigation Satellite Systems (GNSS). GNSS are susceptible to both natural disruptions to radio signal and intentional jamming and spoofing by an adversary. A typical way to provide georeferenced localization without GNSS for small UAVs is to have a downward-facing camera and match camera images to a map. The downward-facing camera adds cost, size, and weight to the UAV platform and the orientation limits its usability for other purposes. In this work, we propose a Monte-Carlo localization method for georeferenced localization of an UAV requiring no infrastructure using only inertial measurements, a camera facing an arbitrary direction, and an orthoimage map. We perform orthorectification of the UAV image, relying on a local planarity assumption of the environment, relaxing the requirement of downward-pointing camera. We propose a measure of goodness for the matching score of an orthorectified UAV image and a map. We demonstrate that the system is able to localize globally an UAV with modest requirements for initialization and map resolution.

   @inproceedings{202112_kinnari_gnss-denied,
     title = {{GNSS}-denied geolocalization of {UAVs} by visual matching of 
                      onboard camera images with orthophotos},
     doi = {10.1109/ICAR53236.2021.9659333},
     booktitle = {2021 {IEEE} {Int.} {Conf.} on {Advanced} {Robotics} ({ICAR})},
     author = {Kinnari, Jouko and Verdoja, Francesco and Kyrki, Ville},
     month = dec,
     year = {2021},
     pages = {555--562},
   }