short-paper
Authors: Hongxiang Lv, Hai Zhu, Mengmeng Xu, Enlai Dong, Fei Wu, Yueyue Fan
UAVM '24: Proceedings of the 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective
Pages 4 - 8
Published: 28 October 2024 Publication History
Metrics
Total Citations0Total Downloads0Last 12 Months0
Last 6 weeks0
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
Get Access
- Get Access
- References
- Media
- Tables
- Share
Abstract
Cross-view geo-localization is a task of matching the same geographic image from differerent views, e.g., drone and satellite. Due to its GPS-free advantage, cross-view geo-localization is gaining increasing research interest, especially in drone-based localization and navigation applications. In order to guarantee system robustness, existing methods mainly focused on image augmentation and denoising, while facing performace degradation when extreme weather considered. In this paper, we propose an end-to-end image retrieval framework, WELN. By integrating the advanced EVA02 netwotk and LPN algorithm, WELN can extract valuable classification features more efficiently even under extreme weather conditions. Additionally, to enhance model robustness, we expand the University-1652 dataset with nine different weather conditions added. Our method achieves state-of-the-art Recall@1 accuracy on University-1652 dataset, with 92.87% for drone-view target localization task and 93.46% for drone navigation task. Besides, we gain the fourth place in the ACMMM24 Multimedia Drone-Satellite Matching Challenge. Our code will be open sourced at https://github.com/koorter/WELN.
References
[1]
Hongji Yang, Xiufan Lu, and Ying J. Zhu. 2021. Cross-view Geo-localization with Layer-to-Layer Transformer. Neural Information Processing Systems.
[2]
Niluthpol Chowdhury Mithun, Kshitij Minhas, Han-Pang Chiu, Taragay Oskiper, Mikhail Sizintsev, Supun Samarasekera, and Rakesh Kumar. 2023. Cross-View Visual Geo-Localization for Outdoor Augmented Reality. 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR) (2023), 493--502.
[3]
Wen Chen, Chenglong Zhang, Yu Peng, Yao Yao, Miaomiao Cai, and Danan Dong. 2024. Enhancing GNSS Positioning in Urban Canyon Areas via a Modified Design Matrix Approach. IEEE Internet of Things Journal, Vol. 11, 6 (2024), 10252--10265.
[4]
Mu Chen, Zhedong Zheng, and Yi Yang. 2023. Transferring to Real-World Layouts: A Depth-aware Framework for Scene Adaptation. arxiv: 2311.12682 [cs.CV]
[5]
Zhedong Zheng, Yujiao Shi, Tingyu Wang, Chen Chen, Pengfei Zhu, and Richard Hartley. 2024. The 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective. In Proceedings of the 32nd ACM International Conference on Multimedia Workshop.
[6]
Depeng Li and Zhigang Zeng. 2023. CRNet: A Fast Continual Learning Framework With Random Theory. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 9 (2023), 10731--10744.
[7]
Yuning Cui and Alois Knoll. 2024. Dual-domain strip attention for image restoration. Neural Networks, Vol. 171 (2024), 429--439.
Digital Library
[8]
Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. EVA-02: A Visual Representation for Neon Genesis. ArXiv, Vol. abs/2303.11331 (2023).
[9]
Tingyu Wang, Zhedong Zheng, Chenggang Yan, Jiyong Zhang, Yaoqi Sun, Bolun Zheng, and Yi Yang. 2022. Each Part Matters: Local Patterns Facilitate Cross-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 2 (2022), 867--879.
[10]
Zhedong Zheng, Yunchao Wei, and Yi Yang. 2020. University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization. Association for Computing Machinery.
[11]
Tingyu Wang, Zhedong Zheng, Zunjie Zhu, Yaoqi Sun, Yi Yang, and Chenggang Yan. 2022. Learning cross-view geo-localization embeddings via dynamic weighted decorrelation regularization. arXiv preprint arXiv:2211.05296 (2022).
[12]
Sijie Zhu, Mubarak Shah, and Chen Chen. 2022. TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 1152--1161.
[13]
Yingying Zhu, Hongji Yang, Yuxin Lu, and Qiang Huang. 2023. Simple, Effective and General: A New Backbone for Cross-view Image Geo-localization. ArXiv, Vol. abs/2302.01572 (2023).
[14]
Yicong Tian, Chen Chen, and Mubarak Shah. 2017. Cross-View Image Matching for Geo-Localization in Urban Environments. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1998--2006.
[15]
Mu Chen, Zhedong Zheng, Yi Yang, and Tat-Seng Chua. 2022. PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation. arxiv: 2211.07609 [cs.CV]
[16]
Tingyu Wang, Zhedong Zheng, Yaoqi Sun, Chenggang Yan, Yi Yang, and Tat-Seng Chua. 2024. Multiple-environment Self-adaptive Network for aerial-view geo-localization. Pattern Recognition, Vol. 152 (2024), 110363.
Digital Library
[17]
Wei-Ting Chen, Zhi-Kai Huang, Cheng-Che Tsai, Hao-Hsiang Yang, Jian-Jiun Ding, and Sy-Yen Kuo. 2022. Learning Multiple Adverse Weather Removal via Two-stage Knowledge Learning and Multi-contrastive Regularization: Toward a Unified Model. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 17632--17641.
[18]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv, Vol. abs/2010.11929 (2020).
[19]
Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. EVA: Exploring the Limits of Masked Visual Representation Learning at Scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 19358--19369.
[20]
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the Knowledge in a Neural Network. In NIPS Deep Learning and Representation Learning Workshop.
[21]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning, Vol. 139. 8748--8763.
[22]
Hongxiang Lv, Hai Zhu, Runzhe Zhu, Fei Wu, Chunyuan Wang, Meiyu Cai, and Kaiyu Zhang. 2024. Direction-Guided Multiscale Feature Fusion Network for Geo-Localization. IEEE Transactions on Geoscience and Remote Sensing, Vol. 62 (2024), 1--13.
[23]
Yireng Chen, Zihao Yang, and Quan Chen. 2023. A Cross-View Matching Method Based on Dense Partition Strategy for UAV Geolocalization. In Proceedings of the 31st ACM International Conference on Multimedia Workshop (UAVM '23). Association for Computing Machinery, 19--23.
Digital Library
[24]
Leyi Dong, Yuhui Wang, Junshi Huang, Xueming Qian, Mingyuan Fan, and Shenqi Lai. 2023. Dual Path Network for Cross-view Geo-Localization. In Proceedings of the 31st ACM International Conference on Multimedia Workshop (UAVM '23). Association for Computing Machinery, 45--49.
Digital Library
[25]
Ming Dai, Jianhong Hu, Jiedong Zhuang, and Enhui Zheng. 2022. A Transformer-Based Feature Segmentation and Region Alignment Method for UAV-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, 7 (2022), 4376--4389.
Digital Library
[26]
Duc Viet Bui, Masao Kubo, and Hiroshi Sato. 2022. A Part-aware Attention Neural Network for Cross-view Geo-localization between UAV and Satellite. Journal of Robotics, Networking and Artificial Life, Vol. 9, 3 (2022), 275--284.
[27]
Runzhe Zhu, Mingze Yang, Ling Yin, Fei Wu, and Yuncheng Yang. 2023. UAV's Status Is Worth Considering: A Fusion Representations Matching Method for Geo-Localization. Sensors, Vol. 23, 2 (2023).
[28]
Tianrui Shen, Yingmei Wei, Lai Kang, Shanshan Wan, and Yee-Hong Yang. 2024. MCCG: A ConvNeXt-Based Multiple-Classifier Method for Cross-View Geo-Localization. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 34, 3 (2024), 1456--1468.
Digital Library
Index Terms
WELN: Siamese Network-based Framework for Geo-localization in Extreme Weather
Software and its engineering
Recommendations
- WAGL: Extreme Weather Adaptive Method for Robust and Generalizable UAV-based Cross-View Geo-localization
UAVM '24: Proceedings of the 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective
As drones become increasingly utilized across various fields, related multimedia applications are also emerging. One significant application is cross-view geo-localization, which leverages aerial drone and satellite imagery data to facilitate drone ...
Read More
- University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization
MM '20: Proceedings of the 28th ACM International Conference on Multimedia
We consider the problem of cross-view geo-localization. The primary challenge is to learn the robust feature against large viewpoint changes. Existing benchmarks can help, but are limited in the number of viewpoints. Image pairs, containing two ...
Read More
- Image and Object Geo-Localization
Abstract
The concept of geo-localization broadly refers to the process of determining an entity’s geographical location, typically in the form of Global Positioning System (GPS) coordinates. The entity of interest may be an image, a sequence of images, a ...
Read More
Comments
Information & Contributors
Information
Published In
UAVM '24: Proceedings of the 2nd Workshop on UAVs in Multimedia: Capturing the World from a New Perspective
October 2024
41 pages
ISBN:9798400712067
DOI:10.1145/3689095
- General Chairs:
- Zhedong Zheng
University of Macau, China
, - Yujiao Shi
ShanghaiTech University, China
, - Tingyu Wang
Hangzhou Dianzi University, China
, - Chen Chen
University of Central Florida, USA
, - Pengfei Zhu
Tianjin University, China
, - Richard Hartley
Australian National University, Australia
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [emailprotected].
Sponsors
- SIGMM: ACM Special Interest Group on Multimedia
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Permissions
Request permissions for this article.
Check for updates
Author Tags
- drone-satellite matching
- features
- geo-localization
- weather noise
Qualifiers
- Short-paper
Funding Sources
Conference
MM '24
Sponsor:
- SIGMM
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Contributors
Other Metrics
View Article Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
Total Citations
Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Oct 2024
Other Metrics
View Author Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in
Full Access
Get this Publication
View options
Media
Figures
Other
Tables
