Shu Kong

Assistant Professor; University of Macau [GitHub] [Google Scholar]
Home Publication Group Contact

Papers


UAL-Bench: The First Comprehensive Unusual Activity Localization Benchmark
H. M. Abdullah, T. Liu, K. Wei, S. Kong, R. Huang
In WACV, 2025
[paper] [code] [data]

Revisiting Few-Shot Object Detection with Vision-Language Models
A. Madan*, N. Peri*, S. Kong#, D. Ramanan#
In NeurIPS Datasets and Benchmarks, 2024
[paper] [code] [challenge]

Reconstructing the diversity dynamics of paleo-grasslands using deep learning on superresolution images of fossil Poaceae pollen
MÉ Adaimé, S. Kong, M. Urban, FA Street-Perrott, D. Verschuren, SW. Punyasena
preprint at bioRxiv, 2024
[paper] [code]

Long-Tailed 3D Detection via Multi-Modal Late-Fusion
Y. Ma*, N. Peri*, S. Wei, A. Dave, W. Hua, Y. Li, D. Ramanan, S. Kong
arXiv:2312.10986, 2024
[paper]

Lidar Panoptic Segmentation in an Open World
A. S. Chakravarthy, M. R. Ganesina, P. Hu, L. Leal-Taixé, S. Kong, D. Ramanan, A. Osep
International Journal of Computer Vision (IJCV), 2024
[video demo] [paper] [code] [webpage]

CriSp: Leveraging Tread Depth Maps for Enhanced Crime-Scene Shoeprint Matching
S. Shafique, S. Kong*, C. Fowlkes*
In ECCV, 2024
[webpage] [paper] [dataset] [code]

Improving Knowledge Distillation via Regularizing Feature Norm and Direction
Y. Wang, L. Cheng, M. Duan, Y. Wang, Z. Feng, S. Kong
In ECCV, 2024 (oral presentation)
[paper] [github]

Few-Shot Recognition via Stage-Wise Augmented Finetuning
T. Liu, H. Zhang, S. Parashar, S. Kong
arXiv:2406.11148, 2024
[website] [paer] [code]

LCA-on-the-Line: Benchmarking Out of Distribution Generalization with Class Taxonomies
J. Shi, G. Gare, J. Tian, S. Chai, Z. Lin, A. Vasudevan, D. Feng, F. Ferroni, S. Kong
In ICML, 2024 (oral presentation)
[webpage] [paper] [code] [poster] [slides] [video]

Towards Unstructured Unlabeled Optical Mocap: A Video Helps!
N. Milef, J. Keyser, S. Kong
In SIGGRAPH, 2024
[website] [video demo] [code] [paper] [data]

The Neglected Tails of Vision-Language Models
S. Parashar*, Z. Lin*, T. Liu*, X. Dong, Y. Li, D. Ramanan, J. Caverlee, S. Kong
In CVPR, 2024
[webpage] [paper] [code]

Instance Tracking in 3D Scenes from Egocentric Videos
Y. Zhao, H. Ma, S. Kong, C. Fowlkes
In CVPR, 2024
[paper] [code] [data]

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want
Z. Sun*, Y. Fang*, T. Wu, P. Zhang, Y. Zang, S. Kong, Y. Xiong, D. Lin, J. Wang
In CVPR, 2024
[paper] [code] [webpage]

Boosting Image Restoration via Priors from Pre-trained Models
X. Xu, S. Kong, T. Hu, Z. Liu, H. Bao
In CVPR, 2024
[paper]

AccessLens: Auto-detecting Inaccessibility of Everyday Objects
N. Kwon, Q. Lu, M. H. Qazi, J. Liu, C. Oh, S. Kong, J. Kim
In CHI, 2024
[paper] [video] [website]

Deep Learning Approaches to the Phylogenetic Placement of Extinct Pollen Morphotypes
MÉ Adaime, S. Kong, S. Punyasena
Proceedings of the National Academy of Sciences (PNAS) Nexus, 2024
[paper] [News] [news]

Roadside Monocular 3D Detection via 2D Detection Prompting
Y. Ma, S. Wei, C. Zhang, W. Hua, Y. Li, S. Kong
arXiv:2404.01064, 2024
[paper]

Prompting Scientific Names for Zero-Shot Species Recognition
S. Parashar, Z. Lin, Y. Li, S. Kong
In EMNLP, 2023
[paper]

A High-Resolution Dataset for Instance Detection with Multi-View Instance Capture
Q. Shen*, Y. Zhao*, N. Kwon, J. Kim, Y. Li, S. Kong
In NeurIPS Datasets and Benchmarks, 2023
[github] [paper] [dataset]

OV-PARTS: Towards Open-Vocabulary Part Segmentation
M. Wei, X. Yue, W. Zhang , S. Kong, X. Liu, J. Pang
In NeurIPS Datasets and Benchmarks, 2023
[github] [paper]

Creating a Forensic Database of Shoeprints from Online Shoe Tread Photos
S. Shafique, B. Kong, S. Kong*, C. Fowlkes*
In WACV 2023
[github] [paper]

Far3Det: Towards Far-Field 3D Detection
S. Gupta, J. Kanjani, M. Li, F. Ferroni, J. Hays, D. Ramanan S. Kong
In WACV 2023
[github] [paper]

Continual Learning With an Evolving Class Ontology
Z. Lin, D. Pathak, Y. Wang, D. Ramanan S. Kong
In NeurIPS 2022
[webpage] [paper]

Towards Long Tailed 3D Detection
N. Peri, A. Dave, D. Ramanan, S. Kong
In CoRL 2022
[webpage] [paper]

Multimodal Object Detection via Probabilistic Ensembling
Y-T Chen*, J. Shi*, Z. Ye*, C. Mertz, D. Ramanan, S. Kong
In ECCV 2022 (oral presentation)
[webpage] [paper] [github] [video demo]

OpenGAN: Open-Set Recognition via Open Data Generation
S. Kong, D. Ramanan
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2022
[webpage] [paper] [pdf (18MB)] [code] [poster] [slides] [watch 12min video presentation]
Journal version of ICCV 2021 paper (Marr Prize / Best Paper Honorable Mention)

Automated Identification of Diverse Neotropical Pollen Samples using Convolutional Neural Networks
S. Punyasena*, D. Haselhorst*, S. Kong*, C. Fowlkes, J. Moreno
Methods in Ecology and Evolution, 2022
[paper] [webpage] [code]

Long-Tailed Recognition via Weight Balancing
S. Alshammari, Y. Wang, D. Ramanan, S Kong
In CVPR, 2022
[github] [paper]

OpenGAN: Open-Set Recognition via Open Data Generation
S. Kong, D. Ramanan
In ICCV, 2021 (Marr Prize / Best Paper Honorable Mention)
[webpage] [paper] [code] [poster] [slides] [watch 12min video presentation]
See journal version published by PAMI [pdf (18MB)]

Camera Pose Matters: Improving Depth Prediction by Mitigating Pose Distribution Bias
Y. Zhao, S. Kong, C. Fowlkes
In CVPR, 2021 (oral presentation)
[webpage] [arxiv] [github] [slides]

Improving the Taxonomy of Fossil Pollen using Convolutional Neural Networks and Superresolution Microscopy
I. Romero, S. Kong, C. Fowlkes, C. Jaramillo, M. Urban, F. Oboh-Ikuenobe, C. D'Apolito, S. Punyasena
Proceedings of the National Academy of Sciences of the USA (PNAS), 2020 (Featured by NSF news)
[paper] [code]

Domain Decluttering: Simplifying Images to Mitigate Synthetic-Real Domain Shift and Improve Depth Estimation
Y. Zhao, S. Kong, D. Shin, C. Fowlkes,
In CVPR, 2020
[webpage] [arxiv] [slides] [github]

Celeganser: Automated Analysis of Nematode Morphology and Age
Linfeng Wang, S. Kong, Zachary Pincus, C. Fowlkes,
In CVMI@CVPR, 2020
[webpage] [arxiv] [github]

Fine-Grained Facial Expression Analysis Using Dimensional Emotion Model
F. Zhou*, S. Kong*, C. Fowlkes, T. Chen, B. Lei
Neurocomputing, 2020
[github] [arxiv] [demo] [models]

Weakly-Supervised Temporal-Language Association with Referring Attention
Z. Fang, S. Kong, Z. Wang, C. Fowlkes, Y. Yang
arXiv:2006.11747, 2020
[arxiv]

Modularized Textual Grounding for Counterfactual Resilience
Z. Fang, S. Kong, C. Fowlkes, Y. Yang
In CVPR, 2019
[paper] [github]

Pixel-wise Attentional Gating for Scene Parsing
S. Kong, C. Fowlkes
in WACV, 2019
[arxiv] [github] [slides] [ROB Entry of Depth Est.] [ROB Entry of Segm.]

Multigrid Predictive Filter Flow for Unsupervised Learning on Videos
S. Kong, C. Fowlkes
arXiv:1904.01693, 2019
[webpage] [arxiv] [github] [demo] [slides]

Recurrent Pixel Embedding for Instance Grouping
S. Kong, C. Fowlkes
In CVPR, 2018 (spotlight presentation)
[arxiv] [models] [github] [poster] [slides]

Recurrent Scene Parsing with Perspective Understanding in the Loop
S. Kong, C. Fowlkes
In CVPR, 2018
[paper] [github]

Image Reconstruction with Predictive Filter Flow
S. Kong, C. Fowlkes
arXiv:1811.11482, 2018.
[paper] [webpage] [high-res paper (44MB)] [github]

Low-rank Bilinear Pooling for Fine-Grained Classification
S. Kong, C. Fowlkes
In CVPR, 2017.
[paper] [github]

Photo Aesthetics Ranking Network with Attributes and Content Adaptation
S. Kong, X. Shen, Z. Lin, R. Mech, C. Fowlkes
In ECCV, 2016.
[paper] [github] [AMT instruction] [patent filed]

Spatially Aware Dictionary Learning and Coding for Fossil Pollen Identification
S. Kong, S. Punyasena, C. Fowlkes
In CVMI@CVPR, 2016
[github] [paper]

Modeling Neuron Selectivity over Simple Mid-Level Features for Image Classification
S. Kong, Z. Jiang, Q. Yang
IEEE Transactions on Image Processing, 2015.
[paper]

Saliency Detection within a Deep Convolutional Architecture
Y. Lin, S. Kong, D. Wang, Y. Zhuang
In AAAI'14 Workshop on Cognitive Computing for Augmented Human Intelligence, 2014
[paper]

A Classification-Oriented Dictionary Learning Model: Explicitly Learning the Particularity and Commonality Across Categories
D. Wang*, S. Kong*
Pattern Recognition, 2013
[paper] [code]

Learning Exemplar-Represented Manifolds in Latent Space for Classification
S. Kong, D. Wang
In ECML/PKDD, 2013
[paper] [code]

Integration of Multi-Feature Fusion and Dictionary Learning for Face Recognition
D. Wang, X. Wang, S. Kong
Image and Vision Computing (IVC), 2013
[paper] [code]

Learning Individual-Specific Dictionaries with Fused Multiple Features for Face Recognition
S. Kong, D. Wang
In IEEE conference series on Automatic Face and Gesture Recognition (FG), 2013
[paper]

Multiple Feature Fusion for Face Recognition
S. Kong, X. Wang, D. Wang, F. Wu
In IEEE conference series on Automatic Face and Gesture Recognition (FG), 2013
[paper] [code]

A Dictionary Learning Approach for Classification: Separating the Particularity and the commonality
S. Kong, D. Wang
In ECCV, 2012
[paper] [code]

A Multi-task Learning Strategy for Unsupervised Clustering via Explicitly Separating the Commonality
S. Kong, D. Wang
In ICPR, 2012
[paper]

Transfer heterogeneous unlabeled data for unsupervised clustering
S. Kong, D. Wang
In ICPR, 2012
[paper]

Learning class-specific dictionaries for digit recognition from spherical surface of a 3D ball
D. Wang, S. Kong
Machine Vision and Applications (MVA), 2012
[paper] [SingleBall_dataset (288MB)] [MultiBall_dataset (121MB)]

Feature Selection from High-Order Tensorial Data via Sparse Decomposition
D. Wang, S. Kong
Pattern Recognition Letters, 2012
[paper] [code]

Abstract/Workshop Papers

  • S. Parashar, Z. Lin, T. Liu, X. Dong, Y. Li, D. Ramanan, J. Caverlee, and S. Kong, "The Neglected Tails in Vision-Language Models", ICML 2024 Workshop on Data-centric Machine Learning Research (DMLR): Datasets for Foundation Models, Vienna, Austria; July 2024.

  • S. W. Punyasena, M. É. Adaïmé, and S. Kong, "Deep learning approaches to the phylogenetic placement of fossil pollen morphotypes", 15th International Palynological Congress and 11th International Organization of Palaeobotany Conference, Prague, Czech Republic; May 2024.

  • S. W. Punyasena, J. T. Feng, S. Puthanveetil Satheesan, and S. Kong, "Development of a high-throughput fossil pollen analysis pipeline", North American Paleontological Convention 2024, Ann Arbor, MI; June 2024.

  • M. É. Adaïmé, S. Kong, M. A. Urban, and S. W. Punyasena, "Reconstructing the diversity dynamics of Late Quaternary East African grasslands using superresolution imaging of fossil Poaceae pollen and deep learning", North American Paleontological Convention 2024, Ann Arbor, MI; June 2024.

  • M. É. Adaïmé, S. Kong, and S. W. Punyasena, "Using machine learning and superresolution imaging of fossil Poaceae to reconstruct the biodiversity dynamics of Late Quaternary East African grasslands", 41st Annual Midcontinental Paleobotany Meeting, Field Museum, Chicago, IL; April 2024.

  • B. Lloyd, M. É Adaïmé, S. W. Punyasena, T. Gallaher, S. Kong, and C. E. Strömberg, "A deep learning approach to the taxonomic classification of grass silica short cell phytoliths", 41st Annual Midcontinental Paleobotany Meeting, Field Museum, Chicago, IL; April 2024.

  • Jia Shi, Gautam Gare, Jinjin Tian, Siqi Chai, Zhiqiu Lin, Arun Vasudevan, Di Feng, Francesco Ferroni, Shu Kong, Deva Ramanan "LCA-on-the-Line: Benchmarking Out of Distribution Generalization with Class Taxonomies", NeurIPS Workshop on DistShift, 2023

  • M. É. Adaïmé, S. Kong, and S. W. Punyasena, "Deep learning approaches to the phylogenetic placement of extinct pollen morphotypes", Geological Society of America GSA Connects (2023 Annual Meeting), Pittsburgh, PA; October 2023.

  • M-E. Adaime, S. Kong, and S.W. Punyasena, "Deep metric learning and the phylogenetic placement of novel fossil pollen", 39th Annual Midcontinental Paleobotany Meeting, Oak Spring Foundation, Upperville, VA; May 2022.

  • M-E. Adaime, S. Kong, and S.W. Punyasena, "Phylogenetically-informed computer vision methods for fossil pollen classification: ecological and evolutionary implications", 38th Annual Midcontinental Paleobotany Meeting, vitural, hosted by University of California Berkley / University of Washington / University of Wyoming; June 2021.

  • M-E. Adaime, S. Kong, and S.W. Punyasena, "Phylogenetically-informed computer vision methods for fossil pollen classification: ecological and evolutionary implications", 38th Annual Midcontinental Paleobotany Meeting, vitural, hosted by University of California Berkley / University of Washington / University of Wyoming; June 2021.

  • Zhiyuan Fang, Shu Kong, Charless Fowlkes, Yezhou Yang " Modularized Textual Grounding for Counterfactual Resilience", Language And Vision workshop joint with CVPR, 2019.

  • I.C. Romero, S. Kong, C.C. Fowlkes, S.W. Punyasena, "Identification of the Cenozoic pollen morphospecies Striatopollis catatumbus (Amherstieae, Fabaceae) using convolutional neural nets", 3rd Annual Digital Data in Biodiversity Research Conference, New Haven, CT; June 2019.

  • Surangi W. Punyasena, Shu Kong, Charless C. Fowlkes "Improving the taxonomic accuracy and precision of fossil pollen identifications", North American Paleontological Convention, Riverside, USA, 2019.

  • Ingrid Romero, Shu Kong, Charless C. Fowlkes, Michael A. Urban, Surangi W. Punyasena, "Automated Neotropical Fossil Pollen Fabaceae Analysis Using Convolutional Neural Networks", GSA Annual Meeting in Indianapolis, Indiana, USA, 2018.

  • Zhiyuan Fang, Shu Kong, Tianshu Yu, Yezhou Yang, "Weakly Supervised Attention Learning for Textual Phrases Grounding", Language and Vision Workshop jointwith CVPR, 2018.

  • Shu Kong, Charless C. Fowlkes, "Low-rank Bilinear Pooling for Fine-Grained Classification", the Fourth Workshop on Fine-grained Visual Categorization joint with CVPR, 2017.

  • Shu Kong, Charless C. Fowlkes, "Recurrent Scene Parsing with Perspective Understanding in the Loop", Southern California Machine Learning Symposium, 2017.

  • I. Romero, S. Kong, C.C. Fowlkes, M.A. Urban, C. D’Apolito, C. Jaramillo, F.E. Oboh-Ikuenobe, F.E., and S.W. Punyasena, "Novel morphological analysis of a fossil Fabaceae pollen type, Striatopollis catatumbus (tribe Detariae)", Geological Society of America Abstracts with Programs (2017 Annual Meeting), doi: 10.1130/abs/2017AM-305572; Seattle, WA; October 2017.

  • I.C. Romero, S. Kong, C.C. Fowlkes, M.A. Urban, C.A. D’Apolito, C. Jaramillo, F. Oboh-Ikuenobe, and S.W. Punyasena, "Cenozoic biogeography of Striatopollis catatumbus (Fabaceae – Detariae)", AASP-The Palynological Society, Nottingham, England; September 2017.

  • Derek S. Haselhorst, Shu Kong, Charless C. Fowlkes, J. Enrique Moreno, David K. Tcheng, Surangi W. Punyasena, "Automating tropical pollen counts using convolutional neural nets: from image acquisition to identification", the iDigBio inaugural conference, Ann Arbor, MI; June 2017.

  • Surangi W. Punyasena, Shu Kong, Charless C. Fowlkes, and Stephen P. Jackson, "Reconstructing the extinction dynamics of Picea critchfieldii - the application of computer vision to fossil pollen analysis ", the iDigBio inaugural conference, Ann Arbor, MI; June 2017.

  • Shu Kong, Charless C. Fowlkes "Low-rank Bilinear Pooling for Fine-Grained Classification", Southern California Machine Learning Symposium, 2016.