3D Geometric Shape Assembly via
Efficient Point Cloud Matching

ICML 2024
Nahyuk Lee* 1     Juhong Min* 1     Junha Lee1     Seungwook Kim1    Kanghee Lee2    Jaesik Park2    Minsu Cho1
1 Pohang University of Science and Engineering (POSTECH)        2 Seoul National University
(* equal contribution)
arXiv Poster Code BibTex

Abstract

Learning to assemble geometric shapes into a larger target structure is a pivotal task in various practical applications. In this work, we tackle this problem by establishing local correspondences between point clouds of part shapes in both coarse- and fine-levels. To this end, we introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable matching between mating surfaces of parts while incurring low costs in memory and computation. Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task. We evaluate the proposed PMTR on the large-scale 3D geometric shape assembly benchmark dataset of Breaking Bad and demonstrate its superior performance and efficiency compared to state-of-the-art methods.

What's this project about?

We introduce a new form of low-complexity high-order feature transform layer, dubbed Proxy Match Transform (PMT), designed to align analogous local embeddings in feature space with sub-quadratic complexity. The PMT layer provides a low-complexity yet high-order approach to geometric shape assembly, effectively approximating conventional high-order feature transforms under particular conditions.

Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task. We incorporate the PMT layer into PMTR, that uses PMTs for both coarse- and fine-level matching steps to establish correspondences on mating surfaces. The proposed architecture comprises four main parts: (1) feature extraction, (2) coarse-level matching, (3) fine-level matching, and (4) transformation prediction.

Efficiency of PMT

To demonstrate the superiority of the proposed PMT, we provide the efficiency comparison between different matchers, e.g., Geometric Transformer by Qin et al. (2022) and Proxy Match Transform (PMT), both during training and inference phases. The results clearly indicates that PMT delivers substantial reductions not only in training/inference time but also in memory requirements. Notably, PMT is approximately ×21.5 more efficient in FLOPS, needs ×3.4 more compact number of parameters, and ×3.28 / ×11.07 less required memory for training/inference phases compared to GeoTr. Such efficiency is crucial, as it facilitates the practical deployment of our fine-level matcher for intricate matching tasks

Experimental Results

BibTeX

@inproceedings{lee2024pmtr,
  author    = {Lee, Nahyuk and Min, Juhong and Lee, Junha and Kim, Seungwook and Lee, Kanghee and Park, Jaesik and Cho, Minsu},
  title     = {3D Geometric Shape Assembly via Efficient Point Cloud Matching},
  booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
  year      = {2024},
}