Neural mesh refinement

Abstract

Subdivision is a widely used technique for mesh refinement. Classic methods rely on fixed manually defined weighting rules and struggle to generate a finer mesh with appropriate details, while advanced neural subdivision methods achieve data-driven nonlinear subdivision but lack robustness, suffering from limited subdivision levels and artifacts on novel shapes. To address these issues, this paper introduces a neural mesh refinement (NMR) method that uses the geometric structural priors learned from fine meshes to adaptively refine coarse meshes through subdivision, demonstrating robust generalization. Our key insight is that it is necessary to disentangle the network from non-structural information such as scale, rotation, and translation, enabling the network to focus on learning and applying the structural priors of local patches for adaptive refinement. For this purpose, we introduce an intrinsic structure descriptor and a locally adaptive neural filter. The intrinsic structure descriptor excludes the non-structural information to align local patches, thereby stabilizing the input feature space and enabling the network to robustly extract structural priors. The proposed neural filter, using a graph attention mechanism, extracts local structural features and adapts learned priors to local patches. Additionally, we observe that Charbonnier loss can alleviate over-smoothing compared to L2 loss. By combining these design choices, our method gains robust geometric learning and locally adaptive capabilities, enhancing generalization to various situations such as unseen shapes and arbitrary refinement levels. We evaluate our method on a diverse set of complex three-dimensional (3D) shapes, and experimental results show that it outperforms existing subdivision methods in terms of geometry quality. See https://zhuzhiwei99.github.io/NeuralMeshRefinement for the project page.

1. Introduction

Three-dimensional (3D) meshes are extensively used in computer graphics and computer vision, playing a vital role in applications such as virtual reality and 3D reconstruction. Although detailed 3D meshes can model complex details, they put a strain on storage and computation. Coarse 3D meshes are favored for efficient storage and quick transmission, but they struggle to accurately represent fine geometric details. To tackle this trade-off, a practical solution is to store and transmit a coarse mesh and then use a refinement method at the user end to restore the details. We study the task of mesh refinement, in which the goal is to refine a coarse triangle mesh into a finer geometric structure with appropriate details.

Classic subdivision methods are highly suitable for interactive mesh refinement. They typically involve two steps: initially, the input mesh elements are divided into smaller ones (e.g., splitting a triangle into four), followed by adjusting the positions of mesh vertices using a weighting scheme based solely on local mesh connectivity. However, these methods fall short of adaptively upsampling a low-resolution mesh. Without user guidance, classic methods tend to overly smooth the entire shape, leading to shrinkage, or amplify tessellation artifacts (Fig. 1). Classic methods based on fixed “one-size-fits-all” weighting rules are determined for their general convergence, and they fail to identify the details that need to be preserved or enhanced during upsampling.

Nonlinear subdivision schemes have been introduced to eliminate artifacts, preserve shape, and generate smooth curves on manifolds (Schaefer et al., 2008). Previous nonlinear approaches take advantage of nonlinearity to generate a much wider class of functions than linear subdivision algorithms. Compared to manually designed nonlinear functions, neural networks can be used to significantly expand the space of geometric details created with a subdivision scheme. Liu HTD et al. (2020) first used a neural network to achieve nonlinear geometry-aware subdivision, effectively avoiding over-smoothing and contraction issues. However, its robustness is insufficient. During inference, when encountering unseen shapes or subdivision levels not observed during training, the subdivision results tend to generate bulging artifacts, potentially compromising the structural integrity of the shape (Fig. 1).

We propose a neural mesh refinement (NMR) method that demonstrates robust generalization. Our key insight is that it is necessary to disentangle the network from non-structural information such as scale, rotation, and translation, allowing the network to focus on