Technical Report PHD-2017-13

Title: Shape Correspondence using Spectral Methods and Deep Learning
Authors: Alon Shtern
Supervisors: Ron Kimmel
PDFCurrently accessibly only within the Technion network
Abstract: The interest in acquiring and analyzing three dimensional representations of the world is ever increasing, fueling a wide range of computer vision algorithms in the field of geometry processing. Spectral analysis has become key component in many applications involving non-rigid shapes modeled as two-dimensional surfaces, and recently, convolutional neural networks have shown remarkable success in a variety of computer vision tasks. In this dissertation, we designed a set of methods and tools that use these paradigms for applications such as shape correspondence, nonrigid deformations, and volumetric optical flow.

Finding the correspondences between pairs of shapes is a fundamental operation in the field of geometry processing. Measures of dissimilarity between surfaces have been found highly useful for this task. A powerful approach for measuring distance between two nonrigid shapes is to embed their two-dimensional surfaces into some common Euclidean space, defining the comparison task as a problem of rigid matching in that space. We introduce a novel spectral embedding, named the "Spectral Gradient Fields Embedding", which exploits the local interaction between the eigenfunctions of the Laplace-Beltrami operator and the extrinsic geometry of the surface. The common embedding relies on the assumption that the Laplace-Beltrami eigenfunctions computed on each shape independently are compatible with each other. However, this assumption is often unrealistic. We address this issue by matching a small number of eigenfunctions, that are relatively stable, using a high order statistics (HOS) approach.

We also analyze the applicability of the spectral kernel distance as a measure of dissimilarity between surfaces, for solving the shape matching problem. To align the spectral kernels, we developed the "Iterative Closest Spectral Kernel Maps" (ICSKM) algorithm. ICSKM extends the Iterative Closest Point (ICP) algorithm to the class of deformable shapes. Instead of aligning the shapes in the three dimensional Euclidean domain, this method estimates the transformation that best fits the embeddings of the shapes into the spectral domain.

In case the data consists of an incomplete, occluded and disconnected parts of a shape the approach we took is to use a small set of representative natural poses. Using these few exemplar shapes, the method expresses an unseen appearance by a low-dimensional linear subspace, specified by a redundant dictionary of weighted vertex positions. The algorithm, called "Fast Blended Transformations" (FBT), finds the deformation that best fits the partial data, minimizing a nonlinear functional that regulates the example manifold in a smooth way, and detects the pointwise mapping between the partial shape and the reference shapes.

Volumetric optical flow is a different way to describe the matching in a three-dimensional dynamic scene. We designed a multi-scale optical flow based architecture for predicting the next frame of a sequence of volumetric images. The fully differentiable model consists of specific crafted modules that are trained on small patches in an unsupervised manner. The approach, called V-Flow, is useful for analyzing the temporal dynamics of three-dimensional images in applications that involve, for example, motion of viscous fluid substances or real magnetic resonance imaging (MRI).

CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the PHD technical reports of 2017
To the main CS technical reports page

Computer science department, Technion