(Reference PyTorch Code: https://github.com/jonkhler/s2cnn)

• Type of Spherical Convolution

1. Spectral Convolution
   • Converting spherical data to frequency domain using Spherical Harmonics (Fourier Transform is used in Spherical Harmonics)
     • Decompose signals on spherical data → Perform convolution on the created frequency domain → Change to spatial domain through inverse transform
     • Spectral Convolution has Rotational equivariance - for rotated data, output is also rotated
2. Not using Spherical Harmonics
   • ex) Spherical U-Net (rotational convolution) https://frost-crate-a82.notion.site/Spherical-U-Net-on-Cortical-Surfaces-Methods-and-Applications-182f7e966c0e8027bb4cc6c5bab3b070?pvs=4
   • For some model, proceed on spherical mesh

• Spherical Harmonics (https://jseobyun.tistory.com/371)

• Use to change spherical domain to frequency domain
• Laplace’s Equation: $\nabla^2f=0$ ($f$: position defined in spherical coordinate system (구면좌표계)) (Spherical domain must satisfy Laplace’s function)
• Laplace’s Equation Expansion:

• Laplace’s Equation result analysis
  • P and Q connecting $(r,\theta,\phi)\rightarrow f$ are defined only by the constants l and m that emerged during the expansion process, but are not related to either $(r,\theta,\phi)$
  • The different P and Q made according to l and m are orthogonal
  • P and Q are basis functions representing spherical coordinate system
  • Spherical coordinate system basis function $Y$:

$$ Y_l^{m}(\theta,\phi)=\sqrt{\frac{(2l+1)(l-|m|)!}{4\pi (l+|m|)!}}p_i^{|m|}(\cos\theta)e^{im\phi}\\ P_l^{(|m|)}(\cos\theta)=(-1)^m\frac{(l+|m|)!}{(l-|m|)!}P_l^{(-|m|)}(\cos\theta) $$

• Spherical Harmonics
  • Extracting the basis function from the results obtained by applying the Laplace’s Equation to the spherical coordinate system and using this basis function, the scalar value defined on the spherical surface can be expressed only as coefficient
    1. An infinite number of scalars, i.e., scalar fields, that exist on the sphere can only be expressed by coefficient (encoding)
    2. If only a portion of base function is used, there is a loss of information, but it can still be represented (compression)

0. CNN & Omnidirectional image

• Representation of the picture on a sphere, not on a plane → CANNOT apply existing CNN structure

1. Related Study: Learning Spherical Convolution for Fast Features from 360 imagery