How to transform images

zensvi.transform: A module to transform images

“ImageTransformer” offers options to transform panoramic imagery into two further view types:

  1. perspective

  2. fisheye

“PointCloudProcessor” provides point cloud generation capabilities based on depth estimation in the computer vision sub-package, thus enabling advanced spatial analysis of street-level imagery.

Preparation

# If zen-svi is not installed
%pip install --upgrade zensvi 

import zensvi

print(zensvi.__version__)

# Import the required libraries
from zensvi.transform import ImageTransformer
from zensvi.transform import PointCloudProcessor
import pandas as pd
import os

Panorama image transform

  • Panorama to perscpective image transformation: Transforms a panorama image to a perspective image.

  • Panorama to fisheye image transformation: Transforms a panorama image to a fisheye image. Types of fisheye transformations include stereographic, equidistant, equisolid, and orthographic.

# Set the input and output directories
dir_input = "path/to/input/images" # directory containing the images to be transformed
dir_output = "./transformed_images" # directory to save the transformed images

# Start transforming images
# Set the parameters for the transformation, field of view (FOV), angle of view (theta), angle of view (phi), aspect ratio, size of the image to show (show_size), use_upper_half
image_transformer = ImageTransformer(dir_input=dir_input, dir_output=dir_output)
image_transformer.transform_images(
    style_list="perspective equidistant_fisheye orthographic_fisheye stereographic_fisheye equisolid_fisheye",  # list of projection styles in the form of a string separated by a space
    FOV=90,  # field of view
    theta=120,  # angle of view (horizontal)
    phi=0,  # angle of view (vertical)
    aspects=(9, 16),  # aspect ratio
    show_size=100, # size of the image to show (i.e. scale factor)
    use_upper_half=False, # if True, only the upper half of the image is used for transformation. Use this for fisheye images to estimate sky view.
) 

Converting to perspective: 100%|██████████| 10/10 [00:05<00:00,  1.78it/s]
Converting to equidistant_fisheye: 100%|██████████| 10/10 [00:00<00:00, 309.64it/s]
Converting to orthographic_fisheye:   0%|          | 0/10 [00:00<?, ?it/s]/data/yihan/miniconda3/lib/python3.11/site-packages/zensvi/transform/transform_image.py:222: RuntimeWarning: invalid value encountered in cast
  yp = np.floor((2 / np.pi) * np.arcsin(r / R) * rows).astype(int)
/data/yihan/miniconda3/lib/python3.11/site-packages/zensvi/transform/transform_image.py:222: RuntimeWarning: invalid value encountered in arcsin
  yp = np.floor((2 / np.pi) * np.arcsin(r / R) * rows).astype(int)
Converting to orthographic_fisheye: 100%|██████████| 10/10 [00:00<00:00, 535.64it/s]
Converting to stereographic_fisheye: 100%|██████████| 10/10 [00:00<00:00, 572.74it/s]
Converting to equisolid_fisheye: 100%|██████████| 10/10 [00:00<00:00, 530.64it/s]

Image to Point Cloud transformation

  • Image to Point Cloud transformation: Converts depth and color images into 3D point clouds.

  • Point Cloud Saving: Supports saving point clouds in multiple formats such as PCD, PLY, NumPy, and CSV.

  • Point Cloud Visualization: Provides visualization tools for inspecting generated point clouds.

# Directories for input and output
dir_input = "path/to/input/images"
dir_output = "./point_clouds"

# Metadata of SVI, controlling the  global attributes of generated point clouds (e.g., ID of images to process, global coordinates, and headings)
# data = pd.read_csv(f"{dir_input}/point_cloud_test_df.csv")
data = pd.read_csv(f"path/to/mly_pids.csv")

# Initialize the PointCloudProcessor with paths to the image and depth folders
image_folder = f"{dir_input}/color"
if not os.path.exists(image_folder):
    os.makedirs(image_folder)
depth_folder = f"{dir_input}/depth"
if not os.path.exists(depth_folder):
    os.makedirs(depth_folder)
point_cloud_processor = PointCloudProcessor(image_folder=image_folder, depth_folder=depth_folder)

# Process multiple point clouds
point_cloud_processor.process_multiple_images(
    data=data,
    output_dir=dir_output, # Output directory to save the point clouds. If None, the point clouds are not saved
    save_format="pcd"  # Format to save the point clouds ('pcd', 'ply', 'npz', 'csv')
)

# Optional: Visualize one of the generated point clouds
point_clouds = point_cloud_processor.process_multiple_images(data=data)
point_cloud_processor.visualize_point_cloud(point_clouds[0])

# Optional: Save the first generated point cloud in additional formats
point_cloud_processor.save_point_cloud_numpy(point_clouds[0], f"{dir_output}/point_cloud_0001.npz")
point_cloud_processor.save_point_cloud_csv(point_clouds[0], f"{dir_output}/point_cloud_0001.csv")