Project 1

Images of the Russian Empire: Colorizing the Prokudin-Gorskii Photo Collection

Project Overview

This project reproduces and implements the automatic alignment and colorization process for the Prokudin‑Gorskii glass plate negatives. The goal is to take a grayscale scan containing three stacked channels, split them into B/G/R channels, align them properly, and merge them into a single color image. Several enhancement steps (cropping, contrast adjustment, white balance) are also included to improve the visual quality.

Methodology

1. Channel Separation

  • The input grayscale image is split into three equal vertical sections: Blue (top), Green (middle), and Red (bottom).
  • Each channel is converted to floating-point precision for further processing.

2. Alignment

Two approaches are available depending on file type:

  1. Single-Level Alignment (default for non-TIFF images)

    • Uses brute-force search within a range of ±15 pixels.

    • Alignment metrics supported:

      • MSE (Mean Squared Error): \(\text{MSE} = \frac{1}{N} \sum (I_1 - I_2)^2\)

      • NCC (Normalized Cross-Correlation): \(\text{NCC} = \frac{\sum (I_1 - \bar{I_1})(I_2 - \bar{I_2})}{\sqrt{\sum (I_1 - \bar{I_1})^2 \sum (I_2 - \bar{I_2})^2}}\)

      • Gradient Correlation: \(\text{Gradient NCC} = \text{NCC}(|\nabla I_1|, |\nabla I_2|)\) where $|\nabla I| = \sqrt{G_x^2 + G_y^2}$ using Sobel operators

        Purpose and Advantages:

        • Robustness to Illumination Changes: Edge features are less sensitive to brightness variations between color channels
        • Better Performance on Challenging Images: Particularly effective for images like Emir where channels have different brightness levels
        • Enhanced Feature Detection: Focuses on structural content rather than absolute pixel intensities
        • Reduced Noise Sensitivity: Edge-based matching is more robust to image noise and artifacts

        Emir Alignment Method Comparison:

NCC Alignment
Standard cross-correlation
Gradient Correlation
Edge-based alignment

Results - Single-Level Alignment:

Visual Results:

Cathedral
G offset: (2, 5)
R offset: (3, 12)
Monastery
G offset: (2, -3)
R offset: (2, 3)
Tobolsk
G offset: (3, 3)
R offset: (3, 6)

  1. Multi-Scale Pyramid Alignment (default for .tif/.tiff)

    • Builds an image pyramid by downsampling channels.
    • Alignment starts at the coarsest level with a small search range.
    • Estimated shifts are propagated to finer levels, doubling at each step.
    • Dynamic search range increases with pyramid depth (from 3 up to 15 pixels).
    • Robust to large displacements and faster than brute-force on high-resolution images.

    Results - Multi-Scale Pyramid Alignment:

    Visual Results:

Emir
G offset: (23, 49)
R offset: (40, 107)
Harvesters
G offset: (17, 60)
R offset: (13, 123)
Icon
G offset: (17, 42)
R offset: (23, 90)
Italil
G offset: (22, 38)
R offset: (36, 77)
Lastochikino
G offset: (-2, -2)
R offset: (-8, 75)
Lugano
G offset: (-16, 41)
R offset: (-29, 92)
Melons
G offset: (9, 82)
R offset: (12, 178)
Self Portrait
G offset: (28, 78)
R offset: (36, 176)
Siren
G offset: (-7, 49)
R offset: (-25, 95)
s
Three Generations
G offset: (14, 53)
R offset: (10, 111)
Church
G offset: (4, 25)
R offset: (-4, 58)

3. Border Cropping

  • Option 1: Automatic Cropping (--auto-crop flag)

    • An algorithm scans rows/columns for brightness transitions (black borders, white patches, content regions) to detect borders dynamically.
    • This ensures that noisy scan margins are removed adaptively.

  • Option 2: Fixed Margin Cropping (default)

    • For .tif/.tiff files, a margin of 200 pixels is cropped.
    • For smaller files (e.g., .jpg), a margin of 30 pixels is applied.

Results - Border Cropping:

Emir Automatic Cropping Comparison:

Before Cropping
With original borders
After Automatic Cropping
Borders removed

4. White Balance Adjustments

(Optional, enabled with --white-balance <method>)

  • gray_world: Forces average R/G/B channel intensities to be equal.
  • white_patch: Scales channels based on the brightest 0.1% pixels.
  • histogram: Performs percentile-based histogram stretching (2–98%).
  • none: No adjustment.

Results - White Balance:

White Balance Method Comparisons:

Lastochikino White Balance Results:

Original
No white balance
Gray World
Average channel equalization
White Patch
Brightest pixel scaling
Histogram
Percentile stretching (2-98%)

Siren White Balance Results:

Original
No white balance
Gray World
Average channel equalization
White Patch
Brightest pixel scaling
Histogram
Percentile stretching (2-98%)

Church White Balance Results:

Original
No white balance
Gray World
Average channel equalization
White Patch
Brightest pixel scaling
Histogram
Percentile stretching (2-98%)

5. Output Generation

  • After alignment and optional enhancement, the channels are stacked into an [H, W, 3] color image.
  • Converted to uint8 for saving.
  • Output file names include alignment shifts, e.g.: out_path/church_g(4,25)_r(-4,58).jpg
  • The program also displays the final aligned image.

Usage Example

python colorize.py --input church.tif --white-balance gray_world --auto-crop

This will:

  • Load church.tif
  • Automatically crop noisy borders
  • Align channels using pyramid + gradient correlation

Challenges & Solutions

  • Large displacements on high‑res images slow brute force search: solved with image pyramids. Runtime reduced from minutes to under a minute.
  • Color fringes / borders: automatic cropping + white balance correction mitigated colored borders.
  • Channel brightness mismatch: gradient matching and local normalization improved robustness.