JPEG


Note: There is a more recent web document on the same topic.

When you save a photo as a jpeg image, your software follows the following steps:
  • Subdivision of the image into 8x8 blocks
  • Fourier analysis of each block using DCT
  • Quantization. The Fourier (cosine) coefficient a_{nm} is divided by c_{nm} (the quantization matrix) and rounded to the nearest integer. This is a lossy procedure.
  • Compression (lossless)
In a typical photo there are many smooth blocks and then the Fourier coefficients corresponding to large frequencies are small and map into zero after quantization. In this way the lossless compression is very effective because the file contains a lot of zeros.

Consider the following test image:
can_jpg1
Technical comment: The corresponding file is a special jpeg  without lossless compression. It contains the same information as the original image (created with GIMP). The same happens with the rest of the images, they represent faithfully the indicated procedures. The black frames around the image do no belong to the images.

Now, we simply delete the Fourier coefficients a_{nm} with n>1 or m>1, only a_{00}, a_{10}, a_{01}, a_{11}, remain. This makes 4/64=1/16 of the information.
can_jpg2
We observe some flaws in the sharp edges. A kind of shadow. Recall that Fourier analysis is only successful when approximating smooth functions (recall the Gibbs phenomenon).

If we delete the Fourier coefficients a_{nm} with n>4 or m>4, the result is:
can_jpg3
In practice, this and the original image are indistinguishable, although we only keep 25/64, less than 40% of the information.

Now, consider only the Fourier coefficients a_{0m} and see the effect:
can_jpg4
All the edges are distorted except the vertical line. This line has width 6px (<8) and height 248px (a multiple of 8). In each block this line is vertically a smooth function, a constant one and horizontally it is a step funtion. By our choice of Fourier coefficients, we have to our disposal the harmonics needed to analize any constant function vertically (n=0) and arbitrary horizontally (0<=m<8).

If we change the role of n and m, considering only the Fourier coefficients a_{n0}, the vertical line is also affected. Note that the height is preserved (because it is divisible by 8 and starts exactly at the beginning of a block).
can_jpg5

The last two images contain more information than the second one 8/64>4/64, exactly double. But we are including in then some information, like a_{07} or a_{70} that is visually less relevant, and we are missing for instance a_{11} and a_{01} or a_{10} that embody important information.

For instance, preserving all the Fourier coefficients but a_{11}, one gets:
can_jpg6

and if also a_{01} is omitted the image is severely affected:
can_jpg7






The code

Matlab programs generating the images
% Filter Fourier coefficients

imag = imread('image.jpg');
%imag = rgb2gray(imag);
imag = im2double(imag);

figure(1)
imshow( imag )

n = 2;
figure(2)
M = zeros(8);
M(1:n,1:n)=ones(n);
img = mask_image(imag, M);
imshow( img )

n = 5;
figure(3)
M = zeros(8);
M(1:n,1:n)=ones(n);
img = mask_image(imag, M);
imshow( img )

figure(4)
M = zeros(8);
M(1,1:8)=ones(1,8);
img = mask_image(imag, M);
imshow( img )

figure(5)
M = zeros(8);
M(1:8,1)=ones(8,1);
img = mask_image(imag, M);
imshow( img )

figure(6)
M = ones(8);
M(2,2)=0;
img = mask_image(imag, M);
imshow( img )

figure(7)
M = ones(8);
M(2,2)=0;
M(1,2)=0;
img = mask_image(imag, M);
imshow( img )
%Function in another file

function [img] = mask_image(imag, M)
T = dctmtx(8);
% Apply DCT
dct_fun = @(block_struct) T * block_struct.data * T';
img = blockproc(imag, [8 8], dct_fun, 'PadPartialBlocks',true);

% Apply mask on Fourier coefficients
mask_fun = @(block_struct) M .* block_struct.data;
img = blockproc(img, [8 8], mask_fun, 'PadPartialBlocks',true);

% Apply inverse DCT
idct_fun = @(block_struct) T' * block_struct.data * T;
img = blockproc(img, [8 8], idct_fun, 'PadPartialBlocks',true);
end