Adding depth

As seen in the last section, there is currently an issue with some parts of objects showing up despite being behind other objects. This problem of eliminating parts of objects hidden by others (Hidden Surface Removal) can be tackled in a variety of ways.

Object space methods will try to order the objects in a way, such that when drawing them in order, the correct ones are shown. This is basically what you will do when painting, drawing the backgrounds first and then going to the foreground, which is why this is called the painter's algorithm. We will come back to this for transparency. There are some issues with it, mainly that such an order does not always exist. Some advanced techniques try to solve this issue, but they are pretty complex.

In pixel-based rasterizers, a much simpler approach is taken. The screen has a fixed resolution and we are producing our primitives inside of that constraint. So it doesn't bring us an advantage to have a (mathematically) perfect drawing order, if we can't see it anyways. The only point of an opaque object that we see is the closest one to the camera.

As we are looking in the $z$ direction, we can measure the closeness by just that $z$ coordinate. In window coordinates, they are in the range $[0,1]$ (of course, without far plane clipping, the upper limit is actually infinite, but that is just a technicality).

The idea is now: Use an additional buffer next to the ones for colors with the same size as your screen. Initialize it with the maximum distance to the camera. When we want to put a fragment into a pixel, we check, if the fragment's depth (the $z$ coordinate) is less than the one stored in the buffer at that pixel. If not, we have already put something there, that is closer to the camera and so the fragment won't be drawn. Otherwise, we can overwrite the current pixel and put the fragment's depth into the buffer. We call this buffer the Depth buffer or Z-Buffer.

Our framebuffer, that stores the currently used writing output will now contain an additional field: The depth buffer.

class Framebuffer {
  constructor() {
    this.color_buffers = {};
    this.depth_buffer = null;
  }
  static new() {
    return new Framebuffer();
  }
}

The depth buffer is initialized as null, since we will make it optional. Sometimes you don't need it.

Additionally, we add some options to our pipeline object in addition to the previously mentioned default clipping plane.

function create_depth_options({
      enable_depth_test = false,
      enable_depth_write = true
    } = {}) {
  return {
      enable_depth_test,
      enable_depth_write
  };
}

class Pipeline {
  ...
  constructor({
    ...
    depth_options = create_depth_options(),
    clip_planes = [
      // Near plane -> required to avoid
      //  zero-divisions and reflections.
      // Other clip planes are optional
      vec4(0.0, 0.0, 1.0, 1.0),
      // Far plane
      // vec4(0.0, 0.0, - 1.0, 1.0),
    ],
    ...
  }){
    ...
    this.depth_options = depth_options;
    ...
  }
  ...
}

These options are enable_depth_test and enable_depth_write and allow us a great range of flexibility. We only do the depth test, if enable_depth_test is true. And we only allow writing into the depth buffer, if enable_depth_write is true.

That setup allows us for example to use already computed depth values for further tests but not update those values. You will see the application in the section for transparency.

When setting up our pipeline, we will create an additional image with one component which we attach as the depth buffer. Our buffer uses float values, but it is common to use integers (24bit integers even!).

When clearing the image at the beginning, we also fill the depth buffer with the highest value that we allow.

There might be a question on your mind regarding the depth value and pixels. Where does it come from, we have only computed it for the vertices (it is just the z coordinate of the vertex after the perspective divide)?

The answer is, we have to calculate it. Luckily, we already created the mechanism for this beforehand: Interpolation. We will just interpolate the z-values of our points. In the next section, we will have a closer look at this operation, as there is actually a lot going on with it mathematically! We will also do the same with the last coordinate of the input points, which contain $\frac{1}{w}$ after the viewport_transform method, where we intentionally stored this value. This will also become clear in the next section. This is very simple code, so we will just show it here. This is also done in shader languages such as GLSL, where gl_FragCoord.w actually contains $\frac{1}{w}$ .

Previously we computed the frag_coord like this:

const frag_coord = vec4(px.at(0), px.at(1), 0.0, 1.0);

We replace this in the two rasterization methods (with just the important pars):

rasterize_line(pipeline, a, b, ...) {
...

  // depth values (after perspective division!)
  //  can be linearly interpolated
  const frag_z = this.interpolate_line(
    a.at(2), b.at(2), t
  );
  // w contains 1/w before perspective division
  //  which can also be linearly interpolated
  const frag_w = this.interpolate_line(
    a.at(3), b.at(3), t
  );
  // the final fragment coordinate
  const frag_coord = vec4(
    px.at(0), px.at(1), frag_z, frag_w
  );

...
}

 rasterize_triangle(pipeline, v0, v1,
        v2, ...) {
...

  // depth values (after perspective division!)
  //  can be linearly interpolated
  const frag_z = this.interpolate_triangle(
    v0.at(2), v1.at(2), v2.at(2), b
  );
  // w contains 1/w before perspective division
  //  which can also be linearly interpolated
  const frag_w = this.interpolate_triangle(
    v0.at(3), v1.at(3), v2.at(3), b
  );
  // the final fragment coordinate
  const frag_coord = vec4(
    x, y, frag_z, frag_w
  );
  
...
}

You can now implement the depth test logic into the write_fragment method. The solution is below.

Exercise:

Implement the depth test in write_fragment in rasterizer.js
- Follow the instructions in the code

Solution:

See the solution

We now have functioning 3D images! As mentioned before, there is still something kinda weird. If you look closely at the textures, they don't really seem to follow the perspective and have some weird distortions around the diagonal.

This is what we will fix in the next section.