Rendering Diagram

This week I read up on rendering with photon mapping in greater detail and was able to come up with a diagram to help understand the process. However, I left out Final Gathering, which is another method to obtain radiance from the photon map. Final Gather is a very costly process but produces more accurate results. Since I'm implementing this on WebGL and aiming for a decent level of performance, I have decided to leave it out for now and make it an optional component.

Next week, I will work on setting up initial WebGL framework, look for usable libraries, and write basic classes. There are still a lot of unknowns in the area that I will need to look into.

Anyway, here's my pretty diagram of the rendering process. Optimizations are highlighted in a brighter blue.

Senior Design - First Post!

It's been a tough process picking my topic. I originally had two ideas: 1) an Android AR app that lets you place furniture in any room, and 2) photon mapper on WebGL. In the end, I thought the AR project was a little too scary since it involves a lot of computer vision, although the prospect of playing around with Android SDK is still very attractive. So I decided I wanted to stick to basic computer graphics topics (which makes it safe because there's tons of resources out there), but taken to a new platform (I always wanted to play with WebGL), and that's how I came about my topic.

Here's the abstract:

Rendering is the process of generating a 2D image from a description of a 3D scene through a computer program, and it is one of the primary focuses of computer graphics. Years of research in the field has perfected the science of rendering such that today’s movies and films are able to produce almost perfect life-like images. With the advanced rendering methods we have today, it is difficult, sometimes impossible, to tell apart computer graphics from real-life photography. However, recently, web technology has been seeing many amazing improvements, and every day we are finding the web to be a more powerful and versatile platform.

The goal of this project is to bring realistic rendering with global illumination capabilities to the web platform through WebGL. The renderer uses regular ray tracing for direction illumination and uses photon mapping for indirect illumination. The renderer is expected to run on the latest WebGL supported browser, on any machine. The user can also modify the scene and the renderer can respond interactively. Because of these constraints, the target scene is quite simple, with only a few primitive objects.

I'll probably be spending the next week doing lots of reading, along with reading up on basic WebGL tutorials and setting up a basic framework, maybe getting a cube to show up on screen.

Results - Iterations

Here's a set of results of the optimization, step by step at each iteration.

More Results - Refinement in Action

After fixing some corner cases, I was able to produce much better results. I also experimented with smaller neighborhoods, as mentioned in my previous post on multi-level optimization. And here are results. They are looking much better! (p.s., the optimization now takes quite a bit of time to run.)

window size: 64x64

window size: 32x32

window size: 16x16

Successful Result

This is very rough, but I think I'm on the right track. I changed how I compute the texture energy. Before I simply used the average value of all pixels in the neighborhood. Now I am comparing the neighborhoods pixel by pixel, and aggregating the distances. Here's an initial rough result.

Multi-level Synthesis

I think I found why my texture optimization wasn't working. I totally missed this part of the paper: "Multi-level Synthesis." The synthesis algorithm that I was working on was only one step in the entire optimization algorithm.

What happens is they first synthesize the texture at a coarse resolution, and then up-sample it to a higher resolution through interpolation after the synthesis has reach convergence. Moreover, at each resolution level, the synthesis algorithm is run using multiple neighborhood sizes from largest to smallest (they used 32x32, 16x16, 8x8).

To summarize the whole algorithm:

1. Initialize target texture to random.
2. For each 32x32, 16x16, 8x8: run optimization until convergence.
3. Up-sample to higher resolution by interpolation.
4. Go back to step 2. Repeat until resolution is at desired level.

The deadline is coming up soon. Hopefully, I can implement all this by then.

Robust Optimization

Implemented "robust optimization" as described in the paper. The optimization now solves for the "robust energy function" which calculates a weighted least squares. The goal is to have those pixels which are already very close to their source not be less affected by others during optimization.

I tested it against the method that I used before and it was only marginally noticeable. Here are the results: