Russian Roulette, RGB Version

What I did this week was to do correct photon absorption and power adjustment. This is done through a randomized method called Russian Roulette. Here's a quick summary: when a photon hits a surface, given the surface is not a 100% reflective (mirror), the photon will bounce off with less power. Instead of casting a photon with decreased power, Russian Roulette method casts the photon with a probability equivalent to the reflectivity of the surface. When integrating (averaging) photon powers to get the reflected radiance, the results are the same. This is an optimization since less photons will have to be cast total, and thus less would have to be integrated.

I implemented this in my photon scattering code on the Javascript side, and visualized it using WebGL, with colored lines representing the photon color (respective RGB power) and the incident direction.

At the moment I have the basic photon scattering and tracing down. The next step would be to do photon gathering or the radiance estimate. I am planning to implement this on the shader, per pixel; however, Norm mentioned that I should be doing this on the Javascript side as well since radiance estimate is also view-independent. I'm not too sure about this and will need to look into it further.

Alpha Review Feedback

I have just gotten the comments back from our "senior review panel" at the Alpha Review. I think mostly it went well and most are satisfied with the progress so far. However, there seem to be two main issues, which I will address here:

1. Motivation for WebGL
The question that most reviewers had was what is the point of WebGL. How is WebGL better than, say, a web plug-in? What are the benefits of putting such a renderer on the web? Are there any web-based features that could make it special?

I think one of the factors here is that I did not realize most people do knot quite know what actually WebGL is, and thus I did not quite explain fully what WebGL is. I would need to give it it's own slide next time I do a presentation. In addition, I'll also have to motivate it more with concrete benefits such as:

1. Being immediately accessible anywhere, since it is native to modern browsers
2. Having direct access to the hardware through GLSL
3. Exhibiting all capabilities of web-based applications, operating in a browser through Javascript, such that it could be very easily extended with other features

2. CPU / GPU Details
There are two things here that reviewers wanted to see: 1) CPU/GPU breakdown with clear division, and 2) justification for putting processes on the CPU/GPU. In general, I feel that is feedback is definitely true, and does somewhat reflect the state of my progress. It is this division of CPU/GPU where I was and still am figuring out. As I move along the process, I find out where the limitations are and what makes more sense, so it might seem unclear now because I still am trying to figure it out for myself.

But in general, I am feeling good about the feedback. Most reviewers are satisfied with the progress and feel that this project is technically challenging.

Sidetracked: Ray Tracer

This week I sidetracked, and I spent most of yesterday and today trying to get a decent ray tracer running on my current framework. I was successful and was able to add refraction with a decent ray tracing depth. Everything in the ray tracing loop had to be hard-coded since recursion is not supported; this is why it took me so long to figure things out.

I'm also running into problems where my renderer wouldn't run in Chrome. I think I'm hitting some kind of memory cap, because when I tried to increase the objects in my scene, it stopped working in Chrome, only the empty room with nothing in it shows up. It still works in Firefox fine. This also has been happening inconsistently. My live example works for me in Chrome, but shows no objects on several other machines.

The live example can be found at http://iamnop.com/ray/. And here is a screen shot:

Initial Photon Scattering

Finally, after many battles, I have simple photon scattering.

First, I ported my existing intersection code from the shader to Javascript. I realize that this is code duplication, but I feel that it is necessary since I see that photon mapping should be done on the CPU side. (Also, this is a question that should be addressed later.) Next I moved the scene list over to Javascript, which is then passed down to the shader. Here, I didn't handle shape types -- for shapes, I have lists of positions, radii, and colors; I didn't want to add another for types. So at the moment photon scattering only works with spheres. To support cubes, I'm planning to pass a different list down to the shader for each shape. Probably not a good idea for extensibility, but it is the most efficient in terms of memory.

Then I performed photon scattering by shooting rays, storing their position and color as they hit objects, and recursing on the reflected rays. At this point, I don't have the correct power/color computation. I'm only directly storing color as they are absorbed, just to see if I can do it correctly.

Next step would be to do correct photon scattering, which involves correct power adjustment and Russian Roulette method for absorption.

Here is a screen shot of photons visualized over a ray traced scene. The color showing is the color of the previous reflected surface, and each photon can be traced correctly.

More Frustration...

Three.js has many useful built-in classes. It even has a Ray class and a Scene class complete with objects, meshes, and intersection code.

Today I was trying to use the built in intersection functionality for photon scattering. Turns out, I spent +5 hours (and counting) trying to figure it out. It should be very simple but for some reason I am not getting any intersections.

I've been posting questions on github asking for help. Here's what's going on:
https://github.com/mrdoob/three.js/issues/680

Passing Arrays to Shader

Finally, I have decided to do photon scattering on the CPU side.

Therefore I would need to figure out how to pass an array of photons to the shader after they have been scattered. Today I spent 4 hours trying to figure out how do pass arrays to the shader.

Since you cannot pass arrays as uniforms into the shader itself, I would need to construct a texture out of photon data and sample it for values. I'm not sure that I could construct a texture and pass it to the shader in Three.js, so I rebuilt my existing Three.js renderer in just pure WebGL, since I know I can construct a texture in OpenGL.

After 4 hours, I figured things out and completed a version of my renderer on WebGL. However, at that point, I also found out that you can pass arrays to the shader in Three.js very easily, just like passing any other types of data.

So it looks like I just wasted 4 hours that I could've spent on the actual photon scattering.

Where to Build Photon Map?

As I sat down to carefully read about photon mapping and think it through, one big question popped up that is stopping me from moving on:

Where should I be building this photon map?

The photon map is view independent, and is static unless something in the scene changes. Thus, it makes sense to build it on the CPU side (Javascript) then pass it down to the shader to do per-pixel ray tracing. However, I have read a few articles (including John CIS565 lecture) that mentions constructing the photon map on the shader, and how it can give a better performance.

Right now, CPU side construction makes more logical sense to me. However, if I choose to do this, I would have to re-implement all the intersection code for shapes, and move the scene from shader back to Javascript, which will be a bit more work. I also will need to look into how I can pass non-primitive data structures down to the shader.

Anyone who knows anything about this please advise?