Thursday, July 25, 2013

CUDA Path Tracer

Interactive real-time path tracer in CUDA.
Advanced Rendering Seminar
University of Pennsylvania, Spring 2013.
 
My last school project, ever! (hopefully)
 

Github: https://github.com/nopjia/tracer
Images: Dropbox Link
Video: http://www.youtube.com/watch?v=mbpqxlJHaBE

This was my first ever attempt at writing a path tracer, and of course, making things real-time (or at least somewhat) is always more fun. It was something I've always wanted to do. I learned a whole lot about physically based rendering, for obvious reasons. But I also learned a lot about coding in CUDA and its in's and out's, which involved writing a lot of pure C code and writing nifty inline structs and functions for both host and device uses.

Although, the renderer doesn't have fancy features, I'm quite happy with it. Despite the lack of acceleration structures, it still runs at ~20fps, and converges around 30 seconds, on my laptop. The next area of improvement would definitely be adding a kd-tree, in order to support larger scenes.

Finally, as usual, I decided to make things interesting by making it interactive. Being able to easily modify the scene on the fly really makes it more fun and creates lots of interesting images.

Head over to the Github page for a whole lot more info!

Thursday, May 16, 2013

WebTube

Here is a long overdue post that I finally got around to write now that school is over! Over the weekend of January 18-20th, I attended the Spring 2013 PennApps Hackathon, the largest student-run hackathon on the east coast, organized by no other than the University of Pennsylvania. Since this was to be my last PennApps Hackathon (this being my 5th time), I decided to have fun over the weekend, hang out with people, and do something fun.

I made WebTube, a fun little web app that lets you browse the internet through the comfort of an old-school CRT TV.

The reception turned out much better than I had expected. I came into the hackathon only to have some fun and not expecting anything. In the end, WebTube won 3rd place overall and Audience Choice Award!

Demo video: http://www.youtube.com/watch?v=8ZUCyN6yvps
Live code: http://iamnop.com/webtube/ (must enable CSS Shaders flag in Chrome)


Technical Details

The core component of this web app is CSS Shaders, which allows access to programmable vertex and fragment shaders for HTML DOM elements. CSS Shaders are an extension to CSS Filters, which are a set of image filters available for CSS, such as grayscale, sepia, brightness/contrast, hue/saturation, etc. CSS Shaders are essentially custom filters that can be controlled through vertex and fragment shaders. This allows for a much greater freedom and very interesting CSS animations and CSS transitions. (See CSS FilterLab for live examples.) Please see related links below for more information related to CSS Shaders.

With understanding of CSS Shaders, WebTube is a very simple app. It has one main DOM for the screen, which allows for different modes, including web and SSH terminal. Multiple CSS transition rules are applied to the screen using CSS Shaders. The vertex shader warps the screen to produce the curvature and the fragment shader applies TV-like effects on the screen.

Links

http://www.adobe.com/devnet/html5/articles/css-shaders.html
http://html.adobe.com/webplatform/graphics/customfilters/cssfilterlab/
http://alteredqualia.com/css-shaders/article/
https://dvcs.w3.org/hg/FXTF/raw-file/tip/filters/index.html#shading-language

Thursday, September 20, 2012

WebGL Volumetric Renderer


 

Live code: iamnop.com/volumetric
(Does not run Windows. Unknown bug with shader compiling.)

Video: http://www.youtube.com/watch?v=VPhnwOpmUqY

GitHub: https://github.com/nopjia/WebGL-Volumetric

As a small side project, I implemented volumetric ray casting in WebGL. The idea is to ray cast a 3D texture, which should be very fast since OpenGL texture lookups are highly optimized; however, this is not doable since 3D textures are not allowed in WebGL.

During SIGGRAPH 2012, I met Luis Kabongo from VICOMTech, who showed me their implementation of a WebGL volume renderer, which they use for medical imaging. They made it possible by, instead of using a 3D texture, using a 2D texture atlas, which is made up of 2D texture slices. This is a very simple solution. I immediate took this idea and implemented my own version.

VICOMTech's volume renderer is for simply viewing 3D datasets, so they implemented it using alpha compositing with no lighting.

I took it a step further for my own renderer and implemented physically-based volume rendering. The lighting model is based on light transmittance. I introduced multiple lights into the scene and use exponential fall-off to calculate transmittance.

As a result, the renderer features physically-based lighting, volumetric shadows, and support for multiple lights. The results turned out very nicely.

Tuesday, September 18, 2012

CodeDJ Live!

Live code here.
(Only works on Firefox, since it uses Audio Data API.)
The app is not most intuitive and has a tiny learning curve.
Please refer to more info section.

Video: http://www.youtube.com/watch?v=bl11T112Jn0

 


A project I did for the PennApps 48-hour hackathon.

About
CodeDJ Live! is a web-based app for programmers to code up visuals in real-time as the music plays. With this app the programmer becomes a Visualizer DJ who produces stunning visuals at a party.

The app is inspired by IƱigo Quilez and his live code demos at SIGGRAPH 2012 Real-time Live event. Similar examples can be seen here and here.

Technical Overview
This app is essentially a WebGL GLSL live coder hooked up to audio spectrum analyzer. The user has access to the GLSL code that is producing the visuals, which can be compiled instantly and displayed.

Technical Details

Music Analyzer
Audio frequency analysis is done using Mozilla's Audio Data API and fast-Fourier transform functions, provided by [blank]'s signal processing library. The Audio Data API provides access to the framebuffer, which contains the decoded audio sample data. This data is fed into the fast-Fourier transform to extract  spectrum frequency data. Frequency data has 1024 channels. This is way too many for the use of this app and needs to be reduced to a manageable and meaningful number. The higher half of the frequencies are barely noticeable, so they are ignored. The lower half are kept and averaged down to 8 channels total for the user to use.

Music Access
Audio Data API requires access to the audio framebuffer data, which is sensitive secured data access. A limitation to this is the "same origin policy", which means it cannot access audio data outside the same root website. This becomes a problem since users need to be able to use their own music for this app. A solution is to host the web app on a public directory on my personal Dropbox, and have users provide their music using their own Dropbox. This is done through the Dropbox javascript API. This works because now the web app and any user provided files are all within the same root website, which is dropbox.com.

Music Streaming
The user logs in through Dropbox API and specifies which folder to contains the music files. Only .ogg and .wav files are accepted, as determined by Mozilla Firefox. Once the user has selected a folder, no music files are downloaded or transferred. Instead, public URLs of the files are generated and stored within the app for that session. These URLs expire within 4 hours. The audio is then streamed per file as they are needed, using the URL. The app does not load more than one audio file at any time.

Visuals Shader
Visuals are generated using a fullscreen quad and a fragment shader. Audio frequency levels are passed into this shader as uniforms. The GLSL code generating the visuals is exposed through a large text editor, where the code can be edited and compiled in real-time to be displayed on the screen.

Postprocessing Shader
Finally, the visuals go through a post-processing shader which applies a series of custom effects to produce dance-club-like styles. This includes over-saturation, bloom, box-blur, and film grain.

Tuesday, April 24, 2012

Some Nice Images

Just throwing up some nice rendered screenshots. 
Click to see full-size (500x400).


Sunday, April 22, 2012

Another Experimental Feature

Decided to throw in another fun experimental feature.

The Displaced Bumps option displaces the distance function according to an arbitrary function (in this case a simple sin wave). It doesn't work very well with square shapes. However, it works great with round box, and it creates a very interesting shape with the knot.