Course project for the Project in Computer Graphics course (Computer Science 503.03 at the University of Calgary) I took in winter 2020 and where I upgraded the RTX render engine I wrote last year to support additional features such as ambient occlusion.
Source code is self-hosted on my home Git-Gogs server which is only accessible from my home network.
Project Summary
Objective
Apply my knowledge of Vulkan to improve neuron rendering in connectome visualizations with RTX ambient occlusion.
Justification
- Opportunity to reuse last year’s rendering project.
- Gain greater familiarity with RTX and the Vulkan API.
- Combination of computer graphics, art, and neuroscience.
- Improve existing connectome visualizations.
Features
- Vulkan rasterizer with depth buffer.
- Ray tracer supports:
- Direct Shadows.
- Ambient Occlusion.
- Diffuse Shading.
- Can switch between rasterizer and ray tracer rendering modes.
- User specified camera movement with the ability to disconnect the mouse to make screenshot taking easier.
- Uses assimp for quickly loading models with many polygons.
- Better/cleaner codebase than the previous version.
- Also supports model transformations using push constants.
- Using a texture to encode the directions in which AO rays should be sent in.
Challenges
- Various bugs such as accidentally destroying my acceleration structure buffer before it was finished copying and thus only being able to support about 5 thousand vertices… After resolving that bug, the ray tracer could easily support 50 million vertices.
- Learning more about the Vulkan API.
- Push constants.
- Depth buffer.
- Not deleting the bottom level acceleration structure when it’s still needed.
- Passing textures to the ray tracing shaders.
- etc.
- Creating a texture that encodes the directions in which rays should be sent for computing AO.
- Random vs Halton sequence for computing the directions in which AO rays should be sent. For some reason the Halton sequence created a repeating image texture so a random sequence was used instead.
Final Report
Here’s a link to the final report I wrote for the course.
Screenshots
Final Neuron Visualization
Neuron rendered with diffuse and ray traced ambient occlusion lighting.
Shading Comparison
Comparison of the various shading modes my render engine supported.
Render Engine Comparison
Ambient occlusion comparison between my render engine (labeled Vulkan) and Blender’s Cycles render engine (labeled Blender). My render engine was using fewer samples but otherwise I tried to match it as closely as possible.
Effect of AO Ray Distance
Comparison exploring the effect that ray distance has on ambient occlusion shading.
AO Sampling
The performance and visual impact of AO sampling without any denoising. As evident 32 samples is sufficient for a good enough effect at a high enough framerate on an RTX 2080.
RTX Shadows
Unfortunately shadows detracted from neuron visualization. This is likely due to neuron’s structure making shadows feel disconnected from the geometry casting them.
AO Direction Textures
Due to time constraints I was unable to resolve why the Halton sequence resulted in an obviously repeating pattern. Instead I just used the image from the random sequence as it worked and time was a bit limited.
Future Direction
The Fresnel node in Blender resulted in a nice effect that highlighted neuron edges really well and might improve neuron and therefore connectome visualizations.
Videos
Existing Connectome Rendering
My Neuron Rendering
Credits
Vulkan Resources
- Alexander Overvoorde’s Vulkan-Tutorial.
- Sascha Willems’s Vulkan Examples and Demos.
- Vulkan API and Documentation.
RTX Resources
- Martin-Karl Lefrançois and Pascal Gautron’s Vulkan RTX Tutorial on Nvidia’s blog.
- Sergii Kudlai’s Vulkan RTX Tutorial.
- Ambient occlusion based on Jakub Boksansky’s RTAO project.
Neuron Dataset
- Neuron data from the Hemibrain connectome project.
- Neurons converted to OBJ files using the SWC Mesher Blender addon.