Sunday, December 21, 2008

Recent home coding: ribparser

I've been spending some time at home writing a very basic RIB parser. RIB being the RenderMan scene description format.
The goal is to be able to read in input simple files and someday maybe even render them. But the real point is to get RIB files in input, because that's a well established standard in the world of production rendering and that's where my focus is right now.

RIB files are text files with commands and parameters which pretty much translate to the RenderMan Interface (or RI) that is also available for C programming (provided a RenderMan license..).
RIB files being so close to an API also give an insight into a state machine for the REYES pipeline ..something that I think is worth looking at, though I'm not sure that it's going to translate well for real-time 3D.

I've looked before into the RenderMan interface with the book The RenderMan Companion by Steve Upstill. The book actually only covers the C interface, but it shows many simple examples on how to create things "procedurally" and how to handle a scene, including creating shadow maps.
An important book to learn about RenderMan, but not too explicit about the actual internal structure of a renderer.

Recently, another book refreshed my interest for the RenderMan interface and off-line rendering in general. The book is called Production Rendering: Design and Implementation. Edited by Ian Stephenson in collaboration with 6 other experts in the field.
The book explains what happens behind the scenes of a production renderer and it goes straight into the RenderMan + REYES pipeline approach. From the state machine to the way primitives are processed and converted into micro-polygons, to how the micro-polygons grids are "shaded" using custom shaders that can run on massive multi-core architectures (as I write, the latest NVidia card has 540 cores 8).
Production Rendering got me really excited and made me realize how close are modern real-time shaders to those meant for off-line renderers. I also like the C++ code snippets that show things like are meant to be explained (show me the code !!) reminds me a bit of my long lost Advanced Animation and Rendering Techniques
by A.Watt and M.Watt (15+ years ago !!).

Then, recently I found this a basic RenderMan implementation in Shell script !! It's called RenSH and it's written by Ian Stephenson (see some of his other RenderMan-related works, like the PS2 implementation).
RenSH clearly very cool and it was also an inspiration for me to go ahead and try to write my own RIB parser... if someone can write a parser and a renderer in shell script, surely I can write something in C/C++ 8)
As it turns out, shell script actually makes it easier to write a parser, because it is a parser itself, and the guy clearly knows the RenderMan interface inside out. But still, it's quite humbling to see the whole state machine, the transformations and the quadratic primitives becoming actual ASCII art 8)

Anyhow, my project is over at the Google code repository. It currently really only gets one test .rib file in input and parses it validating the arguments of a small subset of commands. It will hopefully grow.. but that really depends on many things (like, time 8).
The project is also an excuse to get acquainted with XCode, so the actual project file is in XCode format, while the source is pretty much plain C++ using printf() and std::vector.

Have fun !

Note: The book links are by my Amazon Associates account just because. But, I really suggest, if possible, to shop around and maybe have a look at the actual books. I bought mine at GDC and Siggraph (it was $100 for Production Rendering !!).


  1. the fun part is how you take your time to read books! better to search on the internet for me. :-)

  2. Sometimes it's nice to have all the infos collected and explained in one portable device (the book).

    You can search the Internet for ages and never find what's in the book.
    Or maybe you can find it, but it's scattered in so many places and without any clear explanation.
    Also you can't really bring "the Internet" in the restroom, where so much thinking happens ! ;)

    In this specific case, what's in "Production Rendering" is really nowhere else, so I had to shell those $100 !! (it's cheaper now apparently).

  3. So you think that the renderman format is the c/c++ of graphics languages?

    In that case we need an optimizer for it!!!

  4. I see RenderMan as an input format.. of course one doesn't want to parse a RIB file from scratch and render that in real-time.
    Also what is good for rendering is not necessarily structured well enough to be in a game (needs a different structure to be controlled, etc).. but as far as a appearance goes, it's clear to me that we can't rely anymore on programmers or technical artists to rewrite shaders from scratch, throwing away or ignoring what instead goes in production rendering (taking shaders as an example, while my goal here is mostly to the geometry set up for rendering).

    A RenderMan file should include all that is needed to render a full-quality scene.. of course with references to external texture maps (without considering all the compositing tricks that go into the final product).

    So, more than C++, I see RenderMan as the PDF of publishing files.
    Exporting data for games it's still a pain, with scene structure decided by some custom made exporter, or perhaps using Collada, but with no tangible support for different types of surfaces, displacement mapping, materials and shaders.. all important things for the final rendering quality.

    I think that to get better images we need to learn to waste resources and overdo things like it happens for off-line renderers.. and only then optimize back and shift work to build phases as necessary.

    The other alternative is Mental Ray scene format.. however Mental Ray's shaders are already all in C/C++ which makes it harder to implement without source code.. and also less efficient, because C code is harder to parallelize 8)
    If SIMD is the key to performance, so, the book Production Rendering argues, one may as well interpret a shader, because if each instruction is repeated for 100 pixels, the speed of the actual interpreter is 1/100th as influential.