Monday, January 12, 2009

Reflect this !

It' called Reflection nowadays, it's about inspecting code and rendering visible the internal data structures.

It's a really important thing for shaders when used in a C++ program, as they practically come from a different world. Variable names and types and shader functions need to be exposed to be used.

GUI programming and application state management is another important area.. and where an application or game programmer has to write more of his own stuff (shader reflection is handled by APIs such as Direct3D).
What's lacking in C++ is the so called Run-Time Type Information (or RTTI). Some C++ compilers have some sort of RTTI, but not nearly enough. What is lacking, is the ability to iterate the members of a class and extrapolate member names and values, along with the type.

Here is an example of why this kind of functionality is important.
Let's say that there is a C++ structure that holds the application states:

struct State
    bool RenderWireframe;
    bool RenderNormals;
    int  CurrentWindowX; 
    int  CurrentWindowY; 

If one wants to save these states when the application exits, and reload them when the application starts again (persistent states). One would have to explicitly save every single variable in the structure in a file, and then load again.

// Writing states
fprintf( pFile, "RenderWireframe %i", state.RenderWireframe );
fprintf( pFile, "RenderNormals %i", state.RenderNormals );

// ...

// Reading states back
char tmpName[512];
int tmpIntVal;
fscanf( pFile, "%s %i", tmpName, &tmpIntVal );
if ( 0 != stricmp( tmpName, "RenderWireframe" ) )
    puts( "Wrong order ! Exiting." );
    exit( -1 );
state.RenderWireframe = (bool)tmpIntVal;

..writing is not fun but fairly easy. Reading the values back becomes more problematic.
In the code above I'm assuming a certain order of reading.
But if the user touches the file, or if one is reading a state file that is slightly different because written by the previous version of the application, then everything breaks.

The next best thing is to have a loop, where every line is read in input, the variable name extrapolated (by tokenizing) and look for a match:

while ( !feof( pFile ) )
    fgets( pFile, buff, 512 );
    tokenize( buff, tokens );
    if ( 0 == stricmp( token[0], "RenderWireframe" )
        state.RenderWireframe = (bool)atoi( token[1] );
    if ( 0 == stricmp( token[0], "RenderNormals" )
        state.RenderNormals = (bool)atoi( token[1] );
    // else ...

This works better, but when the number of variables increases, it becomes painful to maintain.

A better thing would be to have a list of variable names, along with their types and a value.
Because C++ doesn't have enough RTTI to do this, one has to do it himself (or herself ?!).

Here is how one would initialize the state and using the state variables:

// Let's call it fat !
FatClass  fatState;

// ...

// Initializing the variables
fatState.AddVariableBool( "RenderWireframe", false );
fatState.AddVariableBool( "RenderNormals", false );

// ...

// Using the variables
result1 = fatState.GetVariableBool( "RenderWireframe" );
result2 = fatState.GetVariableBool( "RenderNormals" );

Now, FatClass would have to internally keep association between variable names, their types and their values.
It also needs to be able to write out those values to a file and read them back in.
Something like this:

FatClass  fatState;

// Loading the state when the application starts
fatState.LoadFromFile( "MyApp.State" );

// Initializing the variables
fatState.AddVariableBool_IfNotExist( "RenderWireframe", false );
fatState.AddVariableBool_IfNotExist( "RenderNormals", false );

// ...

// Saving out, when exiting the application or whenever
fatState.SaveToFile( "MyApp.State" ); this case, the state is first loaded from a file, then the variables that are missing are created with the default values.

All this stuff is quite useful also when doing GUI programming, when the design of every single GUI element is not so important but, instead, getting a lot of parameters exposed quickly is more important.. which is normally the case with internal tools in game development !

I also checked the Insomniac's Nocturnal Initiative Wiki (I remember it being introduced at GDC 2008, a very nice initiative, I hope someday my employer will allow something similar ;).
Insomniac calls these C++ Reflection and Data-Driven Property UI. The latter doesn't have much of an explanation, but it depends on the former, which is more clearly detailed.

Anyhow, between the shader variables wrapping, the application state management and the GUI, I currently have 3-4 different implementations of this "stuff" !!
Maybe I should sit down and try get one implementation right.. ;)


  1. Hey Kaz,

    Isnt this serialization of binary structures what is supposed to be solved by XML files and XML parsers? I personally hate them but at my work everything is done with XML files saving and loading of states.
    Then saving off the variables becomes a simple matter of designing the right class to just dump your variables in along with their name.

    Henk Jan

  2. Mr. Henk,

    You'd still need fill in the members that you want exported or imported with the serialization process. So, I guess it would becomes something like:

    class MyClass : public Serializable
        int mMethod1;
        float mMethod2;
            Serializable::Add( mMethod1 );
            Serializable::Add( mMethod2 );

    Also, in my case, sometimes I can practically give up on having and actual C++ class and just have a "FatClass".
    This saves the burden form having to declare an actual class.. of course it's only good for those variables that don't require much access, because performance is a lot worse 8)

  3. Why am I thinking that C# makes the creation of serializable objects a lot easier ... ?

    I know you're not using C# for development ... but ... 8P

  4. c# has reflection built in, right?

  5. ..I guess ? I have no idea 8)

    Speaking of "tutto rotto".. tonight I tried to talk to my mother via Skype/Google talk... I eventually just called her with my home phone.

    Facking Internet bullshit !!

  6. Paul: I think it does.
    Mr. Angry Tech user: There's no technology that will every please you!!! 8P