Sunday, November 23, 2008

Brain Dump.pptx

The time has almost come for a pitch for future projects at R&D in my company.
I have it fairly clear in my mind what direction we should take, but I find it hard to explain it.
I hit PowerPoint and tried to give an more graphic representation of my thinking, but I had little success so far.

It's frustrating, but I realize that it's also my fault if I can't explain things nicely with some neat graphics.
I think that it's important to be able to scribble on a whiteboard or to make an effective PPT presentation. There are books to learn that.. I see managers going about those manager-like explanations and I realize that those don't come out of nothing (cough cough). There must be an "how to scribble on a whiteboard and leave people in awe" kind of book.

My proposed direction is based on simple concepts, but I've had hard time explaining even to other graphics programmers.. in the game industry.
When I try to explain, I never really get a good reaction... which is both good and bad.
It's bad because it would be nice to share thoughts and experiences. It's good because it could make a case for a salary raise 8)

Generally, I think that real-time 3D graphics as we know it is dead, or it should be.
It's still alive because those that do real-time 3D are stuck in that world, and those that are doing production rendering don't even want to think about real-time.

I once dreamed of a RenderMan vs MentalRay war to get the foot into the door of real-time. But as far as I know, Pixar doesn't have any interest in that... and NVidia (which now owns Mental Images) appears to be very conservative: pushing for DX 11 already... yawnnnnn.

But rendering is bullshit.. I mean, it's more complex than ever before, but it's not where the big problems are with productivity. It takes 20 GB of textures and geometry (mostly texture) to render a frame of the "good stuff", the pre-rendered sequences that make you want to buy the game. Real-time rendering instead, works on much smaller footprints.. about 1/100th of that.

Oh, and please, stop saying that real-time is catching up.. you can put 20 GB of VRAM in every new video-card, but you'll still get some artist to break that barrier the next day. This is not about beefing the system up, it's about being smart with data access.

DXT compression will reduce the needed textures by 1/4th, but that's not nearly as good. Assets need to be structured in a way that it is extremely efficient to randomly access data.
It's very unlikely that a 8k x 8k texture needs to be fully accessed to render a 2k x 1k frame, yet many of those giant textures are likely to contribute to the rendering of a single frame.
A tiled structure allows accessing only portion of a texture. This helps in some cases, but it doesn't help when one needs the whole texture into a tiny 10x10 pixels' polygon. For this task, one needs to access a texture at a given, pre-filtered resolution (mip-map). This is something that for example Photo Realistic RenderMan has been doing for a while, but that the real-time world completely ignores.. because it's assumed that all assets will be pre-loaded in VRAM for that frame, and most likely for the whole stage of that game.

Movies keep relying on production renderers to deal with huge assets, while games keep assuming that artists stay on memory budget or that some build process will resize textures to a workable resolutiion.

Optimized assets build systems are definitely a better solution than asking artists to resize textures.. however build systems are too deached from real-time. A build system will resize your assets so that they fit nicely in your VRAM, but only for a one-time fixed size.

A better idea would be to fuse the build system and the run-time so that assets are "cooked" at different levels depending on the digesting abilities of the target plaform. "Very well done" for the Wii, "Medium Rare" for the PS5, and so on.

I talked mostly about textures, but the concept should be extended to geometry and possibly physics as well.



  1. Mr. Researcher,

    You should look into possibly requesting some time to be able to do some R&D to generate some hard data of the benefits of your ideas to the bottom line (that's $$$!) of the company.

    If your methods will boost productivity and reduce development costs at the same time, I'm quite sure you'll capture the attention of the people who write the checks there.

    What I'm proposing is not about money, but knowing how to "market" your ideas well enough that you'll get the support you need to pursue them.

    One thing I've noticed in myself and amongst other engineers is that at times we lack that ability to sell our ideas in a clear way to others. So people pass on our ideas (not because they are bad), but because we couldn't show how our vision met the goals and needs of their vision. Of course this is true only if you work for someone other than yourself. 8P

    The PR RenderMan TOD stuff sounds similar to iD's MegaTexture thing. I think in the current next-gen consoles aren't optimized for this kind of movement of data. Of course, one could set up a data caching/paging scheme to load these things on the fly, but getting this to work (in addition to whatever's being read off of storage media) at a decent performance is another story.

    Of course, this might be alleviated a little with procedural textures, but this option might not be ideal (depending on the procedural algorithm).

    Oh well, enough future speculation for me. Need to upgrade this PC next year ... will adopt Intel's i7 Processor for home R&D.

  2. Apparently I'll be getting an i7 machine pretty soon 8)

    About generating hard data.. that's a bit of a problem. I'm pressed for time, busy developing and squeezing research along the lines.
    So far the best result has been the progressive texture decompression implementation (not just a test case, but a complete implementation with all formats, etc).. however that was not planned in the project. I just took it upon myself.
    For me, the best "inventions" come out as collateral work in a project. One has to work on something, find a necessity and fix it.

    This completely defies the concept of scheduling work.. but one can't schedule innovation and creativity !

    Lastly, notice that "hard data" is hard to define. An advancement in productivity is only really visible at the end of a sizable project.
    To actually get the bosses on board with a vision, one really has t get busy with PowerPoint and possibly come up with a small demo.. which it doesn't need to employ a technique, but rather give an easy to understand visualization of what one really means.. 90% illustrative work !

  3. Looks like we're both going back to the "Intel Inside" days. 8P

    I didn't mean to imply that hard data was easy to generate (from a point of view in regards to time); over the years I realized it's just a lot easier to convince people with some data or a demo (as you mentioned). If you can make the demo with the 90% PowerPoint animated movies and text, then go for it. :)

    Hope you are taking care of your body with all the business ... coding in bed (because one is sick) is no fun. 8P

  4. Emm I mean I should be getting an i7 at work, not at home... where I'm going strong with iMac + MacBook. In the meantime I got my work machine upgraded to Vista 64 and 8 GB of RAM 8)

  5. I understood you meant i7 at work. 8P I've a MacBook and iPod touch for 開発 purposes as well. I need to finish some work that I'm doing write now before I can fully focus on my own stuff again. I have to admit that using a Mac is a lot less stressful than a PC ... we'll have to share Mac programming tips.

    I think once you get used to XCode, you'll find some things that you'll like (and will hate). But no development platform is perfect I guess.

  6. What does '開発 purposes' mean?

    Looks like a slalom skier next to a house?

  7. Oops ... sorry about that. '開発' kaihatsu which is Japanese for 'development'. I just meant to say that I'm planning on tinkering (code-wise) on the iPod touch to just see what the platform can do.