Keynote: Discovering new development opportunities (Satoru Iwata, Nintendo)

The day starts out with a packed room, all waiting to hear the keynote to be delivered by Satoru Iwata, President of Nintendo.  With the unquestionable success of the Nintendo products worldwide this talk is to pull back the curtain a bit on some the ideas and methods that have lead to this success.

Iwata started off by presenting the obligatory numbers slides showing how much success both the Wii and DS have shown in recent history.  No one can take that away from them, they have several successful platforms currently.

Iwata then started to discuss, arguably the most important developer at Nintendo, Mr. Miyamoto.  He explained that Miyamoto is one of the main reasons for the continued success.  He explained in somewhat detailed terms, how Miyamoto’s development style has achieved this success.

Mr. Miyamoto first starts with a core concept, as do most software projects.  One of the differences comes from where Miyamoto pulls the new ideas from.  He is fascinated with studying humans and their behaviors, specifically, when they are doing something that makes them happy.  He will draw on this, to come up the concept for the new piece of software that he is attempting to create.  For example, he got a dog for his family, and out of this, was born Nintendogs.

Of course, having a good idea for a game concept, and following this through to release of a successful title are 2 very different things.  This brings up the next key point.  This is Miyamoto’s software development style.  He typically will form a very small team (sometimes even just one developer) and they will work on a prototype, or rather a series of prototypes.  At this stage, the graphics are very crude (boxes).  They will work on this for how ever long this takes to perfect the core concept.   At this stage, no even the president of the company will ask how things are going, or when this will be ready for the next stage.  It should be noted that sometimes at this stage, work will be done, but then shelved.  This could happen for various reasons, but almost always, at least some of this will be used at a later time.

If the prototyping has met Miyamoto’s satisfaction, only at this stage will others be brought in on the game.   This is where the polish comes in (graphics) and such, but the core gameplay is pretty much guaranteed at this point.  This saves from the issue of after spending time on later polish items, a core gameplay elements requires a rewrite.  This almost never happens with this style.

Nintendo also has some unique “playtest” elements to the project.  They do not conduct formal playtests.  Instead, Miyamoto will “kidnap” an employee (non-technical) and have them play the game (with no help).  He then checks to see how it works out.  If they are able to understand a play with no help, the dev team has done their job.  If not, its a failure and will be readjusted.

Next, Iwata unveiled the new Virtual console with larger SD support and options to run from SD.  Also some game demos of future titles were shown.  Also, Rhythm Heaven was introduced, and he gave everyone in attendance a free copy, before this can be bought.

Indexer example in C#

using System;
namespace Indexer
{
class Program
{
static void Main(string[] args)
{
TestObject obj1 = new TestObject();
TestObject obj2 = new TestObject();

obj1.AddData(new[] { "this", "is", "test", "one", "!?" });
obj2.AddData(new[] { "this", "is", "another", "test", "!" });

// output to check object
OutputObject(obj1, "OBJ1");
OutputObject(obj2, "OBJ2");

obj1[1] = "was";
obj2[1] = "used to be";

// output to check object
OutputObject(obj1, "OBJ1");
OutputObject(obj2, "OBJ2");
}

public static void OutputObject(TestObject obj, string name)
{
for (int i = 0; i < 5; i++)
Console.WriteLine(string.Format("{0}: {1}", name, obj[i]));
}
}

class TestObject
{
private readonly string[] store = new string[5];

public string this[int index]
{
get { return store[index]; }
set { store[index] = value; }
}

public void AddData(string[] objData)
{
for (int i = 0; i < 5; i++)
store[i] = objData[i];
}
}
}

Memory alignment

I have heard the discussion of memory alignment, or rather the question about is it really necessary. I can say from a standpoint of any system where you would like to carefully manage memory (for speed or scalability), yes, it matters.

Memory bandwidth can quickly become the bottleneck to a system. If we take, for instance, this case. We have a processor that has a memory width of 32 bits. If we are going to fetch something from memory (say an int, which happens to be 32 bits wide). With this situation, as it is aligned, the processor can fetch the value in one cycle.

Many of the data enumerations in DirectX contain a value at the end named x_FORCE_DWORD with a value of 0x7FFFFFFF. This value is 1111111111111111111111111111111 (31) bits. This will guarantee this enum will be at least 32 bits in size.

Run it or rip it - UPDATED!

Just another update on the upcoming NXE for Xbox live. This pertains to the ability to rip the games to the HDD, in hopes of faster load times/better in game performance. I have said before that I reserve judgement on whether this is a good thing or not, and I think further revisions may help sway me this way. My argument is, when I have time and want to play some games, I don't want to wait for installs!

NeoGaf has posted this link ( http://www.neogaf.com/forum/showpost.php?p=13630060&postcount=189 ) where they tested Halo 3 with the install and without. This drives my point exactly. When a game is built with ability to adapt to hardware (no HDD or HDD), you get a much better game. There are some cases shown here where the game actually ran slower if you copied to the HDD, because they are already doing some rather intelligent caching.

So thats my rant for the day. Careful what you wish for, and please don't make me wait to play!?!?!

Run it or rip it!

The upcoming NXE (xbox live dashboard update) will bring with it a feature to rip the game to your hard drive and run off that. This is has been a topic of discussion with a few of my friends because there are some conflicting views. The PS3 camp is saying its great, but they are forced to "install" their games. While this makes the game run faster (eliminates some loading issues that have to be coded special), the downside is the customer is left sitting at basically a file copy screen for minutes before being able to jump into the game.

Lets get to some specifics (with the 360). The 360 has a maximum (best case scenario with a pristine disk) transfer rate of about 12 Mbps. Accessing the same data from HDD would be about 10x the speed. This makes loading the required assets much easier for the programmer. The 360 has only 384 MB of system memory, so there is alot of loading from disk/DVD. To get around this programmers have devised complex caching and prediction algorithms to be sure the user is not stalled. In some cases, companies have chose to load limited assets based on whether the user has a HDD or not, as items are cached there. Note currently no company on the 360 for a retail game loads the entire game there, just individual assets (think cache).

A recent post ( http://www.joystiq.com/2008/10/31/xbox-360-load-time-comparison-dvd-vs-hard-drive/ ) has shown the upside. Quieter system and slightly faster load times. They have shown the transfer rate to be about 1.7GB per minute (so they are seeing about 28 Mbps transfer on a file copy). For an 8GB game this would leave the user sitting (best case) slightly under 5 minutes.

I am not sure this is good thing. What I would like to see if the strategy that Games for Windows uses. This is give the user the option when the put the disk in the first time. To wait or not to wait. If they chose not to wait, load only the require level (lets say for arguments sake 1 GB of data), which would take less than a minute (show some screenshots or something in between ;) ). Then predict the next, lets say level, to be loaded and on idle cycles in the game, copy the files ( of course giving priority to the action rendering and game code if they need the resources). This a lazy loading pattern used in programming for many different types of applications.

I would hope that the hardware engineers are thinking about this for the next gen game consoles to allow the movement of assets from optical to hdd. Ultimately download games would be great, no more copy, but there are costs with this as well. HDDs would have to be much larger (hundreds of GBs or TB) and this is big cost to add to selling a console. Anyway, just my thoughts.

Post Mortem: Advances in Real-Time Rendering Part 1

Global Illumination

The discussion about global illumination was headed by Hao Chen, lead graphics software architect at Bungie.

The idea of using global illumination in real-time applications has become a very important concept in new games.  If the surface does not emit any light (emissive) the formula used to calculate global illumination is listed.

This formula assumes a BRDF (f) has already been calculated.  This presented the Halo3 engineers with 2 challenges.  The first was that while this formula can be solved for a small amount of simple lights (point lights for instance), it is not feasible to solve in real-time as would be needed by the engine.  The second is driven by the fact that Halo contains many different types of surfaces (shiny, dull, etc).  The Phong BRDF model has been used in interactive, real-time environments but again this was only with a small number of point lights.  Also, the engineers did not feel that the Phong model would capture the detail they were looking for.

So the approach taken was to rely on the CookTorrance BRDF model.  The system created could rely on other models, if others were found to be more accurate.

So the final rendering equation would then be:

There is yet another problem.  The diffuse and specular in the above listed equation involve an integral that is quite expensive to calculate.  If the lights were simple point lights this would probably not be an issue, but as we are using a different type of light source it cannot be used as above.  So the team turned to SH (spherical harmonics).

Specifically, there are 2 cases (diffuse and specular lights).  For diffuse lighting there are shadowed and unshadowed cases that need to be calculated.  Unshadowed diffuse can be calculated in the shader using a quadratic polynomial approximation.  Shadowed diffuse can use pre-computed radiance transfer method result combined (as a dot product) with the incoming light.  There is still one remaining issue to calculate accurate diffuse lighting.  The equation encodes the incident radiance as a single point.  This is not accurate when using to light an entire scene.  Usually, to solve this various random samples are chosen and interpolation used to fill in the rest.  This will give ok results on small areas but not on big scenes.  The other strategy, and one chose here, is to build light maps and grid the scene (and add sample points per cell).  In Halo3, the choice was made to use a photon mapper to "bake" the incident radiance into these light maps.  This is an offline process.

The specular reflectance was much harder to calculate.  The problem is glossy surfaces contain a full range of frequencies.  The choice was made here to break this down to 3 frequencies (low, mid, high).  The high is calculated in the shader (BRDF), the middle frequencies are handled by cube maps, and lows are handled by BRDF again (which is parameterized).

 

REFERENCES:

[BASRIJACOBS03] BASRI, R., AND JACOBS, D. W. 2003. Lambertian reflectance and
linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25, 2, pp. 218–
233.
[BLINN77] BLINN, J. F. 1977. Models of light reflection for computer synthesized
pictures. ACM SIGGRAPH Comput. Graph. 11, 2, pp. 192–198.
[CHEN08] CHEN, H. Lighting and materials of Halo 3. Game Developers
Conference, 2008.
[COOKTORRANCE81] COOK, R. L., AND TORRANCE, K. E. 1981. A reflectance model
for computer graphics. In Proceedings of ACM SIGGRAPH 1981, pp. 307–
316. [GOODTAYLOR05] GOOD, O., AND TAYLOR, Z. 2005. Optimized photon tracing using
spherical harmonic light maps. In Proceedings of ACM SIGGRAPH 2005,
Technical Sketches, p. 53.
[GSHG98] GREGER, G., SHIRLEY, P., HUBBARD, P. M., AND GREENBERG, D. P. 1998.
The irradiance volume. IEEE Comput. Graph. Appl. 18, 2, pp. 32–43.
[HUWANG08] HU, Y., AND WANG, X. Lightmap compression in Halo 3. Game
Developers Conference, 2008.
[ICG86] IMMEL, D. S., COHEN, M. F., AND GREENBERG, D. P. 1986. A radiosity
method for non-diffuse environments. ACM SIGGRAPH Comput. Graph.
20, 4, pp. 133–142.
[KAJIYA86] KAJIYA, J. T. 1986. The rendering equation. In Proceedings of ACM
SIGGRAPH 1986, pp. 143–150.
[KSS02] KAUTZ, J., SLOAN, P.-P., AND SNYDER, J. 2002. Fast, arbitrary brdf shading
for low-frequency lighting using spherical harmonics. In Proceedings of the
13th Eurographics workshop on Rendering 2002, pp. 291–296.
[NDM05] NGAN, A., DURAND, F., AND MATUSIK, W. 2005. Experimental analysis of
brdf models. In Proceedings of the Eurographics Symposium on Rendering
2005, pp. 117–226.
[OAT05] OAT, C. Irradiance Volumes for Games, Game Developers Conference,
2005. http://ati.amd.com/developer/gdc/GDC2005_PracticalPRT.pdf
[PSS99] PREETHAM, A.J., SHIRLEY, P. AND SMITS, B. 1999. A Practical Analytic
Model for Daylight, In Proceedings of Siggraph 1999, pp. 91 – 100, Los
Angeles, CA.
[RAMAMOORTHIHANRAHAN01] RAMAMOORTHI, R., AND HANRAHAN, P. 2001. An efficient
representation for irradiance environment maps. In Proceedings of ACM
SIGGRAPH 2001, pp. 497–500.
[RAMAMOORTHIHANRAHAN01B] RAMAMOORTHI, R., AND HANRAHAN, P. 2001. On the
relationship between radiance and irradiance: Determining the illumination
from images of a convex Lambertian object. Journal of the Optical Society
of America, Vol. 18, 10, pp. 2448–2459. [RAMAMOORTHIHANRAHAN02] RAMAMOORTHI, R., AND HANRAHAN, P. 2002. Frequency
space environment map rendering. In Proceedings of ACM SIGGRAPH
2002, 517–526.
[SCHLICK94] SCHLICK, C. 1994. An inexpensive BRDF model for physically-based
rendering. Computer Graphics Forums. 13, (3), 233–246.
[SLOANSNYDER02] SLOAN, P.-P., KAUTZ, J., AND SNYDER, J. 2002. Precomputed
radiance transfer for real-time rendering in dynamic, low frequency lighting
environments. ACM Trans. Graph. 21, 3, 527–536.
[SHHS03] SLOAN, P.-P., HALL, J., HART, J., AND SNYDER, J. 2003. Clustered
principal components for precomputed radiance transfer. ACM Trans.
Graph. 22, 3, 382–391.
[VILLEGASSEAN08] VILLEGAS, L., AND SEAN S. Life on the Bungie Farm: Fun Things
to Do with 180 Servers . Game Developers Conference, 2008.

Latest reads

Just dropping a line on a book that I am reading.  It is focused on advanced .net debugging techniques and tools.