Source Engine

}}
Developer:	Valve
Written in:	C++
Platform:	Microsoft Windows, Playstation 3, Xbox, Xbox 360
License:	Proprietary
Website:	http://source.valvesoftware.com/

The Source engine is a 3D game engine developed by Valve corporation. Its unique features include a large degree of modularity and flexibility, an artist-driven, shader-based renderer, accurate lip sync and facial expression technology, and a powerful, efficient and completely network-enabled physics system (provided by Havok Physics).

Source supports both 32-bit and 64-bit environments and the Microsoft Windows, Xbox, Xbox 360, and PlayStation 3 platforms (the last being launched with The Orange Box). It debuted in October 2004 with Counter-Strike: Source and Half-Life 2.

Technology overview

The Source engine allows for improved graphics compared to those of its predecessor GoldSrc.

(For a full overview, see Source Engine Features on the Valve Developer Community Wiki.)

Rendering

Shader-based

Up to DirectX 9 with Shader model 3.0

3D skybox; an area can be displayed as a skybox with full 3D parallax, at a default scale of 16x.

High dynamic range rendering (HDR)

Animation

Any animation can merge seamlessly with any other animation at any time.

Inverse Kinematics to ensure that characters' limbs react to environments

Networked physics

Originate from Havok 2 physics engine

Highly-tuned by Valve

Processor-efficient

Software only

Fully networked with low bandwidth requirements

Vehicle physics including torque, power, gears, tire material, suspension and mode.

Audio

Full DSP system

Full Surround sound support

Software only

Scalability

Supports DirectX 7 and upwards (until Half-Life 2: Episode One). But GPU must render DirectX 8.1 or better (9.0; 10.0) for the Source Engine 2007 (Half-Life 2: Episode Two, Portal and Team Fortress 2

Modularity allows for all current and future Source projects to scale back to DX7 or DX8.1 (Source 2007) if they desire.

Facial expressions

Full range of human and non-human facial movements

Based on Professor Paul Ekman's Facial Action Coding System

Over eighty-four "digital muscles"

Lip-sync

Works in tandem with facial expressions

Auto-generated, but completely configurable

Stored in sound file itself for localization

Modularity and notable upgrades

Source is designed from the ground up to be highly modular. This allows for the easy upgrade and modification of certain features without breaking other areas of the engine, or breaking engine continuity (that is to say, there need be no 'version jumps' from 1.0 to 2.0). When coupled with Steam, these updates can be distributed retroactively and automatically. For instance, if Source is upgraded to support hardware physics, every Source title on Steam will instantly benefit. Entirely new features such as High Dynamic Range (HDR) Rendering have been shown to require developer input, however.

High dynamic range rendering

First seen in Day of Defeat: Source, then shortly after in Half-Life 2: Lost Coast, HDR was the first major instance of Source's modularity in use. However, whilst in theory all Source engine games and mods were able to use HDR immediately after its release, the game code required to 'hook in' to the new system was not made available to modders until eleven months later, on August 4, 2006. Official licensees and Valve themselves have all made use of the technology since its release.

Facial animation 2

When Half-Life 2: Episode One was released on 1 June 2006, it introduced the second version of Valve's facial animation system. Ken Birdwell explains the upgrade's features in the game's commentary track:

When we designed the Half-Life 2 facial system back in 2000, our goal was to get a natural-looking performance at a moderate distance. For Episode One, we wanted to extend the characters' facial systems to support more intense performances with a wider range of facial expressions, that would hold up better at close range. These facial improvements included increasing the detail around the eyes and mouth, increasing the number of facial shape targets — think of these as movements of muscle groups — by about 50%, rewriting the rules that control how these shapes blend, and increasing the intensity of many of our existing shapes.

Facial animation 3

The 14 May 2007 Meet the Heavy Team Fortress 2 trailer revealed the third iteration of Source's facial animation technology, hardware accelerated on video cards. Valve's accompanying press release reads:

We are pleased to announce that the next generation of facial animation technology for Source will be released in Team Fortress 2. By leveraging the rapid pace of advancement in PC graphics, videogames can now match and actually surpass the facial animation used in feature film and broadcast television production. We've released a video of this real-time, in-game, in-engine facial animation in action in support of ATI's launch of the Radeon HD 2000 series. By running entirely on the GPU of the videocard, the facial animation not only looks dramatically better, but frees up the CPU to perform other tasks like physics or AI.

The animation system was first released in Team Fortress 2.

Soft-Particle system

During the July 2006 Electronic Arts Summer Showcase press conference, Gabe Newell mentioned that a new "soft-particle" system will be introduced into the Source Engine in Half-Life 2: Episode Two. It was first demonstrated in the July 19 Team Fortress 2 teaser, which showed a remarkably realistic flamethrower in its closing moments. The system is primarily designed for multiprocessor systems.

Full-screen motion blur

Team Fortress 2 was one of the first Source games to offer full-screen motion blur. The blur is not the same as the "cinematic" blur used to render Valve's various trailers, and only appears when the player's view is moved at a high speed (e.g. when spinning around or falling). The same effect was also added in Portal and Half-Life 2: Episode Two.

Dynamic lighting and shadowing 2

A new dynamic lighting and shadow mapping system has been developed for Source, complementing the once limited existing system. It launched with the various other new Source features that were developed for Half-Life 2: Episode Two.

In previous Source games, "dynamic" shadows (primarily shadows cast by physics props) would fall in a predetermined direction, manually set by the level designer. With Dynamic Lighting and Shadowing 2, a similar system is employed; props still usually cast shadows in one general direction — but technology has been developed that allows a level designer to enable certain light sources to cast completely dynamic shadow maps. Currently, only one of these shadow-casting light sources is allowed to be visible to the player at one time, a limit imposed at the engine level.

Another improvement with the Dynamic Lighting and Shadowing 2 system is the introduction of flashlight shadows. In previous Source games, the player's flashlight only illuminates an area; with Lighting and Shadowing 2, the flashlight casts dynamic shadows from and onto all nearby objects and geometry, including translucent materials such as chain-link fences, trees and foliage.

A breakdown of the new lighting system's capabilities is shown below:

Lighting and Shadowing system comparison

Original Lighting and Shadowing System	Dynamic Lighting and Shadowing 2
Realtime shadows in a map always cast in the same predetermined direction.	Realtime shadows in a map cast in a predetermined direction unless near a light source specifically enabled to cast realtime dynamic shadows in the level editor.
Shadows cast by models do not self-shadow or affect other objects.	Shadows cast by models can self-shadow and cast shadows across other objects, if near a light source specifically enabled to cast realtime dynamic shadows in the level editor.
Every object is allowed only one dynamic shadow.	Any object can cast multiple dynamic soft shadows, if near a light source specifically enabled to cast realtime dynamic shadows in the level editor.
The player's flashlight illuminates an area.	The player's flashlight casts shadows from models, world geometry, and transparent surfaces onto other models and geometry.

Landscape and Flora Rendering

Large, open natural environments with heavy foliage, traditionally a weakness for the Source engine, is supported as of Half-Life 2: Episode Two. Valve is using alpha-to-coverage for anti-aliasing the alpha textures used to approximate the leaves of foliage.

Cinematic physics tool

Contrary to expectations, "Cinematic physics" are non-interactive animations "baked" (calculated during development rather than in real time) into one-off model files. They are created with an offline physics tool built by former Weta Digital employee Gray Horsfield, which is not currently available publicly.

Multiprocessor optimizations

As a part of the Source engine's transition to next-generation consoles, multiprocessor optimizations have been added, resulting in faster processing on PC hardware with dual or quad core systems and the Xbox 360 and PlayStation 3 consoles.

Gabe Newell:

Yes. We definitely think that content needs to move forward. For example, one of the things we're reacting to is the speed at which microprocessors are coming out. So, Intel has very aggressively moved up delivery of desktop processors with four different cores; we'll have support for that in Episode Two, and we'll definitely go back to affect, you know, Episode One or Half-Life 2 or Counter-Strike Source, so they can take advantage of that. We'll definitely try to keep the existing games — especially the multiplayer games — current as technology evolves.

Valve has demonstrated the new multi core optimizations, which use a multi-threading style they dub "hybrid threading." A Source multi-threading update and benchmark are expected to be released at some point, though a date for either and the content of the benchmark are currently unknown.

Half-Life 2: Episode Two and Portal appear to be the first Source-engine games to employ these new multiprocessor optimizations.

Deformable physics bodies

The Orange Box introduced support for physics-based animation for simulation of things such as floppy clothing and flexible hair ("jigglebones"), and better blending of ragdoll physics with predefined animation. Previously, Source's physics system could not deform objects except between bone connections.

In-game development tools

At release, Team Fortress 2's cache files contained leaked code for several in-game development tools:

Source Filmmaker

The tool used to create Valve's promotional trailers. Users can record themselves as many times as they want and overlap the results, creating the illusion of there being many actors in the world at once. A "Publish to Steam" button is available from the main menu. Supports all of the engine's cinematic effects.

Actbusy editor

Another script-file IDE, this time used to tweak the game or mod's AI Actbusy routines.

VCD Block Editor

Another IDE for editing predefined animations for the face poser tool.

The leaked code has since been both removed from the cache and disabled from running should the user supply backed-up versions of the various libraries.

Cinematic effects

With color correction and film grain already released, Valve intends to add other cinematic effects such as depth of field to Source when hardware is able to render it to their satisfaction. The effects are accomplished with an accumulation buffer for quality, creating enormous overhead; for instance, twenty to thirty motion blur frames need to be rendered for every one frame that the user sees. For a constant frame rate of thirty frames per second, a video card is required to produce between six hundred and nine hundred frames per second. This causes late 2005-era hardware to require a full two seconds to render each frame.

Motion blur and depth of field can be seen in several of Valve's promotional videos, including:

• The Day of Defeat: Source video Prelude to Victory
• The initial Portal trailer [1].
• The Team Fortress 2 trailer Meet the Heavy

Image-Based Rendering

Image-Based Rendering is a technique in which 2D elements are manipulated to appear in a 3D world. In the context of a 3D game, it delivers a significant performance boost by replacing 3D geometry that is far enough away for the transition to be imperceptible with a 2D image. Implementation of the technology in this role can be found applied to Rome: Total War's soldiers, Far Cry's forests and various objects in Guild Wars' environments, such as buildings and flora.

The technology had been in development for Half-Life 2 but was cut. It was mentioned again by Gabe Newel] during 1UP's 'Valve week'

There's this technology that was really exciting that I’d like to see us get into production, which is a different approach to rendering complexity: Moving things into and out of an image domain and then seamlessly interpolating between those motions as you move around. So that everything close to you is physical and geometry, and everything really far away from you is an image, but you have no way of telling that if you do it properly and things can fly out and come back.

The June 2006 Day of Defeat: Source update included an "image-based texture blending shader and Half-Life 2: Episode Two's expansive environments seem ideally suited to the technology, but it has yet to be dated or even officially announced.

File streaming

One of the technologies developed for Half-Life 2's Xbox release was file streaming, wherein a map's resources could be loaded as the player moved around in it rather than in one operation before playability. With the system in place, loading times were reduced to as little as fifteen seconds. The system expanded on the caching system already implemented. There is no time frame for its release, as implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems.

Origins

Source distantly originates from the GoldSrc engine, itself a heavily modified version of John D. Carmack's original Quake engine, as is explained by Valve employee Erik Johnson on the Valve Developer Community wiki.

When we were getting very close to releasing Half-Life (less than a week or so), we found there were already some projects that we needed to start working on, but we couldn't risk checking in code to the shipping version of the game. At that point we forked off the code in VSS to be both $/Goldsrc and /$Src. Over the next few years, we used these terms internally as "Goldsource" and "Source". At least initially, the Goldsrc branch of code referred to the codebase that was currently released, and Src referred to the next set of more risky technology that we were working on. When it came down to show Half-Life 2 for the first time at E3, it was part of our internal communication to refer to the "Source" engine vs. the "Goldsource" engine, and the name stuck.

Source was developed part-by-part from this fork onwards, slowly replacing GoldSrc in Valve's internal projects and explaining in part the reasons behind its unusually modular nature. Valve's development of Source since has been mixture of licensed middleware (Havok Physics, albeit heavily modified) and in-house-developed code. The amount of Quake code remaining in Source is a point of contention. John Carmack commented on his blog in 2004 that "there are still bits of early Quake code in Half-Life 2" and numerous development interfaces are similar to Quake's, at least at a high level - but with both easily likely of being Valve avoiding reinventing the wheel, neither are indicative of a notable reliance on the older engine. Indeed given the fact that Source's 2003 code leak did not produce any such claims it can only be assumed that no incriminating evidence was to be found.

Source distances itself further from Quake as time goes on. The multiprocessor update in particular made major architectural changes.

Common issues

Stutter

The Source Engine uses a caching system, whereby the loading of certain resources is handled and managed on the fly, rather than in a single operation behind a traditional loading screen. Texture and sound data are the primary areas in which this occurs. Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 125 milliseconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested. If either system is held up the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.

Stuttering is system-dependent issue, making it hard to say when it was resolved. However, since an engine update on 2006-02-07 that featured a limited implementation of Source's full file streaming system from the Xbox, complaints have ceased on the official support forums.

While stutter could be caused by poor system performance, it was also noted on hardware that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause has never been fully understood by the public. Most third-party solutions involved bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).

When Half-Life 2 was first released and stuttering became a widely-known problem, community member Mark McWilliams set up a page covering the issue and Valve's communication and work on resolving it. Several updates were released by Valve, the effects of which varied from complete fixes for some users to previously smooth systems becoming "infected" with the problem.

The hitches coincide with the drawing of new areas (a process that is by design not visible without debugging tools).

Looping audio

The Source Engine suffers from an error whereby the asynchronous loading (see Stutter, above) of a new sound file will cause the engine to lock up with looping audio. Because of the nature of DirectX, once the engine enters such a state it will remain on the screen unless the user blindly terminates the program or reboots their computer. The error occurs in a standard Windows library associated with on-board sound cards, and in some cases can be resolved by decreasing DirectSound Acceleration.

Reports of looping audio crashes increased around the release of Episode One. While it is likely that the spike was simply caused by an unusually high number of players, changes to the engine, of which there were many for the game, cannot be ruled out. It has been noted that people who have been able to play Half-Life 2 without any crashes and audio errors in fact do find Episode One to be more prone to that problem. A steampowered.com forum thread is dedicated to discussing the problem and attempting to work out solutions, although the experience of the thread contributors indicates that most suggested workarounds and fixes do not remedy the problem.

Toolset

The Source SDK tools are criticized for being outdated and difficult to use. A large number of the tools, including those for texture and model compilation, require varying levels of text-editor scripting from the user before they are executed at the command line with lengthy console commands. This obtuseness was cited by the University of London when they moved their exploration of professional architectural visualisation in computer games to Bethesda Softworks' Gamebryo-based Oblivion engine after a brief period with Source. Third-party tools provide GUIs but are not supported by Valve.

Valve's Hammer Editor, the SDK's world-creation tool, does have an interface, but one that has not changed significantly since its initial release for GoldSrc and the original Half-Life in 1998.

Before the Valve Developer Community Wiki was opened the SDK's user documentation was also often lacking.

Programming

Unlike most modern game engines, Source requires all program code to be written in C++. This is in particular contrast with the Unreal engine and its UnrealScript language that can be (and is) used to write most game rules in titles developed with it.

In response to this, several Source engine games and mods (including Garry's Mod, Natural Selection 2, Fortress Forever and Smod) have implemented the Lua language. These implementations are not available in either the consumer or licensee SDKs however, and as a result the development communities each have formed (which include entire games and mods running or being developed inside them) cannot migrate to the Source engine in general.

Technology

While the Source Engine may provide some advanced technology such as lip syncing, its rendering techniques have been some what scrutinized for being out of date and old fashioned, in comparison with other Game Engines, such as Unreal Engine 3, or the CryEngine 2 it fails to deliver on ground breaking detail that is offered in other current generation game engines. It also fails to be fully compatible with current generation hardware, not fully supporting or exploiting any multicore processor, and offers no official support for the Playstation 3's Cell Broadband Engine.

Valve Developer Community

On June 28 2005, Valve opened the Valve Developer Community Wiki. The VDC replaced Valve's static Source SDK documentation with a full MediaWiki-powered community site. Within a matter of days Valve reported that "the number of useful articles [had] nearly doubled". These new articles covered the previously undocumented Counter-Strike: Source bot (added by the bot's author, Mike Booth) and Half-Life 2 AI, source control for Source engine mods, and more.

See also

• Source SDK

External links

• Source Engine Features – Valve Developer Community
• The Valve Developer Community