Can I use a 3D FPS game engine to render and control virtual scenes in real-time?

January 16, 2006 9:30 AM

Can I use a 3D FPS game engine (i.e. Half-Life 2 or the like) to render and control virtual scenes in real-time? Game modders, is what I want to do possible?

I'll preface this question by saying that I'm not a gamer and have little experience with FPS (first person shooter) games, let alone modding them. There seems to be a large community of people who do this, and quite a few tools, both official and third-party, that let you modify things like game appearance, character behavior, in-game physics, etc. I'm also somewhat familiar with machinima and have seen a few videos produced using game engines (Red vs. Blue, WOW videos, etc.) I have a project that overlaps with this somewhat but I don't know if what I want to do is possible.

First, I'd like to create a custom map, very simple, with known geometry. This seems doable from what I've read. Then, I'd like to control what computer-controlled characters are in the scene, and how and when they move. This also seems doable.

I'd like to be able to set up a camera at some known point with some known orientation, and just watch the scene and the characters as they move (I don't care about actually "playing" the game). I'd like to be able to control the camera in some manner I want (for example, panning or zooming) and capture the video that this camera would see. I assume that this all is doable, judging from machinima videos I've seen.

The big question I have is - can I do this LIVE? Is it possible to programatically interact with the "game" as all of this is going on? In other words, could I have the game running, and in real-time control the camera position and orientation, as if I were directing? Could I do this programatically, either through some hooks in the developer SDK or by writing custom code? And finally, could I also get access to the final video output in real-time as well? Think of what I want to do as a virtual camera, where controlling it in real-time would produce real-time changes in the output video.

posted by chos to computers & internet (8 comments total)

I can give you at least a partial answer to your question in that what you are describing is possible.

The ability to "camera" in a multiplayer, FPS game is actually done right now in certain leagues for games like Battlefield 2 - the application I'm referencing is for sportscasting. Two "commentators" will be in the game, moving about, observing the teams playing in a virtual-camera-God-mode. Up until recently, this was audio-only, but lately they have been able to stream out the video in realtime, so that you could see what they saw just by logging onto the feed.

Drop a line on Shanzy at EGLN and see if he can direct you to the software package they use for this. It might work. Depending on your specific needs, you might not have to create anything that isn't "off the shelf."
posted by TeamBilly at 9:43 AM on January 16, 2006

The big question I have is - can I do this LIVE? Is it possible to programatically interact with the "game" as all of this is going on? In other words, could I have the game running, and in real-time control the camera position and orientation, as if I were directing?

Making it "LIVE" is the really big issue because almost all machinima (Red vs. Blue is an exception here, I believe) is done through a 'demo' recording utility wherein the game captures all actions by all players over a set period of time and can play it back later on - to be refilmed, etc. as you wish. BF2 does this, as do most other engines (id Software's, Unreal, etc.)

While it's possible to 'fly around' in God mode for filming, this isn't very useful if you need to - again, LIVE - snap immediately from point A facing one direction to point B facing another. Thankfully, you appear to want to use a custom map, which means you could probably setup a teleport system that works only for the Godmode camera.

If you're willing to delve into the SDK, however, the above is moot because you have full control over the majority of the game logic, including where the camera player goes, and when it goes there.

Using the SDK you could change how the HUD looks and works to create realtime effects.
posted by Ryvar at 10:09 AM on January 16, 2006

Your two obstacles will be "programmatically" and "in real time." It's been a while since I've read about machinima, but a lot of high-quality machinima around the time of Quake 2 used human cameras to record action as well as the use of a server demo (and by "human cameras" I mean "using another player as a camera"). There's no real reason why you couldn't continue to do this; server demos allow the director greater freedom to set up shots after the fact, but at the cost of post-production time and the necessity of scripting to do what you want. Human cameras, by contrast, are easy—point and record, just like a normal film camera.

If you want to program the camera players, this is possible; bots are basically computer-controlled players, but you don't need (or want) anything nearly so advanced; you can, for example, throw out the pathfinding algorithms if you're dealing with a static set and stage directions. Whether anything like this has actually been done yet, I don't know, but it's within the realm of possibility and is probably only moderately difficult. However, it's worth mentioning that some machinima uses human camera players on purpose because camera motion feels more natural that way. If you have a lot of pans, for example, you'll find that programmed camera motions will look very mechanical. On the other hand, things like dolly shots and zooms do make more sense using programmed cameras (although why you'd want to use zooms for any purpose other than to change focal length between shots is beyond me).

The live part, however, will be hard to do in game. Cuts are going to be the most difficult problem, but there's a neat way around this. You mention output video; does this mean you'll be outputting to some sort of televised live feed (as in, something outside the game)? If so, you could simply resort to the way most multi-camera live shows are produced: use multiple camera players, set them up however you like, and then when you need to cut to another shot, switch the feed from one camera to the other. Since everything's being piped to a central server before being turned into video, I don't see why this would be all that difficult to do. This would all be handled at the stage where you transform the game scene into video.

For that last part, recording video from the game in real time, I'm not sure what software exists to do this sort of thing, but FRAPS is a good place to start. You may also want to look into the way LAN tournaments such as QuakeCon (does that still exist?) display tournament matches for the rest of the audience. Presumably they don't just point a camera at each opponent's screen, since that would look horrible.
posted by chrominance at 10:29 AM on January 16, 2006

To even attain the basic semblance of a functional scene (in HL2, at least) this would require phenomenal amounts of work on behalf of an individual author. Being able to modify these parameters on the fly is a big impossibility, aside from what others have mentioned as a user-controlled camera. For example, there'd be no way to interface with character paths or animation trigger sequences once the level is compiled and executed. Then again, you have that layer to consider... all animations are handled inside the models, you only set up basic path and sequencing in the level editor itself. The only realistic level of interaction you could have is by setting the cheatmode on the server and using prebound keys to switch between cameras you created earlier in the level. Most LAN events have clients connected directly to the server and they function "in-engine" but have no effect on the game itself. You would need to have a full set of resources installed on whatever remote machine you'd be displaying this with as it would be connected and rendering as a client. AFAIK, there is no reasonable way to "stream" any quality video feeds directly out of an application, let alone HL2 at 60+FPS. Even if you have the willpower to learn Hammer and Max 8, along with a host of export/import plugins, I have my doubts this will even come close to remotely living up to your original concept.
posted by prostyle at 10:58 AM on January 16, 2006

I've never played it, but The Movies is a game specifically designed for making machinima.
posted by breath at 10:59 AM on January 16, 2006

For Half-Life 2 itself, check out Garry's Mod. It should be able to do most, if not all, of what you need; I haven't used it myself but I have seen some movies and stills made with it, and it's extremely versatile.
posted by cyrusdogstar at 12:36 PM on January 16, 2006

Put it this way:

Modding a game like you're think of doing, is the easiest way there is to do what you want to do.

That doesn't necessarily mean it will be easy, just that it will be easier than trying to do it some other way. A game engine will also give the best looking realtime graphics.
posted by -harlequin- at 1:39 PM on January 16, 2006

There seems to be a bit of confusion over exactly what you want to do with the game engine.

I only see:
> Then, I'd like to control what computer-controlled characters are in the scene, and how and when they move. This also seems doable.

The only modern game engine I'm familiar with at all is the Half-Life 2 engine, but yes, doing this part would be fairly straightforward simple. Keep in mind that if you don't plan on using HL2's built-in characters, you'll have to make your own (which can be a rather complex procedure, depending on how much detail you want).

The camera bit is easy. Check out Garry's Mod (mentioned above) if you already have a copy of HL2 -- it's grand fun.
posted by neckro23 at 3:06 PM on January 16, 2006