Civilians

[Alien-Bait]

Good jesus 2000 reloads!? Do you have a day job!?

lol that sounds funny

Alien #1: I think he doesnt see us

Alien #2: yep let get him, he wont shoot anything explosive at us, there might be civvies * *

Soldier: ehhh, there arent any civvies, so I'll just shoot HE shells at those voices

Both Aliens: awwww.... shit

[Hobbes]

It was pretty funny walking around a "supposed" terror mission firing off explosive rounds here and there, without a second thought of saving a civilian! It was like an Arnold Schwarzenegger movie: I see an alien to the left, and blast away at it until it was dead. I see an alien to the right, and I blast some more.

You mean they're not like walking furniture?

[Bomb Bloke]

I kinda sorta take that attitude too... If a civy dies, so be it.

 

Though I don't use explosives (much), as it makes the game far too easy...

[Zombie]

Good jesus 2000 reloads!? Do you have a day job!? 

Do I have a day job? The question should be how many day jobs do I have! Right now, I have three. When I go back to college in the fall, I will have two jobs (not including school). This works fine for me: presently, I have no student loans, and all my schooling costs are paid for by my jobs (with money left over).

 

Granted, I totally agree that 2000 reloads sounds a little excessive. 1500 reloads would probably suffice for the general X-COM community, but you have to remember one thing: I am a perfectionist (to a point). I do know when to hang it up and accept the outcome. If 2000 reloads does not give the data I hope for, I will quit then and there and report my results.

 

The limiting factor for the number of reloads I can do in a time period is dictated by how many "clicks" it takes to get from the end of one mission to the start of another one. At the moment, it takes 10 "clicks" to get one result. 2000 reloads equates to 20,000 mouse clicks! After reloading 200 or so times, my hand starts to look like a claw! This is my max per day (when I get time). Actually, when I get down into a rhythm, 100 or 200 reloads do not seem to take that long. Until I look at a clock that is, and it shows 2 hours went by!

[Blehm 98]

O_o 2000 reloads? You must have like ultrafast reaction time for your hand, and your hand-eye coordination must have increased something. i mean, wow, 2,000 reloads....

[Zombie]

Ahhhh. Results at last! My 2000 reload list was completed with little fanfare from any of my friends so I celebrated alone with a bottle of a good micro-brew beer. I continue my celebration here with the results.

 

First, the statistical data:

MINIMUM:       0
MAXIMUM:      16
MODE:         12
MEDIAN:       11.5
AVERAGE:      11.45
RANGE:        17
StdDev:       2.76
TotalCount:   2000

 

And next, the distribution tallies:

# of Civies    Count     Smoothed     % of Total
    0            6           6           0.30%	
    1            3           4           0.15%	
    2            4           4           0.20%	
    3            4           6           0.20%	
    4            11          8           0.55%	
    5            10          12          0.50%	
    6            15          17          0.75%	2.65%
    7            26          85          1.30%	3.95%
    8            215         155        10.75%	
    9            224         225        11.20%	
   10            236         235        11.80%	
   11            246         247        12.30%	
   12            259         246        12.95%	
   13            232         228        11.60%	
   14            194         197         9.70%	
   15            165         170         8.25%	
   16            150         150         7.50%

For those of you who are interested in viewing a graphical result, I included "smoothed" data set. The "smoothed" column is simply the average of the counts above and below the current value (for instance, the smoothed value of 246 for 12 civies is found by averaging the counts of 246, 259 and 232). Smoothing helps to show a Gaussian data set better than just a plain count.

 

The data shows a definate rise from 0 to 12 civilians (the average) and a decline to 16. Overall, you should expect to see anywhere from 8 to 16 civilians at a Terror Site a whopping 96.05% of the time! Less than 1% (actually 0.85%) should you expect anywhere from 0 to 3 civilians! Do you see now why it was so tough to play a 0 civilian Terror Mission? They only happen 0.3% of the time!

[Pumpkinhead]

Kudos!

 

I have to say, you have a lot more persistance than I do - I would have given up after about 10 reloads

[Zombie]

Thanks buddy!

 

I am very persistant, but I am also very patient. When there is an incentive (like gathering a piece of information that nobody else would even contemplate doing), I'll work till hell or high water trying to find the answer. It is the satisfaction of a job well done that keeps me coming back for more!

 

I hope my data answers your questions about civies at a Terror Site.

[Bomb Bloke]

Dude... 2000 reloads?!

 

Look, I know you said it before, but hey.

 

Would anyone be interested in me having a go at the game files myself?

 

As far as I can see, whenever you exit a tactical game, you change to a different EXE file. To pass the information between these files - such as, for example, the equipment cpatured, civilians killed, etc - it would either have to be passed as parameters, or stored in some temporary location.

 

It might well be possible to write an automated program that loops maps, and records information about them. Capturing the information should be a breeze.

 

The only difficulty I can think of is how to get the program to automatically exit the tactical game. Sure, it'd nearly half the time it'd take to loop maps even without that feature, but you'd really want something that didn't need a human to sit there and watch it at all. That way, hundreds of maps could be done per minute, on a modern system.

 

Other problems would involve the equipment screen, and other places you have to click to continue... I don't know how those parts of the program are structored. Writing code to work around them might involve some more difficult reverse engineering, and if I was to go that far, I might as well just document every little aspect of the game while I was at it.

 

I presume that's already been done, but it would take the fun out of things, anyway.

[Zombie]

Yep, I may be insane, heck, maybe even stupid to do 2000 reloads.

 

Your idea of automating the information gathering aspect sounds good. If I were able to write/use a program to find out how many civies were at an average terror site, I would have jumped at the chance! Sadly, I do not know much about the language X-COM is programmed in. If I did, I would buy a textbook on the subject and teach myself. It all boils down to this: nobody could write such a program except maybe the original programmers of the game. Even then, why would they want to when they already know the answer!

 

Do you think that doing 2000 reloads is fun? Hardly! Seriously, I have better things to do than sit in front of a computer and reload time after time after time. Like I said before, it is the incentive and solution that is alluring.

 

When a question is posed, and the answer is unknown, does anyone step up to the plate and tackle the problem?

*looks around* Not many people I see. Maybe everyone is too busy or they just do not care about the answer much to do some research. All I know is that when a question is asked, I make that query a #1 priority on my list of things to do. And when I resolved the mystery, I gain a sense of accomplishment knowing that someone is no longer confused about what to expect. If this entails 2000 reloads, then so be it!

[Bomb Bloke]

I don't have the source code, so it doesn't overly matter what the original was written in... My decompilers are sub-standard, and I'd presume it illegal to try them on the game.

 

Luckily, I doubt I'd need them for my purposes.

 

A closer look at the other UFO tools might ease the work, but who knows? I might be able to make something in as low as a few hours.

[Pumpkinhead]

I just plugged your numbers into Excel and it's certainly an interesting curve that comes up

 

Do you think they would have made it a random number based around 11 or 12 and then just constrained the upper number to 17 due to object limitations?

[Bomb Bloke]

I would expect the civilian counts to be based on the tiles used to generate the maps. Which would make it very random as to what counts you'd get, but there would certainly be a minimum and maximum count.

 

The average civilian count would be based on the average tiles a map can be made of.

 

Concerning the looping thing;

 

The game flips between tactical and geoscape gamefiles, as I'm sure you know.

 

A save game on the geoscape results in your geoscape stuff being saved.

 

A save game on the tactical map saves all that a geoscape save would, plus any map specific data.

 

When you start a tactical game, a save game is made to the missdat folder, which includes all of your geoscape information, plus the tactical map which is to be played. This means it is the geoscape game file which generates the map. The first thing the tactical gamefile does is load the information it will use for the mission, then ask the player to equip the soldiers before the game starts.

 

Now, the entire game is managed by one little batch file, which switches between the two game files until the user quits. It should be possible to create an edited version, which allows you to quickly loop maps. My idea is, before the mission starts (about a second or so before your craft reaches the destination on the geoscape), you save the game. When the tactical game starts, my batch file throws that save game into the missdat folder, thus placing the skyranger in front of the mission again. Thus, when you quit the tactical mission, you don't have to load the game, as it has already been 'loaded' for you.

 

When the tactical map ends, it must place the outcome of the mission into some file or other, so the geoscape can work out what happened and how that affects X-Com (your score, items, etc). If I can work out the format of that file, it would be simple enough to write a program, throw it into UFO's game loop, and have it log the outcome of each and every tactical mission that you loop.

 

Thus, all you'd have to do is click the start mission button, the ok button on the troop select screen, the abort mission buttons on the tactical map, and the ok button on the score screen.

 

That still requires user input, and it still adds up to a lot of clicks to do a few thousand loops, but you'd no longer have to load the game (that would be automated), nor would you have to write down the information you wanted yourself (it would all be logged for you).

 

Thus I reckon loops could be done more then twice as fast. The catch is you'd need to use a specific save file each time you wanted to use it, but that's not asking much - I believe UFOUtil asks for about the same.

 

There is another catch with this method, and it's a big one. The ONLY THING I would be able to log from missions ended instantly, would be the amount of civilians on a terror map. If you quit a normal map, you only get told you scored 0, because you didn't do anything.

 

The other solution is to hack through the map format. If I can work out how the maps are stored, then I can see how each and evey item and unit is placed. That means that the tactical game file becomes rudundent; I could remove it from the game loop and simply put my logging file in the gap. That would require a lot more work for me, however, as I'd need to work out the more complex map files, and write code to translate them, but it would mean that all you'd have to do would be to click the start mission buttons, and ok on the score screen. That's about three mouse clicks per mission; and it would add up to very fast looping.

 

That method would be able to tell you map size, alien counts, spawn points, alien load-outs, civilian counts, if any/all power sources survived a crash, how many aliens died in a crash, etc.

 

Perhaps a map modder type guy could show me around the map format. I'm sure Hobbes knows a thing or two about it.

 

I could also make the logging program display some figures as it went. For instance, the amount of loops done thus far. And of course, a viewing program would come with it all, one which could make graphs and so forth out of the information gathered.

 

Either of the two solutions would not affect normal gameplay, running the original batch file would result in the game playing normally. The looping batch files would be seperate. I reckon that's about the fastest I could make it without actually reverse engineering the game engine, but again, if I did that, there wouldn't be any secrets left.

 

I have been pondering having a go at the engine for a while now, actually... The problem is, it would require a rediculous amount of effort. Actually, that's an understatement. I doubt I could do it at all without some serious research into machine code, plus a few solid years to get it all sorted out into some reasonably readable source code (Computer programs are more or less written in text, in a series of logcal statements. Computers can't read this text as programs, so this 'source code' is 'compiled' into real computer code, which is pretty much unreadable by all except the most patient humans).

 

The benefits would be complete knowledge of how the game worked, meaning I could upgrade it to include extra tilesets (without replacing old ones), extra planets to fight over, real time mode, bigger and better research trees, support for mod packs, etc. It would be great. It would also be a heck of a lot of work, but I reckon the end result would be so fun, it would be worth it.

 

I typed the above into notepad while working on the game files themselves, but now I'm back online, it seems every man and his dog has already broken all the files I spent the afternoon working on. Which is good, I suppose.

 

I managed to get some idea of how the MAP files are set out, each tile on the map gets a whole 32 bits to itself. I couldn't quite work out what those 32 bits MEANT, but I tried to take the map file and work out how it linked to the PCK files. My attempts to convert PCK files to GIF files resulted in very pretty colors, but no furthar understanding of the layout.

 

It would seem Daishiva knows about PCK files... I'll bug him.

[NKF]

I'll help you a bit there. Each tile consists of 4 bytes (your 32 bits).

 

The first byte represents the ground tile.

 

The second byte represents the walls that go along from north to south. The wall sits on the on the left edge of the tile.

 

The third byte is the same, but for walls that go east to west and is along the top edge of the tile.

 

The last byte represents the object space. Well, object space as in trees, furniture, dirt blocks, and the 'solid' walls that you can throw things into, walk through, and levitate into.

 

This is for the large map block that gets pieced together for the battlescape itself.

 

For the individual map modules, the first three bytes in the file are the module's dimensions. X, Y and elevations. Although the pck references used here are different from the one in the battlescape map file. Each layer in the map takes up 4 * x * y bytes. For every elevation, there should be one map layer.

 

- NKF

[Bomb Bloke]

Uh, some of that is a little vague for me... Comparing that with the notes I found on daishiva's site, I'll try and interpret your description of the four bytes back at you.

 

The first byte is the ground tile... Does that mean a mapping to the picture to be used for the tile? If not, what is it?

 

The next two bytes show the walls... somehow. I don't really get that. Daishiva says the byte order is ground, west, north, content...

 

Ok, so lemme get this straight. As far as I can see, a tile could have up to six walls. One one each side of the compass, one on the bottom, and one on the top. The bottom wall ('floor') is made redundant by the wall on the top ('ceiling'), so I guess that makes five. It would seem less then a byte should be needed, all up... Five bits.

 

Now, I've got a binary dump I made of the avenger.map file in front of me... The top of the avenger (the roof) seems to have no wall information. So that means there are no floors, only tiles you can't fall into. Oops, that can't be right either, as there is no 'roof' wall information on the bottom layer, either. Arg. Just one byte per wheel (accompanied by three empty bytes), and six more for the ramp. (also accompanied by three empty bytes).

 

But I degress, my point is I don't understand the two byte wall format.

 

Actaully, now I read through it, if those middle two bytes are wall information... then only two tiles in the avenger have any wall information. The windscreen. However, I don't have access to a normal battlescape map at the moment, so I can't compare... (I just got my text version of the bit layout of the avenger.map file).

 

Which seems... odd. The file seems to be filled with a byte, two empty bytes, another byte, repeat until map complete.

 

And I really don't get the content thing, either... I figured that tiles would need some sort of space information (ramps and stairs boost your soldiers in the air), would that be it?

 

Some tiles have no 'ground' or 'wall' information, but they do have 'content'.

[NKF]

Well, each tile is just made up of four elements.

 

The floor (which doubles as the ceiling of the tile beneath it). If it's empty ( 0) it's just treated as air and can be passed through. If set to some other value, it just a references to the image to display at that tile location. Normally you'd point a floor tile to it.

 

The horizontal wall and the vertical wall (Sorry, when I said north-south, east-west, I meant when viewed from the overhead map) . Each tile only has these two walls. The adjacent tile will provide the wall to box the previous tile in.

 

The last bit is the space where furniture and other static objects like walls and dirt blocks go. You'll notice that the UFO walls along the west (left) and north (top) will go in this space. I think the right Skyranger wall uses this space as well.

 

- NKF

[Bomb Bloke]

Ok, so byte one says what floor tile is used.

 

Byte two shows one wall, byte three the other. I'm guessing that this doesn't just say, 'here be a wall', but rather, 'here be a wall of xxx type'. Am I right?

 

The forth byte stick an item in the square, which means you could take a chair from a ufo and place it on the ground outside (as the furniture does not consist of floors).

 

So I'm guessing another piece of 'furniture' might be stairs, which allow you to walk up and down. Other furniture (such as dirt, or chunks of the avenger) would prevent you from entering the square at all, while smaller things (like a chair) would simply act like smaller squares, depending on the statistics of that 'furniture'.

 

So I guess any of these four bytes might map to the PCK file for image data - byte 1 ask for x floor, byte 2 asks for x wall, byte 3 asks for another wall, and byte 4 asks for something else (furniture or smoke, etc).

 

Ok... If that's all correct, I reckon I should be able to decode those files with ease now. Thanks.

 

(Presuming I can find where the stats for stuff like stairs and the like are kept, as opposed to the image data. )

[NKF]

Just some additional info that might help:

 

Remember when I noted that the first three bytes represent the map module dimensions: These are for the .map files in the maps\ directory.

 

The one in the battlescape, which is in your savegame and missdat directories has its dimensions stored in wglob.dat (only available in a battlescape save - naturally).

 

The first two bytes (an unsigned short integer) are the number of turns elapsed so far (yours + the aliens). This is what your grenade timers reference to check whether or not it's time to explode.

 

The next two bytes (another unsigned short integer) counts the number of game turns so far (basically the same as above but divided by two).

 

The next six bytes (or three unsigned short integers) will be the map dimensions and elevation. Unsigned shorts... that means if they had the technology back then, and if only the engine was built for it, we'd be able to get up to 65536 x 65536 x 65535 maps to work with! That'd be enough for a decent sized city plus you'd be able to get well past the earth's atmosphere as well! ARGH. I do NOT want to think about searching for the last alien in such a monstrosity.

 

The rest of the file might have a lot of information that I'm not really certain about at the moment. Some other bight people out there will know for sure.

 

So if you just want to check the battlescape map sizes, wglob.dat is the file to look at.

 

Still, I'm not entirely certain that the size of the map will determine civilian spawn amounts.

 

- NKF

[Bomb Bloke]

ARGH. I do NOT want to think about searching for the last alien in such a monstrosity.

Heheh... You'd want the 3D blaster bombs from the PocketPC version, plus Apoc's physics (which allow you to level buildings with ease).

 

"I think he's in there, sir"

 

"That's a twenty story building, soldier. You'd better not have lost him."

 

"No problem, sarge. One missile into the base there, and -"

 

FWOMPH!

 

*building collapses*

 

*death cry of alien is heard*

 

Don't mind me. Anyway, so I presume the forth byte of the chunks in the map files maps to entries in some sort of 'furniture' file? And that file maps to images?

 

And the second and third bytes map to walls, which I presume also have their own file, along with mappings to images?

 

I'm just wondering how the battlescape map might referance these things. I presume once that map is built, it 'forgets' which units were made to create it.

 

Hrm... That might actually take me a bit of research, as to how they translate. The other thought is that usually the craft are indestructable, but in the mountains I hear that is not always the case... I haven't tested that myself. (I'm shamed. I must be the only one. )

[Faskalia]

Hmmm seems to be a rather harsh discussion.....

 

But one thing really concerns me: NFK's second reply to this thread.

Is it really possible to force Crissalids to aid you, or was it just a missunderstandig?

 

 

Quoting:

Look at the X-Com Chryssalid. A mind controlled Zombie will have its allegience set to X-com and the flag that tells the game that the unit is under mind control will be set. If this zombie is killed, its allegience will default to alien after the chryssalid hatches. But the flag that tells the game that the unit is under mind control will still be set. So, at the end of the turn, well, you get the idea.

Ending Quote.

 

Sounds for me as if Crissis born from an mindcontrolled zombie will join XCOM.

 

I need try it out *away for several experiments on mindcontrol*

 

p.s. does it work for tftd

[Bomb Bloke]

'Harsh'?

[Zombie]

By "harsh" I think that Faskalia means "intense". This discussion is diving pretty deep into the intricacies of the game for a novice reader. We are also veering off the subject of civilians somewhat. Perhaps a new thread should be started? Just a thought.

 

 

Faskalia:

In short, yes it is possible to "force" Chryssalids to aid X-COM.

 

Let me try to explain this (paraphrasing NKF). First, move one of your soldiers (say, Don Nash) to a Chryssalid. Allegiance of Don is initially set to X-COM. Now, after the Chryssalid has turned Don into a Zombie, his allegiance is set to alien. After mind controlling Don the Zombie his allegiance is returned back to X-COM. Pretty simple so far.

 

The game also sets a flag saying that the unit Don the zombie is under mind control. Now, shoot Don the zombie until he turns into a Chryssalid. Here is where things start to get tricky. The allegiance of the newly formed Don the Chryssalid is restored to alien status. However, the game never erases the flag data which says that Don-the-Zombie-now-turned-Chryssalid is under mind control. So after the round is over, Don the Chryssalid is permanently under X-COM mind control! Are your concerns silenced? (Heh, I rephrased NKF's reply so well that even I understand it now!)

 

----------

 

Pumpkinhead:

The upper number is 16, not 17 as you mentioned. Anyhow, the average is based around 12 except that the probability of 0-7 civilians is reduced severely from the anticipated bell-shaped Gaussian curve. According to my data, the probability of having 0-7 civilians is about 4% vs the nearly 50% predicted with a Gaussian distribution having an average of 8! That is a big difference.

 

I do not know whether the civilian counts are based off the tiles used to generate a map. If I were to hazard a guess, I'd say no. Why? The game would be a complete mess if it generated civilians based off tiles where the distribution is wild such as my data suggests. I would expect the game to use a simple Gaussian algorithm to determine which tiles are used to generate the map. This would result in a smooth and ordered method of generating tiles without referring to a distorted Gaussian subset which would gobble up processing power and ultimately slow down map production.

 

In the end, the game probably just generates a random number where the overall average would be 12, but imposes a secondary restriction saying that if the number is below 8 it must have a distribution of around 4%. Seems simple to me...

[Hobbes]

So I guess any of these four bytes might map to the PCK file for image data - byte 1 ask for x floor, byte 2 asks for x wall, byte 3 asks for another wall, and byte 4 asks for something else (furniture or smoke, etc).

(Presuming I can find where the stats for stuff like stairs and the like are kept, as opposed to the image data.  )

From my experiences in map editing this is how it seems to go:

 

The 'tiles' don't refer to map locations but to .mcd entries. Each location on the map can have 4 tiles: ground, west wall, north wall and content (objects). The data on each of the 4 bytes tells the game engine which .mcd entry should be placed there, thus each map location (x, y and level) can have 4 entries assigned to it.

When the map is being processed the mcd entries are loaded from the .mcd files and combined, forming a single 256 entry list. The value of each byte tells the engine which 'entry' from this list should be used on that tile.

The stats for stuff like chairs are from on the .mcd files: amongst other things they contain the information about which .pck image goes with that entry, what type of tile it is (ground, walls, etc.), and other physical characteristics, like its shape when calculating line of sight, resistance to explosions, etc. There's a post by BladeFireLight in the Modders section that details how the data for each entry is organized.

 

Ok, so lemme get this straight. As far as I can see, a tile could have up to six walls.

 

Each map location can only have three 'walls': ground, north and west wall. What happens is that a north wall (as an example) serves as the south wall for the location above it. This simplifies map construction since you don't have to use two tiles for a wall but also brings a limitation: there can't be any walls on the lower edges of the map.

In some cases there aren't walls at all (like the cockpit of the Avenger) but only a 'content' tile, to make map generation more easy. In those cases the data of the mcd entry is changed to give it 'walls' and 'ground' characteristics as well.

 

To Zombie:

First, congratulations on the statistical work you have done. The probability of a 0 civilian site sounds very accurate: When you first started compiling the data I had my doubts about it because of the probability of their occurance but a 0.04% sounds much better

Your guess about the map tiles not being used to generate the civilians is correct. The engine uses the .rmp files that go with each map to generate the civilians. Each map has a series of route 'nodes' that are used to spawn units (alien, X-Com and civilian) and to set patrol routes for the aliens. Like with the mcd entries, the engine seems to make a list of all nodes in the entire battlefield, with a 255 limit.

Each node contains a series of settings, like its map location, type of unit and other information. It also contains a flag that can be set between 0 (no spawn) or a value between 1 and 10 (spawn). The range of the flag values probably indicate the probability of an alien being spawned there, although there is no confirmation of it.

 

All of this information is available on the help file of Daishiva's Map View. And if you want to know more the best person to ask is Blade, since he wrote the help file.

[Bomb Bloke]

I now know nearly everything I need to start writing my logging program.

[Zombie]

I was just doing a little testing today on something which bothered me for a very long time. "Which unit will an alien prefer to attack first: an X-COM soldier, or a civilian"? So I set up a fairly simple testing scenario. Went to a Terror Site, MC'd one Sectoid Navigator, killed off the rest of the aliens, and then corralled a civilian in the room upstairs from the small house to keep it from wandering off. Then I sent my troops in to clear the furniture. After that, I sent the Sectoid upstairs to stand with its back against the east wall and arranged my test soldier one square away from the civilian on the west wall. (Imagine a "T": X-COM soldier stands on the left arm of the "T", the civilian stands at the right arm, and the Sectoid stands at the bottom).

 

Ok, this isn't a perfect testing scenario as I had to manually edit the civilian to be rooted to one spot so it didn't move, but it should suffice. After running a bunch of trials, it appears that the primary target for the aliens is X-COM units, while the secondary targets are civilians. To verify, I moved my test soldier to stand against the same wall the Sectoid was at. When it was the aliens turn, the Sectoid ignored the helpless civilian (which was in its direct line-of-sight), turned, and then shot my soldier. I did this multiple times and X-COM units always seem to be first in the pecking order.

 

I guess we can use this to our advantage. If you are really a gallant player, send your troops to draw fire away from the civilians. Also, the difference between a live and dead civie is a 60 point shift, so keeping them alive by any means possible is helpful. Rookies are -20 points if killed so they are a perfect choice for cannon fodder as the difference will be 10 points in your favor if the civilian is kept alive. Still, who cares about civilians?

 

- Zombie