23 replies to this topic

[magic9mushroom]

Okay, so I have a save in the DOS version of UFO on May 31 with six Hyperwave Decoders on six continents. I was wondering about the way the game picks a zone for Alien Terror missions, and when Zombie mentioned ZONAL.DAT I figured I'd hack it and then see what showed up on June 1. What I found I've described elsewhere, but I noticed something else - an Ethereal Small Scout on Alien Retaliation at 02:30. This I found a bit odd, since I hadn't fought any Ethereals yet (as you'd expect). After getting a clean version of the save I reloaded a few more times, and I kept getting an Alien Retaliation mission sent to the same zone (North America), but the race was random.

"Ah-ha", I thought. "I have finally tracked down the 'scheduled' Retaliation missions. Maybe these are where the Alien Appearance Ratio values for Retaliation are used!" So, after making a backup, I played through to July 31 and saw the ratios coming up on August 1 (since the Appearance Ratios for June-July are supposed to be 20% for each, it's a bit hard to recognise whether they're there or not).

So then, for the August 1 scheduled Retaliation at 02:30 with a random race (there was one in July too, but I didn't bother testing it), I got:

10 Sectoid
15 Floater
8 Snakeman
9 Muton
8 Ethereal

in my 50 trials.

"Uh-oh," I thought, "that doesn't look like the 1/2/1/2/4 ratio given for August on UFOpaedia at all. That looks more like a flat distribution, or maybe a slight bias to Floater."

Along the way, I noticed that the Alien Terror missions in August (which I'd been seeing thanks to the Medium Scout also showing up at 02:30 on the 1st) seemed to have a lot of Ethereals. With dawning horror, I went back to my save in May and did some more testing on the Alien Terror Medium Scout that showed up on June 1 (for the June Terror Site).

5 Sectoid
4 Floater
16 Snakeman
0 Muton
0 Ethereal

in 25 trials.

That's a pretty clear 1:1:3:0:0. I.E., the ratio listed as April/May on UFOpaedia.

So:

1) Scheduled Retaliation, not generated by an interception and with a randomised race, is real. It happened 100% of the time in June/July/August, the first UFO always showed up at 02:30 on the 1st, simultaneous with the Alien Terror Medium Scout, but unlike the Alien Terror always seemed to target a particular zone each month (reloading changed the race of both, and the zone for Terror, but not the zone for Retaliation; I'm 100% sure about this given the 50+ trials). They don't appear to count as the Alien Mission for the month, since I was also getting another non-Terror mission (I vaguely recall getting two others at one point).

What I don't know: when or why "scheduled" Retaliations start, what controls the zone they target, what controls their race (I know Ethereals in June are possible and that August doesn't show an excess of Ethereals; that's it).

2) The Alien Terror mission for June appears to use what UFOpaedia calls the April/May ratio. I haven't tested July or August rigorously.


I am now very confused, and I think we may not know as much as the UFOpaedia suggests we do.

[Space Voyager]

The game still has its mysteries, huh? Go X-COM!1!1!11!11

[Hobbes]

Space Voyager, on 10 July 2018 - 09:30 AM, said:

The game still has its mysteries, huh? Go X-COM!1!1!11!11

Due to OpenXCom there isn't much of a mystery there

You can see all the major aspects regarding how alien missions are generated here: https://www.ufopaedi...nown_(OpenXcom)

magic9mushroom, on 09 July 2018 - 06:20 AM, said:

What I don't know: when or why "scheduled" Retaliations start, what controls the zone they target, what controls their race (I know Ethereals in June are possible and that August doesn't show an excess of Ethereals; that's it)

.

* There are 2 independent triggers for the scheduled Retaliation missions: November 1999 on Superhuman (lower difficulties mean later activations) or researching The Martian Solution
* The scheduled alien Retaliation always happens in a region with an XCom base (100%)
* Race is random: 20% chance for every race, regardless of the month. In theory you can get a scheduled retaliation mission consisting of Ethereals in February if you are able to research The Martian Solution until the end of January (and it will take place in the region of your starting base!)

Quote

I am now very confused, and I think we may not know as much as the UFOpaedia suggests we do.

The page dealing with alien missions on the Enemy Unknown section of the UFOPaedia is a good effort, but it was based in empirical observation made by players 15 years ago without knowing how the original engine actually worked. I was the original author of the page detailing how it works in OpenXCom, which reverse-engineered the original code.

[magic9mushroom]

Hobbes, on 11 July 2018 - 12:28 AM, said:

You can see all the major aspects regarding how alien missions are generated here: https://www.ufopaedi...nown_(OpenXcom)

That page includes incorrect information - I just went through June and there's only one alien mission (disregarding ongoing missions from May).

Quote

* There are 2 independent triggers for the scheduled Retaliation missions: November 1999 on Superhuman (lower difficulties mean later activations) or researching The Martian Solution
* The scheduled alien Retaliation always happens in a region with an XCom base (100%)
* Race is random: 20% chance for every race, regardless of the month. In theory you can get a scheduled retaliation mission consisting of Ethereals in February if you are able to research The Martian Solution until the end of January (and it will take place in the region of your starting base!)

This I'll believe; it matches up with my observations. However, I think there's more going on than simply being limited to a region with an X-Com base; which X-Com base is targetted does not appear to be randomised the same way other Alien Missions are.

[Hobbes]

magic9mushroom, on 11 July 2018 - 05:22 AM, said:

That page includes incorrect information - I just went through June and there's only one alien mission (disregarding ongoing missions from May).

Correct, I forgot that month 0 corresponds to January, so 2 missions start in July (month 6)

Quote

This I'll believe; it matches up with my observations. However, I think there's more going on than simply being limited to a region with an X-Com base; which X-Com base is targetted does not appear to be randomised the same way other Alien Missions are.

Because there's regional weights also involved when selecting regions with XCom bases for Retaliation missions?

[magic9mushroom]

Hobbes, on 11 July 2018 - 11:17 AM, said:

Because there's regional weights also involved when selecting regions with XCom bases?

Not what I meant.

When I reload that May 31 save, the location and race of the Terror mission for June get rerolled (presumably at 0:00 June 1). The location and type of the non-Terror mission, too (presumably also race but I forget the exact particulars). But NOT the location of the scheduled Retaliation. The race changes, but it targets North America 100% of the time. Which means either it's not rolled (and is set deterministically somehow) or had already been rolled prior to the save.

[Hobbes]

magic9mushroom, on 11 July 2018 - 11:28 AM, said:

Not what I meant.

When I reload that May 31 save, the location and race of the Terror mission for June get rerolled (presumably at 0:00 June 1). The location and type of the non-Terror mission, too (presumably also race but I forget the exact particulars). But NOT the location of the scheduled Retaliation. The race changes, but it targets North America 100% of the time. Which means either it's not rolled (and is set deterministically somehow) or had already been rolled prior to the save.

The default region is set for North America when determining mission regions, so that might explain it. I can bring this up with Warboy for him to check the original code to see what might be causing your observed behavior.

Just a couple of questions:
* When you mention June, you mean 1999 or 2000? (1999 means you researched The Martian Solution)
* Is North America the location of your first base?

[magic9mushroom]

Hobbes, on 11 July 2018 - 12:15 PM, said:

The default region is set for North America when determining mission regions, so that might explain it. I can bring this up with Warboy for him to check the original code to see what might be causing your observed behavior.

Just a couple of questions:
* When you mention June, you mean 1999 or 2000? (1999 means you researched The Martian Solution)
* Is North America the location of your first base?

June 1999. I had, indeed, researched The Martian Solution (it's a save left over from a June 1 Cydonia run).

My first base was Europe. North America was my second.

[Hobbes]

Just mentioned this to Warboy on Discord and here's the original game code

if ( v_GS_time_year != 1999 || v_GS_time_month )// not at game start
  {
	xc_GS__Setup_Alien_Mission();
	if ( v_GS_Elapsed_Months_count > 5 )
	  xc_GS__Setup_Alien_Mission();
	if ( v_GS_Elapsed_Months_count >= 14 - v_DIFFICULTY_LEVEL || a_GS_Research_Builtin[57].researched2 & 1 )
	{
	  v2 = p_GS_Loc_DAT;
	  v1 = 0;
	  while ( 1 )
	  {
		if ( v2->obj_type == lot_XCom_Base )
		{
		  v3 = xc_GS_Get_RegionID_from_LatLon(v2->cur_x, v2->cur_y);
		  if ( p_GS_Missions_DAT[7 * v3 + 6].UFO_Counter == -1 )
			break;
		}
		++v1;
		++v2;
		if ( (_WORD)v1 >= 50 )
		  goto LABEL_13;
	  }
	  v4 = 56 * v3;							 // retaliation
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].UFO_Counter + v4) = 0;
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].UFO_SubCounter + v4) = 0;
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].MissionTimer + v4) = 5;// 2.5 hours
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].MissionAlienRace + v4) = xc_RandMod(4);
	}
  }

According to him, the original game is simply selecting the first base in loc.dat and sending retaliation there, assuming there's not a retaliation there already. Which makes sense, if you consider that this is a way of ensuring that the player gets hit where it hurts (starting base) when the scheduled retaliations start.

[magic9mushroom]

Hobbes, on 11 July 2018 - 12:36 PM, said:

Just mentioned this to Warboy on Discord and here's the original game code

if ( v_GS_time_year != 1999 || v_GS_time_month )// not at game start
  {
	xc_GS__Setup_Alien_Mission();
	if ( v_GS_Elapsed_Months_count > 5 )
	  xc_GS__Setup_Alien_Mission();
	if ( v_GS_Elapsed_Months_count >= 14 - v_DIFFICULTY_LEVEL || a_GS_Research_Builtin[57].researched2 & 1 )
	{
	  v2 = p_GS_Loc_DAT;
	  v1 = 0;
	  while ( 1 )
	  {
		if ( v2->obj_type == lot_XCom_Base )
		{
		  v3 = xc_GS_Get_RegionID_from_LatLon(v2->cur_x, v2->cur_y);
		  if ( p_GS_Missions_DAT[7 * v3 + 6].UFO_Counter == -1 )
			break;
		}
		++v1;
		++v2;
		if ( (_WORD)v1 >= 50 )
		  goto LABEL_13;
	  }
	  v4 = 56 * v3;							 // retaliation
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].UFO_Counter + v4) = 0;
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].UFO_SubCounter + v4) = 0;
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].MissionTimer + v4) = 5;// 2.5 hours
	  *(__int16 *)((char *)&p_GS_Missions_DAT[6].MissionAlienRace + v4) = xc_RandMod(4);
	}
  }

According to him, the original game is simply selecting the first base in loc.dat and sending retaliation there, assuming there's not a retaliation there already. Which makes sense, if you consider that this is a way of ensuring that the player gets hit where it hurts (starting base) when the scheduled retaliations start.

That would explain it. First base was Europe, and there was a (triggered) Retaliation in progress against it, so it went after the second.

Looks like I've got some serious work to do on UFOpaedia.

[Hobbes]

magic9mushroom, on 11 July 2018 - 12:45 PM, said:

That would explain it. First base was Europe, and there was a (triggered) Retaliation in progress against it, so it went after the second.

Looks like I've got some serious work to do on UFOpaedia.

And actually this discussion just revealed a difference in behavior between the original game and OpenXCom - in OXC the region for scheduled retaliation it is randomly selected - Warboy didn't notice this quirk on the original game (first base is always selected if there isn't a retaliation there yet) and decide to make it all random when implementing it.

This is the reason why I never updated the UFOPaedia page on the original alien missions using the info from OXC and decided to make a separate page instead describing how it works on OXC - OXC tries to be as faithful as possible to the original game but there are slight differences.

[magic9mushroom]

Speaking of Warboy, would you mind asking him about the code dealing with Alien Appearance Ratios?

I'm positive the Terror mission for June is with the ratios that UFOpaedia currently claims are April/May (1:1:3 Sectoid:Floater:Snakeman), but it's rather hard to test ratios quickly (particularly for non-Terror missions).

[Hobbes]

magic9mushroom, on 14 July 2018 - 10:54 AM, said:

Speaking of Warboy, would you mind asking him about the code dealing with Alien Appearance Ratios?

I'm positive the Terror mission for June is with the ratios that UFOpaedia currently claims are April/May (1:1:3 Sectoid:Floater:Snakeman), but it's rather hard to test ratios quickly (particularly for non-Terror missions).

This is the OpenXCom ruleset for Terror missions  

- type: STR_ALIEN_TERROR
points: 10
objective: 3
spawnZone: 3 #Mission zone for terror missions
raceWeights:
0:
STR_SECTOID: 30
STR_FLOATER: 70
1:
STR_SECTOID: 60
STR_FLOATER: 40
3:
STR_SECTOID: 20
STR_SNAKEMAN: 60
STR_FLOATER: 20
5:
STR_SECTOID: 10
STR_SNAKEMAN: 20
STR_ETHEREAL: 40
STR_MUTON: 20
STR_FLOATER: 10
7:
STR_SECTOID: 20
STR_SNAKEMAN: 30
STR_ETHEREAL: 20
STR_MUTON: 20
STR_FLOATER: 10

If the June Terror mission used the April settings then you'd never see Mutons or Ethereals in June terror missions.

[magic9mushroom]

Hobbes, on 18 July 2018 - 03:37 PM, said:

If the June Terror mission used the April settings then you'd never see Mutons or Ethereals in June terror missions.

Exactly. I don't. Only Sectoids/Floaters/Snakemen. The Alien Terror mission for June is using the third set of race weights (20/60/0/0/20), which have up until now been called "April/May". From that code it looks like OpenXCom is putting Mutons/Ethereals in June Terror, which is not in accordance with my experience. I can confirm that July does use the fourth set (10/20/40/20/10) but I don't know anything about earlier or later months, or non-Terror missions.

What I was looking for is a code dig on what set of values it calls for which month in the original .exe. I'm not capable of doing that myself; I can check values at a specified position but I have no idea how to disassemble or read executable code.

[NKF]

A while back I ran a test and edited the appearance table with a hex editor to make the races homogeneous so that they would generate just one alien race for all missions except in the month pairing I wanted to test.

My findings were that the new race would not show up immediately, but rather started to show up as the month progressed. Once the test period ended and the next month pairing started, the test race continued to show up for a while, and then ceased completely.

So what you're probably seeing is the remainder of what was scheduled in the April/May block. In comparison, I see in OpenXcom that a newly introduced race can show up right away. Ethereals as the first encounter in June for example.

I think I was testing the often mentioned assertion that once a region is targeted for infiltration that it never ends. I didn't really pursue this experiment very far though, as this is the extent of my findings.

- NKF

[magic9mushroom]

NKF, on 19 July 2018 - 05:45 AM, said:

A while back I ran a test and edited the appearance table with a hex editor to make the races homogeneous so that they would generate just one alien race for all missions except in the month pairing I wanted to test.

My findings were that the new race would not show up immediately, but rather started to show up as the month progressed. Once the test period ended and the next month pairing started, the test race continued to show up for a while, and then ceased completely.

So what you're probably seeing is the remainder of what was scheduled in the April/May block. In comparison, I see in OpenXcom that a newly introduced race can show up right away. Ethereals as the first encounter in June for example.

I think I was testing the often mentioned assertion that once a region is targeted for infiltration that it never ends. I didn't really pursue this experiment very far though, as this is the extent of my findings.

- NKF

No. It's not an overrun. I know what an overrun looks like. Alien Terror starts with a Medium Scout at 02:30 on the 1st like clockwork (unlike most missions, which take a week or so to send the first scout), and the race isn't set yet in the save on the 31st; it's a roll for June.

(The Terror Site itself can overrun, but what I'm reporting seeing is the first Medium Scout, with a Hyperwave Decoder telling me the race and that it's on Alien Terror.)

[Hobbes]

magic9mushroom, on 19 July 2018 - 03:54 AM, said:

What I was looking for is a code dig on what set of values it calls for which month in the original .exe. I'm not capable of doing that myself; I can check values at a specified position but I have no idea how to disassemble or read executable code.

Today I asked Warboy to check it out and you just found a major difference on what UFOPaedia.org describes and the actual game behavior

General missions code

if ( v_GS_Elapsed_Months_count >= 1 )
	 v23 = 1;
if ( v_GS_Elapsed_Months_count >= 3 )
	 v23 = 2;
if ( v_GS_Elapsed_Months_count >= 6 )
	 v23 = 3;
if ( v_GS_Elapsed_Months_count >= 9 )
	 v23 = 4;
p_GS_Missions_DAT[acts_offset].MissionAlienRace = a_mission_race_0[miss_num_ + 7 * v23].race[xc_RandMod(9)];

Terror mission code (handled on the Council Monthly screen)

LABEL_13:
v5 = 0;
if ( v_GS_Elapsed_Months_count >= 1 )
v5 = 1;
if ( v_GS_Elapsed_Months_count >= 3 )
v5 = 2;
if ( v_GS_Elapsed_Months_count >= 6 )
v5 = 3;
if ( v_GS_Elapsed_Months_count >= 9 )
v5 = 4;
v8 = xc_RandMod(9);
v9 = 7 * v5;
v7 = 0;
v6 = a_mission_race_0_5[v9].race[v8];

Whomever wrote the page on alien missions on the UFOPedia assumed that there was a 2 month interval between the introduction of new races when instead it's 3 (which fits with your observations of no Mutons/Ethereals on June). When OpenXCom was implemented, the 2 month interval was used incorrectly and Warboy just fixed it on the latest nightly version.

This is the a_mission_race_0 array that is used to select the race (00 = sectoid, 01 = snek, 02= ethereal, 03 = muton, 04 = floater)

00 00 00 00 00 00 00 01 04 04 00 00 00 00 00 01 01 04 04 04 00 00 00 00 00 00 04 04 04 04 00 00 00 00 00 00 01 01 04 04 00 00 00 01 01 04 04 04 04 04 00 00 00 04 04 04 04 04 04 04 00 00 00 00 00 00 01 01 04 04 00 00 00 00 00 00 01 04 04 04 00 00 00 00 00 04 04 04 04 04 00 00 00 00 04 04 04 04 04 04 00 00 00 00 00 01 01 04 04 04 00 00 00 00 01 01 04 04 04 04 00 00 00 00 00 00 04 04 04 04 00 00 00 01 01 01 04 04 04 04 00 00 00 01 03 03 03 04 04 04 00 00 00 03 03 03 04 04 04 04 00 00 00 00 00 04 04 04 04 04 00 00 01 01 03 03 03 03 03 04 00 01 01 01 01 03 03 03 04 04 00 00 01 01 01 01 01 01 04 04 00 00 01 01 03 03 03 03 04 04 00 00 01 03 03 03 03 04 04 04 00 03 03 03 03 04 04 04 04 04 00 00 00 04 04 04 04 04 04 04 00 01 02 02 02 02 03 03 03 04 00 01 01 01 01 02 03 03 04 04 00 01 01 02 02 02 02 03 03 04 00 00 01 01 02 02 03 03 04 04 00 00 00 01 03 03 03 04 04 04 00 00 00 03 03 03 04 04 04 04 00 00 00 00 00 04 04 04 04 04 00 00 01 01 02 03 03 03 03 04 00 01 01 02 02 02 03 03 02 04 00 00 01 01 01 02 02 03 03 04 00 01 02 02 02 02 03 03 04 04

[magic9mushroom]

So, the blocks are:

January
February-March
April-June
July-September
October and onward

Am I right?

[Hobbes]

magic9mushroom, on 20 July 2018 - 02:34 AM, said:

So, the blocks are:

January
February-March
April-June
July-September
October and onward

Am I right?

That is correct. It has already been fixed on the latest nightly OpenXcom build by Warboy.

[magic9mushroom]

Hobbes, on 20 July 2018 - 02:41 AM, said:

That is correct. It has already been fixed on the latest nightly OpenXcom build by Warboy.

I noticed. Would be nice if he'd credited me too (I did some work here), but oh well.