MTR Psi Testing - Revision history

Hobbes: categorizing

2013-07-23T03:08:18Z

categorizing

← Older revision		Revision as of 03:08, 23 July 2013
Line 691:		Line 691:

	:I haven't had a chance to scrutinize all of this yet, but it looks great. I'm glad to see my original testing (and formula) has held up -- it was just missing a term, the slope (and the distance, which I'm glad has proven to be a simple term). I'll be giving it a close look in the coming week. Nice work.--[[User:Ethereal Cereal\|Ethereal Cereal]] 20:45, 10 September 2006 (PDT)		:I haven't had a chance to scrutinize all of this yet, but it looks great. I'm glad to see my original testing (and formula) has held up -- it was just missing a term, the slope (and the distance, which I'm glad has proven to be a simple term). I'll be giving it a close look in the coming week. Nice work.--[[User:Ethereal Cereal\|Ethereal Cereal]] 20:45, 10 September 2006 (PDT)

			[[Category:Enemy Unknown/UFO Defense]]

Zombie: /* Background */ Relinking.

2009-01-02T02:57:11Z

Background: Relinking.

← Older revision		Revision as of 02:57, 2 January 2009
Line 5:		Line 5:
	= Background =		= Background =

	The equations governing [[Psionics\|psionic]] success have long been a subject of mystery and debate. If they were better understood, players could compare versus [[Alien_Stats\|alien stats]] and know exactly how much [[Psionic Strength]] and [[~~Psi~~ Skill]] is needed, both as a minimum, and where one is maxxed out versus all aliens.		The equations governing [[Psionics\|psionic]] success have long been a subject of mystery and debate. If they were better understood, players could compare versus [[Alien_Stats\|alien stats]] and know exactly how much [[Psionic Strength]] and [[Psionic Skill]] is needed, both as a minimum, and where one is maxxed out versus all aliens.

	The Official Strategy Guide supplies psionic equations, but while they are very intriguing, they make no mathematical sense, as if the math operators are typo'd or something.		The Official Strategy Guide supplies psionic equations, but while they are very intriguing, they make no mathematical sense, as if the math operators are typo'd or something.

Zombie: /* Background */ Relinking.

2009-01-02T02:35:11Z

Background: Relinking.

← Older revision		Revision as of 02:35, 2 January 2009
Line 5:		Line 5:
	= Background =		= Background =

	The equations governing [[Psionics\|psionic]] success have long been a subject of mystery and debate. If they were better understood, players could compare versus [[Alien_Stats\|alien stats]] and know exactly how much [[~~Psi~~ Strength]] and [[Psi Skill]] is needed, both as a minimum, and where one is maxxed out versus all aliens.		The equations governing [[Psionics\|psionic]] success have long been a subject of mystery and debate. If they were better understood, players could compare versus [[Alien_Stats\|alien stats]] and know exactly how much [[Psionic Strength]] and [[Psi Skill]] is needed, both as a minimum, and where one is maxxed out versus all aliens.

	The Official Strategy Guide supplies psionic equations, but while they are very intriguing, they make no mathematical sense, as if the math operators are typo'd or something.		The Official Strategy Guide supplies psionic equations, but while they are very intriguing, they make no mathematical sense, as if the math operators are typo'd or something.

Zombie: /* Background */ Redirecting.

2007-02-01T22:34:25Z

Background: Redirecting.

← Older revision		Revision as of 22:34, 1 February 2007
Line 5:		Line 5:
	= Background =		= Background =

	The equations governing [[Psionics\|psionic]] success have long been a subject of mystery and debate. If they were better understood, players could compare versus [[~~Alien_Stats_%28UFO_Defense%29~~\|alien stats]] and know exactly how much [[Psi Strength]] and [[Psi Skill]] is needed, both as a minimum, and where one is maxxed out versus all aliens.		The equations governing [[Psionics\|psionic]] success have long been a subject of mystery and debate. If they were better understood, players could compare versus [[Alien_Stats\|alien stats]] and know exactly how much [[Psi Strength]] and [[Psi Skill]] is needed, both as a minimum, and where one is maxxed out versus all aliens.

	The Official Strategy Guide supplies psionic equations, but while they are very intriguing, they make no mathematical sense, as if the math operators are typo'd or something.		The Official Strategy Guide supplies psionic equations, but while they are very intriguing, they make no mathematical sense, as if the math operators are typo'd or something.

MikeTheRed: /* Some notes on Panicking */ Added link to Berserk data

2006-10-01T15:37:19Z

Some notes on Panicking: Added link to Berserk data

← Older revision		Revision as of 15:37, 1 October 2006
Line 289:		Line 289:

	While doing testing for 5b, I made notes about how often Berserk or Panic was seen. By the third turn, every single muton was freaking out; only once in 10 turns did one of the 16 mutons not freak. (I tested for 12 turns, but started counting Berserk rates on the third turn.) So they were undoubtedly pegged at [[Morale]] 0, or close to it. A check of Unitref[58] after ending the last test turn showed all 16 mutons at Morale 15, as expected after one "recovery" freakout. There were 55 Berserks out of the 159 freakouts, for a rate of 34.6% Berserk versus Panic.		While doing testing for 5b, I made notes about how often Berserk or Panic was seen. By the third turn, every single muton was freaking out; only once in 10 turns did one of the 16 mutons not freak. (I tested for 12 turns, but started counting Berserk rates on the third turn.) So they were undoubtedly pegged at [[Morale]] 0, or close to it. A check of Unitref[58] after ending the last test turn showed all 16 mutons at Morale 15, as expected after one "recovery" freakout. There were 55 Berserks out of the 159 freakouts, for a rate of 34.6% Berserk versus Panic.

			:''For more on Berserk versus Panic percents, I summarized all my data on the [[Berserk]] page

	If 36.09% is the correct Panic percent from test 5b, the chance of at least one successful psi Panic attack being performed by soldiers doing 10 attacks is:		If 36.09% is the correct Panic percent from test 5b, the chance of at least one successful psi Panic attack being performed by soldiers doing 10 attacks is:

MikeTheRed at 05:26, 30 September 2006

2006-09-30T05:26:56Z

← Older revision		Revision as of 05:26, 30 September 2006
Line 642:		Line 642:

	For what it's worth, that old highly-tested datapoint of mine, 95/16 vs. 25/0 producing 49.26% MC successes (1192/2420), should actually have been 51.79%. The N is high enough that this may be a real difference: 51.79 - 49.26 = 2.53%, which is about 61 extra successes I should have seen with 2420 attempts. This means about 1 in 20 of expected successes weren't seen (61/(1192+61) = 4.89% ≈ 1/20. I can only imagine that the extreme tedium of those tests plus the fact I was manually tracking it at that point must have led to quite a few errors in recording. Or maybe it was simply due to chance.		For what it's worth, that old highly-tested datapoint of mine, 95/16 vs. 25/0 producing 49.26% MC successes (1192/2420), should actually have been 51.79%. The N is high enough that this may be a real difference: 51.79 - 49.26 = 2.53%, which is about 61 extra successes I should have seen with 2420 attempts. This means about 1 in 20 of expected successes weren't seen (61/(1192+61) = 4.89% ≈ 1/20. I can only imagine that the extreme tedium of those tests plus the fact I was manually tracking it at that point must have led to quite a few errors in recording. Or maybe it was simply due to chance.

	~~= Future Work =~~

	~~Future work that I (or anyone else) might do could include:~~
	*Further testing of 190/250, to look for a failed MC (Test 8b).
	*Finer tuning of the low ends and high ends. For example, if you vary the Defense Skill by changing psi skill by 1, it changes the DS by 0.2. Would this lead to a slight refinement (change) of the Constant? (At the decimal level.) Also compare the indented paragraph at my Test 6c.

	~~Still, all in all, the equations are simple and robust, and perfectly fine for gauging the effectiveness of particular psi levels versus aliens, and vice versa.~~

	= Psi Testing Tips =		= Psi Testing Tips =
Line 674:		Line 666:
	#If you'll do lots of testing, have your initial savegame saved when the turn counter is ''n''1 (e.g., 21). Then just subtract 20 to see the number of turns. In this example, you will have to perform testing for X-COM turn 30 to get an even 10 turns.		#If you'll do lots of testing, have your initial savegame saved when the turn counter is ''n''1 (e.g., 21). Then just subtract 20 to see the number of turns. In this example, you will have to perform testing for X-COM turn 30 to get an even 10 turns.
	#If you put numbers in the name of your soldiers, it's easier to keep notes on them. If you also do it for your targets, then put e.g. Soldier 1 next to Alien 1, you can do "1 on 1" testing, such as pairing specific psi-value sets (AS and DS) or, e.g., comparing successful panic attack counts to the targetted alien's [[Morale]] (to verify that your counters are working right).		#If you put numbers in the name of your soldiers, it's easier to keep notes on them. If you also do it for your targets, then put e.g. Soldier 1 next to Alien 1, you can do "1 on 1" testing, such as pairing specific psi-value sets (AS and DS) or, e.g., comparing successful panic attack counts to the targetted alien's [[Morale]] (to verify that your counters are working right).
	#Avoid having elevator/lifts or other objects continuously cycling their graphics near the place you're doing testing. They will slow down the game a small but noticable extent. Also known as, when you're doing ten panics in a row, each panic takes a little bit longer to do its little graphic... I would guess that it makes each attack about 30% longer. Since each attack only takes a second, this only matters to folks doing hundreds or thousands of psi attacks.		#Avoid having elevator/lifts or other objects continuously cycling their graphics near the place you're doing testing. They will slow down the game a small but noticable extent, if they can also currently be seen on the screen. Also known as, when you're doing ten panics in a row, each panic takes a little bit longer to do its little graphic... I would guess that it makes each attack about 30% longer. Since each attack only takes a second, this only matters to folks doing hundreds or thousands of psi attacks.
	#Don't hold weapons, so your crew won't reaction-fire. If you might be MCed, drop all weapons (of course).		#Don't hold weapons, so your crew won't reaction-fire. If you might be MCed, drop all weapons (of course).

MikeTheRed: /* 9e: Final Line Tuning */ - It's soup yet! (but have I been talking to myself?)

2006-09-30T05:09:44Z

9e: Final Line Tuning: - It's soup yet! (but have I been talking to myself?)

@@ Line 558: / Line 558: @@
 And that's enough of that.
-== 9e: Final Line Tuning ==
+== Hmm ==
 [[Image:PsiTest9LineEquation.gif|thumb|right|250px|Graph of Well-Tested Line (Test 9a). Note how it is shifted to the "right" by approximately 0.1 AS.]]Many things have been revealed, and much is known about the psi equations. One final thing that continues to bother me is that results repeatedly seem slightly off from what is predicted.
@@ Line 579: / Line 579: @@
 In the meantime, the psi equations are in great shape. Maybe I'll be able to figure out a little more precision, but ultimately that's all it will be.
 = Psi Equation Finalized =

MikeTheRed: /* Psi Testing Tips */

2006-09-23T02:43:43Z

Psi Testing Tips

← Older revision		Revision as of 02:43, 23 September 2006
Line 647:		Line 647:
	#If you'll do lots of testing, have your initial savegame saved when the turn counter is ''n''1 (e.g., 21). Then just subtract 20 to see the number of turns. In this example, you will have to perform testing for X-COM turn 30 to get an even 10 turns.		#If you'll do lots of testing, have your initial savegame saved when the turn counter is ''n''1 (e.g., 21). Then just subtract 20 to see the number of turns. In this example, you will have to perform testing for X-COM turn 30 to get an even 10 turns.
	#If you put numbers in the name of your soldiers, it's easier to keep notes on them. If you also do it for your targets, then put e.g. Soldier 1 next to Alien 1, you can do "1 on 1" testing, such as pairing specific psi-value sets (AS and DS) or, e.g., comparing successful panic attack counts to the targetted alien's [[Morale]] (to verify that your counters are working right).		#If you put numbers in the name of your soldiers, it's easier to keep notes on them. If you also do it for your targets, then put e.g. Soldier 1 next to Alien 1, you can do "1 on 1" testing, such as pairing specific psi-value sets (AS and DS) or, e.g., comparing successful panic attack counts to the targetted alien's [[Morale]] (to verify that your counters are working right).
			#Avoid having elevator/lifts or other objects continuously cycling their graphics near the place you're doing testing. They will slow down the game a small but noticable extent. Also known as, when you're doing ten panics in a row, each panic takes a little bit longer to do its little graphic... I would guess that it makes each attack about 30% longer. Since each attack only takes a second, this only matters to folks doing hundreds or thousands of psi attacks.
	#Don't hold weapons, so your crew won't reaction-fire. If you might be MCed, drop all weapons (of course).		#Don't hold weapons, so your crew won't reaction-fire. If you might be MCed, drop all weapons (of course).

MikeTheRed: /* Psi Testing Tips */ another tip

2006-09-23T01:03:22Z

Psi Testing Tips: another tip

← Older revision		Revision as of 01:03, 23 September 2006
Line 642:		Line 642:
	##Keep track of the number of turns you have performed testing. Always make the same number of attempts per turn (or make notes of how many you didn't do, on a soldier by soldier basis, if e.g. successful MCs don't let you make all the attempts you might have).		##Keep track of the number of turns you have performed testing. Always make the same number of attempts per turn (or make notes of how many you didn't do, on a soldier by soldier basis, if e.g. successful MCs don't let you make all the attempts you might have).
	##When testing is done, subtract the number of attempts from U[85], then divide the remainder by 2. This should always give an integer (no decimals) for the number of successes. If it's not an integer, you screwed something up... if many of them are off, you probably have the number of attempts incorrect. If only 1 or 2, those soldiers probably got skipped inadvertently.		##When testing is done, subtract the number of attempts from U[85], then divide the remainder by 2. This should always give an integer (no decimals) for the number of successes. If it's not an integer, you screwed something up... if many of them are off, you probably have the number of attempts incorrect. If only 1 or 2, those soldiers probably got skipped inadvertently.
			##If you are doing very extended testing, the counter will roll over (one byte only goes up to 255). But it's no big deal even if it rolls over several times. (E.g., 9a lasted 25 turns X 10 attempts = 250/soldier; the high group was X 3 for successes and thus 750 at 100% success.) You will know the correct approximate number to within a couple percent as predicted by the equation; just add enough 256's to have it make sense. The widest variation is in the middle of the range (50%); the least, at the edges. Binomial variance is p(1-p)N where ''p'' is the probability; standard deviation of the sample is SQRT(variance/(N-1)). You can convert SDs into confidence intervals using the lookup table at (7) [http://mathworld.wolfram.com/StandardDeviation.html here].
	#Note that the Unitref[85] approach both helps avoid mistakes in writing stuff down, as well as gives a check that you got the number of attempts correct. It also speeds up testing since you don't have to slow down to write. But it does take a little time to read and interpret.		#Note that the Unitref[85] approach both helps avoid mistakes in writing stuff down, as well as gives a check that you got the number of attempts correct. It also speeds up testing since you don't have to slow down to write. But it does take a little time to read and interpret.
	#Put your initial test savegame in one game slot, and save it to another. This way you can backtrack if e.g. you realize you accidentally left some counts in Unitref[85].		#Put your initial test savegame in one game slot, and save it to another. This way you can backtrack if e.g. you realize you accidentally left some counts in Unitref[85].

MikeTheRed: /* Test 9: Fine Tuning */

2006-09-20T22:47:54Z

Test 9: Fine Tuning

← Older revision		Revision as of 22:47, 20 September 2006
Line 550:		Line 550:
	For the record: I highly doubt it's stored, if it is computed up front. 1) Directly editing PST and PSK in Unitref ''does'' change AS and DS, and 2) there's nothing in Unitref unaccounted for that looks anything like this. Whatever it is would (probably) have to be a Word (two bytes) since both AS and DS can be higher than 255 (at least in hacked games), and the psi equation still works fine. Non-hacked soldiers are unlikely to ever get that high, though (a 100/100 soldier is AS 200 and DS 120).		For the record: I highly doubt it's stored, if it is computed up front. 1) Directly editing PST and PSK in Unitref ''does'' change AS and DS, and 2) there's nothing in Unitref unaccounted for that looks anything like this. Whatever it is would (probably) have to be a Word (two bytes) since both AS and DS can be higher than 255 (at least in hacked games), and the psi equation still works fine. Non-hacked soldiers are unlikely to ever get that high, though (a 100/100 soldier is AS 200 and DS 120).

	A repeat test was done, with DS lowered from 45.8 to 45.2 (45/1). This should replicate the above experiment exactly... the DS fraction only has to be the tiniest bit higher than the AS fraction (2.1) to make for the possible 1 (Scenario B) or 0 (Scenario C) result. Findings: 1.46% success (7/480). Confirmation.		A repeat test was done, with DS lowered from 45.8 to 45.2 (45/1). This should replicate the above experiment exactly... the DS fraction only has to be the tiniest bit higher than the AS fraction (2.1) to make for the possible 1 (Scenario B) or 0 (Scenario C) result. Findings: 1.46% success (7/480). Confirmation that individual terms are truncated.

	Readers might wonder if I'm obsessing about the psi equation (smile). But the fact is that with the variability involved, and potential that the equations are actually more complex, any particular conclusion needs more than one test. It could have been a confluence of odd things in the equation, chance that a result came up the way it is, or even plain old mistakes by me. In science, findings are not considered confirmed unless they are replicated. Preferably by an entirely different lab... but I don't see that happening with all these fine points, any time soon. :P		Readers might wonder if I'm obsessing about the psi equation (smile). But the fact is that with the variability involved, and potential that the equations are actually more complex, any particular conclusion needs more than one test. It could have been a confluence of odd things in the equation, chance that a result came up the way it is, or even plain old mistakes by me. In science, findings are not considered confirmed unless they are replicated. Preferably by an entirely different lab... but I don't see that happening with all these fine points, any time soon. :P

	In that spirit, another take on the above scenarios: AS 2.9 (5/29) vs. DS 45.2 (45/1). Deliberately chosen as a counter-example / test expected to fail. (It can't separate B or C, because the AS fraction is higher than DS's.) If B or C is true, the expected rate is 1.79%; if A is true, 3.04%. Results: 2.08% (10/480). ~~It's a farther stretch from these~~ results ~~to A~~ (~0.7) than ~~to B or~~ C (~0.3). Another confirmation that truncation occurs. And that's enough of that.		In that spirit, another take on the above scenarios: AS 2.9 (5/29) vs. DS 45.2 (45/1). Deliberately chosen as a counter-example / test expected to fail. (It can't separate B or C, because the AS fraction is higher than DS's.) If B or C is true, the expected rate is 1.79%; if A is true, 3.04%. Results: 2.08% (10/480). B or C are much closer to the results (a difference of ~0.3%) than C (~0.7%). Another confirmation that truncation (in general) occurs.

			And that's enough of that.

	== 9e: Final Line Tuning ==		== 9e: Final Line Tuning ==
Line 564:		Line 566:
	In many other tests, it was also my impression that other lines were skewed to the right a little. Yet it's hard to be certain, with the variability involved. Further, not all tests were conducted in the same way... for some results something might have been skewed a little to the left (or up or down), but it might (might!) be because the test (and resultant equation) was looking at things a different way. For example, a DS line shifted to the left is the same as AS being shifted to the right.		In many other tests, it was also my impression that other lines were skewed to the right a little. Yet it's hard to be certain, with the variability involved. Further, not all tests were conducted in the same way... for some results something might have been skewed a little to the left (or up or down), but it might (might!) be because the test (and resultant equation) was looking at things a different way. For example, a DS line shifted to the left is the same as AS being shifted to the right.

	Keep in mind that although we've made an "elegant" final equation, it doesn't mean X-COM uses that precise form. It might (might!) already have, e.g., some or all of the slope multiplied into the other numbers, which then leads to a tiny truncation event which adds 0.1. Also notice that shifting the line to the right by 0.1 would be the same as adding 5 to AS prior to dividing by 50, or 0.5 to DS PSK prior to dividing by 5. Finally, the +0.1 to the line raises a flag over how programmers (or mathematicians) sometimes put tiny values into an equation, in order to make sure something's not zero (or whatever). Just how any of these might work into the psi equations (if at all) stumps me for now.		Keep in mind that although we've made an "elegant" final equation, it doesn't mean X-COM uses that precise form. It might (might!) already have, e.g., some or all of the slope multiplied into the other numbers, which then leads to a tiny truncation event which adds 0.1. Also notice that shifting the line to the right by 0.1 would be the same as adding 5 to AS prior to dividing by 50, or 0.5 to DS PSK prior to dividing by 5. Finally, the +0.1 to the line raises a flag over how programmers (or mathematicians) sometimes put tiny values into an equation, in order to make sure something's not zero (or whatever).

			Or for an entirely different take: There might be truncation of the percent success value, and/or it may change a discrete value range within X-COM makes a random roll somehow, with the net effect being a little shifting. If so, it would probably have to be based on percent chance of ''failing'' getting a truncation to move the line to the right, because it causes a slightly increased chance of success.

			Just how any of these might work into the psi equations (if at all) stumps me for now.

	At this point, I'm not certain how to study this more. Simply collecting more data on the Test 9a line doesn't seem particularly productive. It probably won't do anything except lend a tiny bit more precision, at the cost of lengthy testing.		At this point, I'm not certain how to study this more. Simply collecting more data on the Test 9a line doesn't seem particularly productive. It probably won't do anything except lend a tiny bit more precision, at the cost of lengthy testing.